Stanford CME296 Diffusion & Large Vision Models | Spring 2026 | Lecture 8 - Trending Topics

Diffusion predicts noise to reverse corruption

Diffusion models learn a reverse process by minimizing an L2 loss that estimates Gaussian noise added to images, derived from an evidence lower bound (ELBO) on the data likelihood.

Score matching estimates data distribution gradients

Score-based methods compute the gradient of log probability to navigate from noise to data using Langevin dynamics, avoiding intractable normalizing constants while revealing that the score equals negative noise divided by a coefficient.

Flow matching reframes generation as mass transport

Flow matching treats generation as moving probability mass via vector fields (velocities) from a prior to a target distribution, governed by the continuity equation and numerically solved as an ODE.

SDEs unify discrete approaches

Stochastic differential equations generalize discrete noising into continuous forward processes with drift and diffusion terms, where DDPM represents variance-preserving and score networks represent variance-exploding formulations.

Reverse processes require score estimation

The reverse-time SDE depends on the score function, meaning models must estimate this quantity to transform noise back into clean images through either stochastic or deterministic trajectories.

Rectified flow enables faster inference

Rectified flow variants straighten probability paths between distributions, allowing high-quality image generation with significantly fewer numerical integration steps than traditional curved diffusion trajectories.

VAEs compress and structure latent spaces

Variational autoencoders reduce high-dimensional pixel redundancy into compact latent representations by combining reconstruction loss with KL divergence regularization toward a prior distribution.

Classifier-free guidance strengthens alignment

Guidance techniques interpolate between conditional and unconditional model predictions during inference to enhance alignment between generated images and text prompts without requiring separate classifier training.

Multi-modal encoders bridge vision and language

Architectures like CLIP use contrastive learning to align image and text representations in shared spaces, enabling effective conditioning for text-to-image generation systems.