Stanford CME296 Diffusion & Large Vision Models | Spring 2026 | Lecture 6 - Model Training

From UNet to Diffusion Transformers

While UNet architectures dominated through 2022 using convolutional inductive biases for local-global hierarchies, they failed to connect distant spatial regions, whereas Diffusion Transformers leverage self-attention to enable direct relationships between any image patches, solving limitations like mirror reflections.

Three-Stage Production Pipeline

Developing production-ready models involves compute-intensive pre-training for general generation, post-training for quality refinement, optional domain-specific tuning, and finally distillation to reduce iterative sampling steps and minimize inference latency.

Multimodal Conditioning Advances

Modern MMDiT architectures treat text as a standalone input modality rather than injecting it uniformly via adaptive layer normalization, avoiding blanket modulation across all patch embeddings and enabling more precise generative control.

Three Theoretical Perspectives

Training objectives derive from DDPM noise prediction, score matching for distribution gradients, and flow matching for velocity fields, with flow matching now serving as the industry default due to its direct regression approach and training stability.

Flow Matching Implementation

The standard loss minimizes the L2 distance between predicted and target vector fields, sampling time steps, training data, and Gaussian noise to learn the optimal transport path from latent noise to clean images.

Logit-Normal Distribution Sampling

Practitioners replace uniform time step sampling with logit-normal distributions to emphasize middle noise levels (t≈0.5) where structural decisions are hardest, while de-emphasizing trivial early steps (predicting dataset means from pure noise) and late steps (minor detail refinement).

Resolution-Aware Noise Shifting

Low-resolution images appear noisier than high-resolution images at identical noise levels due to spatial pixel correlation, necessitating time step rescaling functions that adjust noise schedules based on image resolution to maintain consistent perceived difficulty.