Stanford Online

Stanford Online

This lecture explains why high-dimensional pixel space (approximately 1 million dimensions for standard images) is computationally intractable for diffusion models, and how Variational Autoencoders (VAEs) solve this by compressing images into structured latent spaces that follow standard normal distributions, enabling efficient and meaningful generation.

Stanford Online

Albert Gu analyzes the fundamental tradeoffs between State Space Models (SSMs) and Transformers, framing SSMs as "brain-like" fixed-size state compressors that enable linear inference complexity versus Transformers' "database-like" KV cache approach that scales quadratically but enables precise retrieval.

Stanford Online

This lecture explores JEPA (Joint Embedding Predictive Architecture) as an energy-based framework for world modeling that operates in latent space rather than pixels, with Hazel Nam introducing Causal JEPA—a method using object-centric slot attention and aggressive masking to teach models physical object dynamics and interactions.

Stanford Online

Stanford's CS25 introductory lecture traces the evolution from hand-engineered features to Transformer architectures, explaining how self-attention mechanisms enable parallel processing and long-context modeling, while exploring how billion-parameter language models develop emergent reasoning capabilities through next-token prediction on internet-scale data.

Stanford Online

This seminar introduces 'missile-scale' robotics (~1kg) as a critical gap between micro and macro robots, demonstrating that mechanical redundancy (morphological intelligence) enables reliable locomotion in unpredictable terrain without sensors by applying Shannon's information theory to legged locomotion, while biological gait-switching strategies can overcome inherent speed limitations.

Stanford Online

This seminar presents a paradigm shift in robot learning by replacing teleoperation with direct capture of human egocentric experience using wearable sensors, demonstrating that scaling human data—combined with alignment techniques like optimal transport—enables dramatic performance gains and zero-shot task transfer to robots.

Stanford Online

This lecture introduces GPU architecture for language model training, explaining the shift from serial CPU execution to parallel GPU throughput, the critical importance of memory hierarchies, and the SIMT programming model essential for efficient deep learning systems.

Stanford Online

This lecture introduces flow matching as a third paradigm for generative modeling, explaining how it deterministically transports probability distributions from Gaussian noise to data through learned vector fields, while contrasting its velocity-based mechanics with diffusion and score matching approaches.

Stanford Online

This lecture surveys modern transformer architecture evolution by analyzing 19+ recent dense language models, revealing universal adoption of pre-normalization and RMSNorm for training stability and hardware efficiency, while tracing the field's shift from post-GPT-3 experimentation to Llama 2 convergence and recent divergence toward stability-focused designs.

Stanford Online

Tracy Fullerton presents a framework for understanding games not as static technical artifacts but as ephemeral emotional events created through the player's unique encounter with the work, introducing 'readings' as a method to capture and value these personal experiences with the same critical depth applied to literature and film.

Stanford Online

This lecture covers resource accounting fundamentals for training large language models, including FLOPs calculations and memory estimation, explores numerical precision trade-offs from FP32 down to FP4, and introduces einops as a readable alternative to PyTorch tensor operations using named dimensions.

Stanford Online

This lecture introduces Stanford CS336's philosophy of building language models from scratch to understand fundamentals rather than relying on abstractions, addressing how researchers can navigate the disconnect caused by industrialized, closed frontier models by focusing on transferable mechanics and efficiency-minded mindsets.