Stanford Online

Stanford Online

Language models exhibit a fundamental 'reversal curse' where they fail to generalize relational knowledge (like reversing 'A is B's parent') when learned through parameter updates, yet perform perfectly when the same information is provided in-context, revealing distinct computational modes for parametric versus contextual learning.

Stanford Online

YC's Garry Tan and Diana Hu explain how AI coding agents are creating '1000x engineers' and enabling tiny teams to generate tens of millions in revenue within months rather than years. They detail the shift from AI copilots to autonomous software factories requiring rigorous testing frameworks and strategic prompting skills to achieve production-grade output at unprecedented scale.

Stanford Online

This seminar explores how personalized foundation models trained on wearable data can predict intervenable health events (like cravings or blood pressure spikes) to enable just-in-time digital interventions, but emphasizes that HCI is critical to solve adherence challenges, inconsistent labeling, and misaligned evaluation metrics.

Stanford Online

This lecture establishes that training data—not architecture—is the most critical and secretive component of modern language models, examining the technical impossibility of crawling the 'entire internet,' the three-stage data pipeline from raw web to specialized post-training, and the tightening legal constraints of copyright law and terms of service that increasingly restrict what can be legally used for training.

Stanford Online

This lecture explores how to evaluate language models, examining different definitions of "good" from benchmark scores to economic usage, diving deep into perplexity as a foundational metric and its limitations, and tracing the evolution of exam-based benchmarks like MMLU from novel challenges to saturated metrics.

Stanford Online

This lecture explores practical implementations of scaling laws for language models, focusing on the MiniCPM paper's techniques for stabilizing training across scales, including Maximal Update Parameterization (MUP) for consistent learning rates and Warm-up Stable Decay (WSD) schedules for efficient training continuation.

Stanford Online

This lecture covers practical training of text-to-image diffusion models, detailing the evolution from UNet to Diffusion Transformer architectures, the three-stage training pipeline (pre-training, post-training, and tuning), and critical optimizations including flow matching loss functions, logit-normal time step sampling, and resolution-aware noise scheduling.

Stanford Online

Jensen Huang argues that computing is undergoing its first fundamental reinvention in 60 years, shifting from pre-recorded, general-purpose, on-demand processing to generated, accelerated, continuously running agentic systems. He reveals that NVIDIA achieved a 1-million-x speedup over the last decade through extreme 'co-design' of hardware, software, and algorithms, fundamentally outpacing Moore's Law.

Stanford Online

General Matter CEO Scott Nolan argues that energy has superseded compute as the primary bottleneck for AI scaling, requiring an urgent shift from exhausted 'stranded' power to massive new baseload generation—specifically nuclear energy—which is itself constrained by uranium enrichment supply chains.

Stanford Online

This seminar outlines a roadmap for autonomous surgical robotics to address critical healthcare labor shortages, proposing a physics-based approach built on four pillars—perception, modeling, planning, and control—that achieves sub-2mm precision through real-time digital twinning rather than relying on data-scarce foundation models.

Stanford Online

Inference now dominates AI economics, with OpenAI generating 8.6 trillion tokens daily—exceeding frontier model training compute in under four days. Unlike training, autoregressive inference cannot parallelize across sequences, making it fundamentally memory-bandwidth bound rather than compute bound, with batch sizes under 295 on H100s failing to saturate GPU capacity.

Stanford Online

This lecture transitions from theoretical foundations to practical architecture design for diffusion models, explaining how U-Net structures leverage convolutional inductive biases, hierarchical downsampling for global context, and skip connections to preserve local details while maintaining strict dimensional requirements for iterative denoising.