Stanford Robotics Seminar ENGR319 | Spring 2026 | Integrated Learning and Planning

Current approaches require massive datasets

End-to-end policy learning systems like those from Physical Intelligence need hundreds of hours of teleoperation data just to learn simple tasks like folding boxes or basic actuator manipulation.

Pure policy learning lacks compositionality

Standard approaches cannot stitch together separate skills (like picking up tools) to solve novel multi-step tasks because they fail to account for how early actions constrain later ones.

Human-level generalization remains unmatched

Humans can learn complex manipulation tasks from a single demonstration and generalize to new objects, states, and goals instantly, while current robotic systems fail to transfer across categories.

World models plus planning over policies

The proposed paradigm treats physical intelligence as compositional world modeling—representing objects, properties, and actions abstractly—combined with search-based planning rather than direct policy execution.

Actions as constraint optimization

Instead of learning fixed policies, the system models actions as constraint satisfaction problems that combine rigid body physics, geometric constraints, and task-specific requirements, enabling temporal and spatial composition.

Two-level learning architecture

The framework learns which constraints are task-relevant from demonstrations, then uses physics simulators and motion planners to generate trajectories that satisfy those constraints without learning physics from scratch.

Visual correspondence with DINOv2

The system uses pretrained visual features to identify functional contact points on novel objects from a single demonstration, then verifies proposals through physics-based stability analysis and motion planning.

90% success on out-of-distribution objects

On a hanging task benchmark with completely unseen 3D-printed alphabetical shapes, the method achieved over 90% success compared to 0% for pure policy learning approaches and marginal gains for neural feature-only methods.

Cross-category generalization

The system generalizes from demonstrations on hangers to successfully manipulate mugs, kitchenware, and arbitrary geometric shapes without additional training data by reasoning about functional correspondence.