Robots Are Finally Starting to Work

Generalist models outperform specialists

The Open X-Embodiment paper demonstrated that a single policy trained across 10 different robot platforms performed 50% better than individual models optimized for specific hardware, proving scaling laws apply to robotics.

Cross-embodiment solves hardware drift

Unlike single-robot approaches that suffer from hardware variations and software changes every few months, training on diverse platforms forces models to learn abstract 'robot control' rather than specific motor mappings.

Emergent zero-shot capabilities appearing

Current models can now perform complex precision tasks and multi-object reasoning zero-shot—capabilities that required hundreds of hours of data collection just last year.

Vision-language models enable semantic control

Research like RT2 and PaLM-E showed that adapting powerful vision-language models to output robot actions transfers internet-scale knowledge to low-level control, allowing commands like 'move the can to Taylor Swift' without specific training.

SayCan bridged planning and semantics

SayCan was the first demonstration of using language models for robotic planning, leveraging common sense knowledge to reduce the need for task-specific robot data.

The three-pillar hierarchy

Robotics requires solving semantics (now unlocked via LLMs), planning, and real-time control—the latter being the final frontier where vision-language models are now making breakthroughs.

The data capture problem vs. generation problem

While robotic data is constantly generated in labs and industry, there has been no incentive to capture it in standardized formats; Open X-Embodiment was the first effort to aggregate multi-robot data but remains 'a drop in the bucket' compared to what's needed.

Mixed autonomy enables immediate scaling

Current systems are deployable today using a 'peeling an onion' approach: deploy a strong base model with human oversight for edge cases, then improve through real-world exposure rather than waiting for full autonomy.

Real-world validation with YC companies

Physical Intelligence partnered with Weave to deploy laundry-folding robots in commercial laundromats, demonstrating that current models can handle deformable objects and unstructured environments with human assistance.