How To Build End-To-End Physical AI Systems for Robots
TL;DR
This short course presents a unified pipeline for developing humanoid and general-purpose robot policies, spanning synthetic data generation, policy training, and deployment, and gains a practical understanding of how simulation, world models, and foundation models compose into a scalable, end-to-end system for generalizable physical AI.
Abstract
Building intelligent robots that map sensor inputs and language instructions directly to actions, is a rapidly advancing area, but progress is bottlenecked by the cost of real-world data and the risk of physical testing. This short course presents a unified pipeline for developing humanoid and general-purpose robot policies, spanning synthetic data generation, policy training, and deployment. At the core of the pipeline, world foundation models serve a dual role: generating diverse training distributions from limited teleoperation data through controllable augmentation, and acting directly as action policies by predicting future states conditioned on language and sensory context. These capabilities feed into reinforcement and imitation learning in high-fidelity simulation, where cross-embodiment vision-language-action (VLA) models learn robust sensorimotor policies. The resulting policies are then deployed on humanoid platforms via real-time edge inference, closing the loop from data generation to physical execution. Participants will gain a practical understanding of how simulation, world models, and foundation models compose into a scalable, end-to-end system for generalizable physical AI.