The GPT Moment for Robotics Is Here

Wang’s goal is a model that can control different robots across the tasks they are physically capable of. He describes a gradual path: start with a capable base model, deploy it with human help when needed, and improve it through experience in real settings.

Wang traces progress from SayCan’s language-based planning to PaLM-E and RT-2, which connected vision-language models to robot actions. Open X-Embodiment combined data from different robots; Wang says its generalist policy performed 50% better than policies optimized for individual platforms. He also cautions that the dataset did not solve robotics evaluation and is small relative to the field’s overall data needs.

Wang and Quan Vuong describe work with Weave on folding varied, unseen laundry items, and with Ultra on packing products into shipping pouches in a warehouse. The Ultra system ran across a full day with some human intervention. Wang says the approach turns some engineering challenges into operational ones, such as finding suitable tasks and collecting useful data.

Physical Intelligence runs many robot evaluations with models hosted in the cloud, using overlapping action chunks to account for inference delays. Wang’s advice for vertical robotics startups is to understand existing workflows, find where robots can help, use scrappy hardware, collect data, and evaluate performance. A mixed human and robot system can help reach economic break-even before scaling deployments.

Wang says Physical Intelligence publishes research and open sources the same π0 and π0.5 model weights it uses internally, with the aim of helping other teams build. He describes gaps in robotics infrastructure, including tools for data collection, annotation, and evaluation. He also shares a prototype that monitors large pre-training runs and takes corrective action, improving compute utilization by about 50%.