Why Robotics Still Isn't Solved - But Could Be Soon

The host argues that robotics has repeatedly been declared nearly solved, from AlphaGo and MuJoCo to teleoperation datasets and vision-language-action models. They identify four stubborn challenges: inaccurate physical world models, difficult action representations, limited tactile sensing, and embodiment drift as hardware changes. Collecting teleoperation data is also demanding, making it hard to scale.

Marcel presents Multi-Scale Embodied Memory, developed during an internship at Physical Intelligence, to help policies handle tasks longer than short demonstrations such as folding clothes or making a sandwich. Dense visual memory supports dexterous actions, while a high-level policy stores a compressed text summary of recent events. In examples, memory helps robots stop cooking at the right time and change strategy after repeated mistakes.

Milan Ganai describes R&B Encore, a self-supervised process for finding useful reasoning traces for different robot forms and tasks. A proposer generates candidate explanations, and a validator scores them for brevity, generalization, and whether they help predict actions. The reported results favor movement and gripper-position reasoning for manipulation, and structural affordances for legged navigation; reasoning can also be dropped at inference to avoid added latency.

Tyler Lum presents Sim-to-Real, which uses massively parallel reinforcement learning in simulation to train a single policy for a dexterous hand and arm. The policy tracks object poses from a human video, allowing it to handle unfamiliar tools and tasks without retraining; tests covered 12 unseen tool and behavior combinations. Pose tracking caused about 60% of failures, while random disturbances during training helped the robot recover from dropped objects.

Nico argues that robotics startups can begin with a paying customer problem, off-the-shelf hardware, and teleoperation, then use deployment experience to guide data collection and model improvement. He emphasizes customer-site replicas, repeatable evaluation, failure tracking, and data tools suited to multimodal, multi-rate physical data. Bill and Guanming of General Intuition then describe optimizing world action models by reducing diffusion steps and avoiding full future-frame decoding, reporting inference at 500 milliseconds per 16-action chunk on Jetson.