World Models, JEPA And The Path To Sample-Efficient RL

The speakers frame intelligence per sample as how much a model learns from each new example, and argue that current models need too many examples to adapt to new tasks. They use Newtonian physics and spacecraft navigation to illustrate how a reliable model of the world can let a system act without repeatedly learning from the environment.

Using a drone as an example, the researcher describes how a known transition model and model predictive control can produce an efficient flight plan. When another agent’s actions make the dynamics uncertain, the problem becomes harder to optimize. The speakers compare chess and Go, where Monte Carlo tree search can explore a limited action space, with driving and robotics, where the number of possible actions and changing conditions make repeated planning costly.

The speakers distinguish behavior cloning, which predicts actions from states, from model-based reinforcement learning, which also predicts future states. They discuss Schmidhuber’s World Models and the Dreamer series, which use learned models to generate imagined rollouts for policy training. The conversation points to video and egocentric recordings as potentially useful data, even when they lack action labels, and mentions DreamZero’s use of a pretrained Wan video model and about 500 hours of teleoperation data.

JEPA predicts the next encoded state rather than reconstructing every pixel, which can reduce the cost of learning transitions. The speakers explain that simple latent prediction can collapse to constant outputs, so methods such as VICReg or SIGReg help preserve useful representations. They also describe experiments using embedding prediction as a cheaper proxy for next-token training.

The speakers identify weak physics-informed generalization, fast adaptation to unfamiliar conditions, and the latency of test-time planning as open challenges. They expect world models to become more common in robotics and self-driving, while noting practical gaps in tactile sensing and friction estimation. They compare learning from experience and replay with the brain’s use of sleep and memory consolidation.