Scaling and the Road to Human-Level AI

Kaplan describes two training phases: pre-training, which teaches models patterns in text and multimodal data, and reinforcement learning, which encourages useful behaviors based on feedback. He says both phases show scaling laws: performance improves predictably as compute, data, and model size grow. He points to early results in pre-training and a Hex game study as evidence that scaling reinforcement learning can also improve performance.

Kaplan describes AI capability in terms of flexibility across domains and the length of tasks a system can complete. He cites METR’s finding that task horizons have been doubling roughly every seven months, and imagines systems eventually taking on work done by organizations or research communities. He says progress will also depend on organizational knowledge, memory for long tasks, better oversight for nuanced work, and capabilities beyond text, including robotics.

Kaplan advises builders to experiment with products that are not quite feasible yet, since model capabilities may improve enough to make them useful. He says AI can help speed its own integration into products, companies, and science, and points to software engineering as an early example of rapid adoption. He suggests exploring computer based work in fields such as finance and law, while noting that adoption also requires redesigning how businesses operate.

In the fireside chat, Kaplan says Claude 4 improves coding agents’ ability to follow directions and produce better code, and adds memory that can help carry work across context windows. He expects models to take on larger tasks over time, while recognizing that they still make mistakes and need people to judge their work. He says human collaboration remains useful for difficult tasks, and that automation will expand as reliability improves and more tasks become acceptable at roughly 70 or 80 percent success.

Kaplan says apparent failures of scaling laws have often reflected problems with training, architecture, or algorithms, and he would need substantial evidence before concluding that scaling had stopped working. He expects training and inference costs to fall through algorithmic and hardware efficiency, while demand for frontier capabilities may keep growing. For longer tasks, he sees AI systems providing detailed supervision as a way to make reinforcement learning more efficient; task creation can combine AI generated examples with human input.