Dwarkesh.com

The talk argues that today’s AI progress depends heavily on enormous volumes of task-specific training data, so current models are far less sample-efficient than humans. It compares the data used to train language, robotics, and self-driving systems with the much smaller amounts of experience people need, and responds to objections involving evolution, sensory input, and model scale. The speaker argues that scaling model size alone cannot close the efficiency gap, while AI labs can still profitably automate common tasks because training costs can be spread across many users. The talk closes by asking whether AI systems can automate research that eventually solves the sample-efficiency problem.