Fei-Fei Li: Spatial Intelligence is the Next Frontier in AI

When Li began working on computer vision, data was scarce and standard algorithms struggled to generalize. She and her student gathered images from the internet, built a visual taxonomy, and released ImageNet openly so researchers could train and compare their systems. In 2012, a convolutional neural network called SuperVision sharply improved challenge results, bringing together data, GPUs, and neural networks.

ImageNet focused on identifying objects, but Li wanted computers to describe whole scenes, such as a conference room and what is happening in it. Around 2015, her lab published early work on image captioning with Andrej Karpathy and others. She connects that progress to today’s generative models, which can create images from text.

Li says AI needs models that capture 3D structure and can understand, generate, and reason about the world. Vision is difficult because cameras project a 3D world onto 2D images, spatial data is less accessible than text, and models must work across both generated and real environments. She sees potential applications in design, games, robotics, and eventually the metaverse.

Li describes starting a dry-cleaning business at 19 to support her family and attend Princeton, then taking on new challenges in academia, at Google, and at Stanford’s Human-Centered AI Institute. She says she enjoys building from the ground up and looks for intellectual fearlessness in students and hires. Her advice to people pursuing research is to follow intense curiosity while choosing problems that academia can meaningfully address.

Li says a healthy AI ecosystem can include both open and closed approaches, depending on a company’s business strategy, and argues that open-source efforts should be protected. On world-model data, she says World Labs is taking a hybrid approach that values both quantity and quality. For academic researchers, she points to interdisciplinary AI, scientific discovery, theory, explainability, causality, and small-data problems as areas to explore.