Humanity's Last Exam

Ian Fisher explains how Poetic uses a recursively self-improving meta-system to build or optimize reasoning harnesses on top of language models. These harnesses combine code, prompts, data, and model calls to improve performance on hard tasks, and can be adapted as newer models become available. He contrasts this approach with fine-tuning, which can be costly and quickly overtaken by newer models, and describes benchmark results and optimization costs as examples. He also discusses Poetic’s early-access program and advises engineers to experiment with AI and build things themselves.

Alexandr Wang traces Scale AI’s path from an early service for human labor to data work for self-driving companies, frontier model labs, and enterprise and government AI applications. He argues that differentiated data, environments, and evaluations help organizations build specialized models, while reasoning and reinforcement learning are opening new scaling curves. He expects work to shift toward people directing agents, with human judgment still needed to set goals, coordinate systems, and handle failures. He also discusses competition with China in AI, highlighting energy, data, manufacturing, and defense as important factors. His advice to builders is to care deeply about quality and adapt quickly.