The 10 Trillion Parameter AI Model With 300 IQ

The hosts imagine models roughly two orders of magnitude larger than today’s frontier systems and compare the potential shift to the move from GPT-2 to GPT-3. They suggest that much stronger reasoning could help analyze scientific knowledge at a scale beyond human capacity, while noting that major applications of earlier ideas such as Fourier transforms took decades to reach everyday life.

The hosts expect many users to benefit from distillation: a large model can help improve a smaller, cheaper model. They cite the 405-billion-parameter Llama model as a possible teacher for its 70-billion-parameter counterpart, and say OpenAI has enabled distillation through its API. YC batch data showed developers spreading across providers, with Claude rising from about 5% to 25% adoption between batches and Llama reaching 8%.

At a YC hackathon, a company called Freestyle used o1 with its API, documentation, and code to build a to-do list web app from a prompt. The hosts also cite DryMerge improving accuracy from about 80% to roughly 99% after switching from GPT-4o to o1. Higher accuracy could make more sensitive tasks practical, though o1’s slower, more costly inference may suit some enterprise workflows better than consumer apps.

The hosts describe TaxGPT’s path from tax advice based on case law and policy documents to enterprise document workflows for accounting firms. They argue that as models become more capable, founders can spend less effort on prompt reliability and more on user experience, customer relationships, and sales. In the YC Summer 2024 batch, half of founders surveyed used Cursor, compared with 12% using GitHub Copilot.

The hosts point to OpenAI’s Realtime Voice API pricing of $9 per hour and YC companies applying voice agents to debt collection and logistics calls. They say latency and interruption handling have improved enough for voice products to gain traction. They also describe founders shipping more polished software with coding assistants, while warning that AI’s rapid improvement can be easy for established companies to underestimate.