OpenJarvis

The talk argues that an agent’s harness: the software layer that connects an LLM to tools, memory, compute, and the outside world, can matter as much as the model itself. It traces harnesses from simple prompting and tool use to systems that manage context, coordinate persistent subagents, and improve their own prompts or code. The speakers present Prime Agent and OpenJarvis as examples of harness design for long-running research and on-device personal AI, and describe QM as YC’s open-source work assistant. Across the examples, they emphasize expressive agent-controlled tools, long-horizon evaluation, efficient context management, and careful permissions.

The talk brings together five presentations on specialization across AI chips, GPU kernels, inference infrastructure, and simulation. Stuart explains how ParallelKittens uses different transfer mechanisms and scheduling strategies to make multi-GPU kernels fast and maintainable. John argues that improving local models and accelerators could shift much inference away from data centers, while Mark describes how AI-generated GPU kernels create new challenges for benchmarking and correctness. Misha discusses matching different inference phases to different hardware, and Brennan shows how entity-component-system design can make GPU-based game simulation more than 100 times faster than CPU baselines.