Building Dota Bots That Beat Pros

Brockman says larger models may develop new capabilities as hardware gets faster, pointing to a language model that learned sentiment classification while predicting the next character in Amazon reviews. Sidor argues that carefully understanding existing methods can matter more than adding elaborate features: fixing bugs in image classification enabled much larger batches and faster training.

Sidor says scaling existing reinforcement learning algorithms took more work than inventing exotic models, and strong engineers could contribute without an AI PhD. Brockman describes choosing Dota because it ran on Linux and had useful APIs. Altman recounts building a repeatable game setup, porting a scripted bot from Lua to Python, and connecting it to a Gym-like environment.

Sidor describes collecting public replays, filtering for expert players, and training a model to imitate their behavior. Imitation could reproduce actions such as creep blocking without understanding their purpose, so reinforcement learning was used to improve the behavior. A TrueSkill scoreboard helped the team track progress as experiments and targeted changes strengthened the bot.

The team entered The International unsure how the bot would perform, then tested it against professional players. An unfamiliar early Wand build exposed a weakness, while another experiment learned to bait opponents by acting weak. The team adjusted training and features under tight deadlines; the bot later went 5–0 against SumaiL, and a human tester learned a strategy from watching it.

Sidor emphasizes distributed systems, careful code, and basic linear algebra and statistics; Brockman adds humility and choosing where engineering discipline helps. Brockman says nontechnical people can contribute to AI safety by learning about AI and taking part in ethical discussions. He describes games as scalable virtual environments for testing algorithms and giving people experience with advanced AI.