RNN

The talk explains how recursion can give AI models more reasoning depth at inference time without requiring more parameters. It compares hierarchical reasoning models (HRM) with tiny recursive models (TRM), highlighting repeated refinement of latent states and truncated backpropagation as key ideas. The speakers argue that these methods can solve tasks such as Sudoku and ARC with small, task-specific models, without step-by-step reasoning traces. They close by suggesting that recursive reasoning could complement the broad representations learned by large language models.

The talk traces the development of Transformer models through three key steps: LSTMs, seq2seq models with attention, and the Transformer architecture. It explains how LSTMs helped neural networks retain information across long sequences, while attention let decoders access encoder states directly. Transformers removed recurrence, allowing parallel processing and better performance on translation benchmarks. The talk also connects Transformer variants such as BERT and GPT to the large language models used in products like ChatGPT and Claude.