Looped Transformers and Recurrent Depth in Large Language Models | Tech Twitter
Looped Transformers and Recurrent Depth in Large Language Models
Press Space to continue
I put together a mega write-up on GPT-6 Astra & looped transformers.
How looped transformers / recurrent depth works, cost-tradeoffs, whether it hides reasoning traces, with lots of figures and a tour of recent looped transformer research.