Press Space to continue
Curated tech Twitter/X posts on AI, startups, and product development—signal without the doomscroll, updated on weekdays.
Press Space to continue
Press Space to continue
"BeNice! Closing Looping’s Inference-Memory Gap via KV-Sharing" Looped Transformers reuse the same layers multiple times to improve performance without adding parameters, but their KV-cache memory still grows with every loop. This paper proposes a training method that lets different loops reuse the same KV-cache without losing accuracy, by combining stochastic-depth pretraining with a fine-tuning objective that makes earlier loops compatible with the final loop's cache. On a 1.3B model,…
