"Memory Attention" Instead of computing a completely new value for every token, this paper reuses the contextual key and adds a learned memory of what that token generally represents. So rather than generating everything dynamically with dense compute, part of the model’s knowledge becomes a cheap lookup that can even live on CPU, adding capacity without proportionally increasing GPU compute. https://alphaxiv.org/abs/2609.28399