DAIR.AI@dair_ai3d ago

We love papers that go deeper down the stack. This is a good example that shifts from agentic context engineering to agentic cache engineering. They also apply recursive self-improvement to cache policies to further reduce cost by 7.1% over expert design. Specifically, this work focuses on KV cache management in long-running agents. When an agent summarizes or trims its history, the prefix change invalidates the cache after it, and the retained context must be prefilled again. Cache engineering gives the agent two operations on its own KV cache. One removes entries from future attention, and the other shifts the positions of the keys that remain. The trajectory text stays unchanged. This essentially provides a programmable interface for agents to define and refine their own cache engineering strategies. A rolling drop of old tool responses scores 1.7 points higher and costs 27.7% less than threshold-triggered summarization. Their AKV serving engine adds up to 1.2x throughput and 1.5x faster P95 latency. https://t.co/fks8lJ46P3 Chat with Paper: https://t.co/wnCyl6Oxqi