Press Space to continue
This matters because builders and AI practitioners can now
Press Space to continue
Not Diamond just rebuilt coding agent evals around the user. Their router reports roughly Opus xhigh quality at 20 to 60% lower cost across three benchmarks. How they got that number is the better story. Most evals hand the model one task and let it run alone for a handful of turns, which undersells how messy real sessions get. The developer keeps talking, context piles up, and the cache expires while someone steps away for lunch. Session length and cache dynamics drive more of the real…
tomas_hk@tomas_hkSep 1
Today we’re releasing our methodology for evaluating model routing with interactive benchmarks, which represent agent cost accumulation better than static benchmarks do. Across leading benchmarks, we achieve Pareto-dominance, exceeding Opus xhigh quality at 20–80% lower cost. https://t.co/COqhkP9GPT