Press Space to continue
Patronus AI launches SpeedrunBench for testing agent optimization
Press Space to continue
Patronus AI just dropped SpeedrunBench. This benchmark makes frontier agents SPEEDRUN video games. No partial credit. No "close enough." Just frames on a clock and a leaderboard that doesn't lie. They are not just testing if agents can play, they are testing if agents can get BETTER through an autoresearch loop, pushing all the way to near TAS-level play. 100+ hours of agent replay footage. Public leaderboard. A live game app anyone can try at http://speedrunbench.com. This might be one of…
anandnk24@anandnk24Sep 3
Today, we’re releasing SpeedrunBench: the first benchmark that measures how fast agents can beat video games. We asked frontier models not to simply complete video games, but to speedrun them - across 10 titles like Mario Kart and Pokemon. On simpler games, agents get close to the human world record. However, on more complicated games like Pokemon Blue, the best models Kimi-K3 and Opus 5 are ~4x off the world record. Benchmark, paper, and demo below. @PatronusAI