Finding signal on Twitter is more difficult than it used to be. We curate the best tweets on topics like AI, startups, and product development every weekday so you can focus on what matters.
it's basically impossible to interpret evals by looking at just | Tech Twitter
Press Space to continue
it's basically impossible to interpret evals by looking at just at the pass/fail scores these days
many of the failures I see in benchmarks are due to overly strict hidden tests, in some cases the model's answer makes more sense than the expected eval result
