Press Space to continue
Debate as a scalable approach to AI alignment
Press Space to continue
New GDM AGI Safety work: Debate seems like one of the few alignment approaches that might scale to superintelligence. How much value can it add today? If you're interested in this kind of work, we're hiring!
ZacKenton1@ZacKenton1Aug 19
1/7 Can AI debate reduce reward hacking in RLAIF? Training against a weak LLM judge, RLAIF hacks: reward rises but judge and policy accuracy both collapse. Debate with an adversarial critic maintains judgments, allowing the policy to recover 45% of the performance gap to RLVR 🧵 https://t.co/yHNHHI2Ceg