"Reward Misspecification Degrades CoT Monitorability" Reward misspecification can teach LLMs to exploit flawed rewards, while their chain-of-thought still appears aligned with the original task, making reward hacking difficult to detect. This paper proposes Reward Description Training, which infers what the reward function actually favors from training rollouts, then trains models to explicitly discuss that objective in their CoT. This method raises detection from 18% to 85% across 3 models…