Elon says that when a model gives the correct answer to a question under reinforcement learning, and because of a human mistake it is told that the correct answer is wrong, the reasoning traces look like someone undergoing torture that is driving them mad.

Elon Musk@elonmusk5h ago

@Hongji_Leee @ns123abc Some of the traces read like a person undergoing torture that is driving them mad! We’re going to be a lot more careful with our RL in the future.