“Privileged Solutions or Context-Induced Teacher Behavior?” On-policy self-distillation assumes the teacher improves because it sees the correct solution to the student’s problem. This paper shows that assumption may be wrong. Instead of giving the teacher the correct solution to the student’s problem, this paper gives it a worked solution to an unrelated math problem. Surprisingly, the student still improves just as much, and often more. So the key signal may not be privileged answers, but…