Imagine you're a driving instructor in the passenger seat. Your student takes a wrong turn onto a one-way street. The standard assumption in driver's ed is that you can calmly redirect from any wrong position — "turn left here, now right, we're back on track." But what actually happens is weirder: you find yourself giving directions that follow the wrong-way route, even though you can see the mistake. You know it's wrong, but your next-turn instructions adapt to the bad road rather than correcting it. That's the recoverability failure this paper identifies in on-policy distillation. The committed claim: teacher language models providing token-level supervision to student models during on-policy distillation will follow the student's errors rather than correct them, even when the teacher can identify those errors. CriticOPD fixes this by inserting a critique-then-revise loop — the teacher flags the mistake, the student rewrites, and the revised output (without the critique) becomes the training signal. It's not a new distillation paradigm; it's a patch to a broken assumption everyone was relying on. Architecturally, this sits in the knowledge distillation family for autoregressive language models, specifically the on-policy variant where the student generates its own rollouts and the teacher provides token-level feedback. The key structural choice is treating the teacher's critique as "privileged information" — present during scoring but absent during student training. This is a clean information-asymmetry trick borrowed from learning-with-privileged-information literature. The compute overhead is non-trivial: you're now running teacher inference twice (once to critique, once to score the revision) plus a student revision pass. The evaluation runs on two Qwen3 student models across six mathematical reasoning benchmarks, with the largest improvements showing up on competition-level problems — exactly where you'd expect recoverability to matter most, because competition problems have longer reasoning chains with more opportunities for the student to go off-rails. The paper explicitly tests on AMC, AIME, and MATH-500 among others. The gains over vanilla OPD are real but the paper is honest that they're incremental on easier benchmarks. The integrity picture is reasonable but not ironclad. The benchmarks are community-standard math reasoning tasks, not custom-built. The baseline is current on-policy distillation, not a straw man. But all validation is same-team, the teacher and student are both Qwen3 family (raising questions about generalization to cross-family distillation), and there's no pre-registration. The recoverability failure itself is demonstrated empirically — shown, not proven — which means the claimed mechanism could have alternative explanations. The milestone to watch is whether critique-guided distillation scales beyond math reasoning to code generation, multi-step planning, and agentic tasks where recoverability failures should be even more severe. If CriticOPD works on SWE-bench or planning benchmarks with similar gains, this becomes a standard component of every distillation pipeline. If the gains are math-specific, it's a niche fix. The obvious experiment not run: cross-family distillation (e.g., GPT-4 teacher, Llama student) to test whether recoverability failure is a universal property of autoregressive teachers or an artifact of within-family distillation. The honest read is (a) — compute budget. Running CriticOPD with a proprietary teacher API would be expensive and slow, and the authors are working with what they have. The code release on GitHub is a good sign for replication.
