Imagine you're a restaurant chain that spent years perfecting a signature dish at each of your five locations. Each chef nailed their local menu, but no single chef knows all five recipes. You could retrain one chef from scratch on all five cuisines — expensive. Or you could have the specialist chefs write down their recipes and teach a single understudy in a week. The understudy's training is cheap, but you still paid for the five specialists. This paper asks: when is the understudy route actually cheaper end-to-end? The committed claim: on-policy distillation (OPD) from a student's own GRPO checkpoint recovers that checkpoint's accuracy at 50× fewer distillation FLOPs, and a multi-teacher OPD pipeline consolidating domain-specialist GRPO checkpoints into one generalist student matches joint-GRPO accuracy with 42% fewer total FLOPs across two domains. This is not a new algorithm — it is a rigorous compute-accounting exercise that tells you when to use which training recipe. The ladder here is GRPO itself, the reinforcement learning method popularized by DeepSeek. When OPD uses an external post-trained teacher, it generally loses to GRPO in accuracy per training FLOP — and often inflates response length, a known pathology of KL-based distillation. The reversal happens only when the teacher is the student's own GRPO checkpoint, a specific and somewhat circular setup. The 50× efficiency number is real but misleading in isolation: it measures distillation FLOPs only, ignoring the GRPO run that produced the teacher. The honest total-compute comparison — specialist GRPO training plus multi-teacher distillation versus joint GRPO from scratch — lands at 42% savings on two domains and the paper reports 8% distillation overhead on four domains. Architecturally, this sits in the on-policy knowledge distillation family using reverse-KL divergence, where the student generates its own rollouts and a frozen teacher provides dense per-token supervision. GRPO is the RL baseline — a group-relative variant of policy optimization that scores rollouts against each other rather than using a learned reward model. The key structural insight is that the teacher's identity matters enormously: external teacher loses, self-checkpoint teacher wins, multi-domain specialists win bigger. The compute accounting is done in FLOPs, not wall-clock, which sidesteps hardware-specific parallelism effects but makes the comparison clean. Integrity is mixed. The paper compares against GRPO at matched settings across multiple student sizes (1.5B, 7B), two model families, and two domains (math and code), which is more thorough than most distillation papers. But the benchmarks are held-out accuracy on tasks chosen by the authors, not pre-registered, and there is no independent replication. The 50× number and the 42% number are the load-bearing claims — the former is distillation-only and the latter is total-pipeline, and the paper is honest about distinguishing these, which is a good sign. Response-length inflation is reported as a failure mode of vanilla OPD, which adds credibility. The milestone that matters is scaling multi-teacher OPD beyond four domains. At two domains, distillation costs 5.2% of specialist FLOPs; at four, 8.0%. The open question is whether this scales sublinearly, linearly, or superlinearly as you add domains — and whether the generalist student maintains per-domain accuracy or starts losing quality at 8, 16, 32 specialist teachers. The next concrete number to watch: multi-teacher OPD at 8+ domains with the distillation overhead still under 15% and no accuracy degradation. The obvious experiment not run: scaling to more than four domains and testing whether the consolidation holds. The paper stops at four, which is enough to show the trend but not enough to characterize the scaling curve. My read is (a) compute budget — each additional domain requires a full GRPO specialist run, and the combinatorial cost grows. The authors likely had the infrastructure for four clean comparisons and chose to publish rather than wait. The deeper missing experiment is head-to-head against concurrent work like DITTO or online distillation variants that interleave RL and distillation steps rather than treating them as separate stages.
