One Training Example Is (Almost) All You Need: On-Policy Distillation's Dirty Secret Is That It's Starved for Compute, Not Data

One Training Example Is (Almost) All You Need: On-Policy Distillation's Dirty Secret Is That It's Starved for Compute, Not Data | BedrockNews