Imagine you're learning to judge wine by copying the scores of a critic who secretly hates Riesling. You taste thousands of bottles, you get really good at predicting the critic's scores, and your temperature-adjusted confidence intervals look perfect. But you're still wrong about Riesling — and no amount of practice or re-scaling will fix it, because the error isn't in your palate, it's in who taught you. That's the core mechanism of this paper: when a student model learns from a teacher's soft probability targets instead of actual outcomes, the student inherits the teacher's systematic bias, and standard calibration diagnostics can't detect it. The committed claim: in Gaussian surrogate models where finite-sample risk is exactly computable, three ubiquitous ingredients of candidate-scoring systems — knowledge distillation from a larger model, frozen pretrained representations, and temperature calibration — each have provable failure modes that interact badly. Specifically, teacher bias transfers permanently to the student, frozen representations help only below an explicit sample-size threshold, and recalibration via temperature scaling removes excess risk only when the teacher's coefficient has the correct sign. When it doesn't, residual error survives calibration and hides from standard diagnostic checks. The architecture is deliberately classical: Gaussian linear models and probit regression where closed-form solutions exist, plus Luce's choice axiom for the multi-option scoring setting. This isn't a new model — it's a scalpel applied to the assumptions behind existing models. The authors derive exact bias decompositions showing that student-teacher agreement (the metric everyone checks) is uninformative about whether the student has inherited a bad prior. They prove that under Luce's axiom, candidate scores are coherent across event refinements if and only if they aggregate via log-sum-exp — and they use this as a testable diagnostic. The empirical validation uses 231,521 credit-card complaints from the Consumer Financial Protection Bureau, predicting whether a complaint ends in monetary relief. The ladder is clear: a student trained on actual outcomes undercuts the teacher's decision cost by 8%. A student trained on the teacher's probabilities converges exactly to the teacher's cost — never below it, no matter how much data. A 30B parameter model doing zero-shot inference remains worse than the logistic teacher even after recalibration, and departs from the log-sum-exp coherence condition by 17.5 decibans. These are concrete, named numbers on a real regulatory dataset. The integrity picture is strong for a theory paper. The Gaussian surrogate results are exact — not simulated, not approximated, analytically derived. The empirical dataset is public (CFPB complaint database), the code is in ancillary files, and the baselines are honest: the logistic teacher is simple but appropriate for the task. The authors don't claim their Gaussian analysis covers all architectures; they're explicit that these are surrogates chosen because they yield closed forms. The main integrity question is external validity: do these failure modes persist in deep nonlinear models? The authors argue the bias-transfer mechanism is architecture-agnostic, but that claim rests on the linear decomposition. The practical upshot is a diagnostic protocol: if you're deploying a distilled scoring model, you must check its probabilities against realized outcomes, not just against the teacher or against calibration curves. Agreement with the teacher tells you distillation worked, not that the teacher was right. Temperature scaling fixes scale errors but not direction errors. Frozen representations save labeled data but plateau at a predictable sample size. Each ingredient is useful — none is sufficient — and the failure modes compound silently. The obvious successor experiment is running these diagnostics on a production-scale deep network — not a Gaussian surrogate — and measuring whether the bias-transfer and calibration-hiding effects persist at the same magnitude. The authors almost certainly know this is the gap; the likely reason it's not here is that the paper's value proposition is the exact analytical machinery, and adding empirical deep-learning experiments would muddy the contribution. The next paper will be exactly that bridge.
