A Pre-Registered Stress Test for the Quick-and-Dirty Math Behind LLM Quantization

Second-order Fisher approximations predict quantization error surprisingly well — until they don't. MoE models break the quadratic assumption, and bf16 gradients silently corrupt predictions fourfold.

Imagine you're a civil engineer estimating how much a bridge will sag under load. You use a simple formula — Hooke's Law, force times spring constant — that works beautifully for small deflections. But past a certain load, the bridge starts behaving nonlinearly: the steel yields, the cables stretch unevenly, and your neat formula stops matching reality. This paper asks the equivalent question for LLM quantization: how far does the simple second-order approximation — the one baked into every Hessian-aware quantizer — actually predict what happens when you compress a model's weights? The committed claim: the standard ½Δ⊤FΔ Fisher-information screen reliably predicts *relative* KL divergence from the unquantized model across a 16-fold range of perturbation scales, int4 group sizes, and rounding modes — but absolute predictions are heavy-tailed and require trimming to become useful. After trimming, 10 of 12 tensor-block cells on Qwen3-4B and OLMo-2 land within 15% of measured KL. This is not a new quantization method. It is a metrological audit of the math everyone already uses, run on five models and pre-registered before results were observed. The most operationally important finding is a landmine: computing per-sequence gradients in bf16 introduces fourfold prediction errors. The authors trace this precisely to arithmetic in the lower half of the backward pass, where bf16's limited mantissa silently corrupts the Fisher estimate. Switching to fp32 for that portion eliminates the error. If you are running Hessian-aware quantization pipelines and haven't audited your gradient precision, this paper is telling you your predictions may be garbage. The MoE result is structurally interesting. Qwen3-30B-A3B exhibits nonquadratic response at small perturbations — int8 KL scales as α^0.63, not α^2. This means the second-order screen fundamentally misbehaves for mixture-of-experts architectures at the perturbation scales that matter for practical quantization. The quadratic lens is the wrong lens for these models, and nobody had documented that cleanly before. The practical deliverable is a screener pipeline: one streamed fp32 Fisher pass, trimmed tensor-block predictions with confidence bands, a flag for aligned tensors, and per-layer format allocation. Tested on the held-out Mistral-7B-v0.3 family, all six predictions fall within band, all three KL budgets are met, and vLLM throughput predictions land within 0.4%. The throughput prediction is the killer number — it means you can choose quantization formats without running the actual serving benchmark. Batch-invariant kernels run at 0.55× throughput relative to standard kernels, and changes in reduction order introduce a noise floor of 0.17–0.22 nats per sequence. These are the friction costs of doing the measurement honestly. The authors don't hide them — they quantify the tradeoff between measurement fidelity and compute cost, which is exactly what a metrological paper should do. The integrity posture is unusually strong for an ML paper. Pre-registration before results were observed, testing on a held-out model family not used during development, explicit quantification of numerical precision effects, and honest reporting of where the approximation breaks (MoE models, absolute predictions). The main gap is the absence of released code and independent replication — the pipeline is described but not yet publicly available as far as the abstract indicates.