A Single GPU Can Now Prove a 32-Billion-Parameter LLM Didn't Lie About Its Outputs

Sable proves bit-exact inference of full-scale LLMs in under 2 minutes on one GPU, at 6.4 cents per request. The first system to make verifiable AI inference economically plausible.

Imagine you order a steak at a restaurant and the kitchen has a glass wall. You can watch every chop, every flip, every seasoning decision. Now imagine the kitchen is sealed, but after you eat, a notary hands you a cryptographic receipt proving every step happened exactly as the menu described — same cuts, same heat, same timing — and forging that receipt is computationally impossible. That is what Sable does for large language model inference. You send a prompt, get tokens back, and receive a proof that the model you were promised actually produced those exact tokens, with no substitution, no post-hoc editing, no cheaper model swapped in behind the curtain. The committed claim: Sable is the first argument system that proves bit-exact inference of production-scale quantized transformers (32B parameters) on a single GPU, with proving overhead of roughly 41-50× serving time and a cost of 6.4 cents per 4,096-token request at public GPU rental rates. This is not a toy demonstration on a small model — it covers Qwen2.5-32B, Qwen3-32B, and the mixture-of-experts Qwen3-30B-A3B. The core technical move is fixing an integer-only forward pass (F_O) with no floating-point step whatsoever: every rounding, normalization, softmax, and sampling decision is deterministic given the inputs. This eliminates the non-determinism that has historically made proving neural network inference either impossible or astronomically expensive. The ladder matters here. The only named predecessor is zkLLM, and Sable claims 40-64× less time per parameter per token. That is not an incremental gain — it is the difference between a research curiosity and something that could plausibly be deployed. The cost math is critical: at 6.4 cents per 4,096-token request, verifiable inference is roughly 40-50× more expensive than unverified serving, which prices it out of casual chatbot use but puts it squarely in range for high-stakes applications — legal discovery, medical reasoning, financial compliance, regulatory auditing — where the cost of an undetected model swap or output manipulation dwarfs the proving overhead. Architecturally, Sable lives in the sumcheck/IOP family (interactive oracle proofs), not the zkSNARK circuit-compilation approach that most ZK-ML work has pursued. The constraint system uses relations over the 31-bit KoalaBear prime, with a formal guarantee that no wraparound occurs — meaning the integer arithmetic is truly integer arithmetic, not modular arithmetic pretending to be. Witness commitment uses multilinear polynomials over Reed-Solomon codes with BLAKE3 Merkle trees and a mixed-length BaseFold opening. The prover streams layer-by-layer on one GPU (an NVIDIA RTX PRO 6000 Blackwell), which means memory scales with one layer's activations, not the full model. This is what makes 32B-parameter proving feasible on consumer-grade (well, prosumer) hardware. The integrity picture is unusually strong for a cryptographic systems paper. The soundness bound is 2^{-103} in the random oracle model, which is not a simulation — it is a mathematical proof. The accuracy validation uses standard benchmarks: WikiText-2 perplexity (greedy token match at 86.3% of positions vs BF16), MMLU (within 1.6 points of BF16), and GSM8K (within 1.7 points). These are honest numbers — the integer quantization does lose something, and the authors report it plainly rather than cherry-picking metrics where it doesn't. Code is released on GitHub. Verification takes 19-56 seconds on a single CPU core, which is fast enough for practical deployment. The proof is explicitly not zero-knowledge, which is the most important caveat. The verifier sees the model weights, which means Sable targets open-weight models only. This is not a limitation the authors bury — they state it in the abstract. Making it zero-knowledge would protect proprietary weights but would likely balloon proving time substantially. The obvious next experiment is exactly that: can you add a ZK layer without destroying the performance budget? The authors almost certainly know the answer is 'not yet with acceptable overhead,' which is why they scoped the system to open-weight models and shipped what works. The milestone to watch is not qubit counts or accuracy percentages — it is the overhead multiplier. At 41-50× serving time today, the target is sub-10× overhead, which would make verifiable inference viable for a much broader class of applications. Hardware improvements (next-gen GPUs, NTT accelerators) and algorithmic improvements to the sumcheck protocol could get there within 2-4 years. If someone demonstrates verifiable inference of a 70B+ model at under 10× overhead, the economics shift from 'high-stakes only' to 'default for any regulated industry.'