When Your Small Language Model Is on a Budget: Spend on Fewer Bits or on Shuffling the Answer Choices?

A practical cost-accounting study finds 6-bit weights + 4 answer-order passes beats higher precision for small LLMs on MCQA benchmarks. Simple result, useful decision table.

Imagine you're packing for a flight with a strict carry-on weight limit. You can bring one heavy, high-quality toolkit, or you can bring a lighter toolkit plus a second bag of different tools. This paper asks the same question about running small language models on constrained hardware: should you spend your fixed inference budget on heavier (higher-precision) weights, or on lighter weights plus multiple passes that average out the model's sensitivity to the order of answer choices? The committed claim: for small local LLMs doing multiple-choice QA, 6-bit quantized weights lose nothing compared to fp16 — they actually score slightly higher — and the savings are better spent on up to four cyclic reorderings of the answer options. The decision table is clean: use 6-bit weights, spend the leftover budget on option-order averaging, stop there. At 4 bits the 0.6B model's answer-slot skew jumps from 0.15 to 0.53, but four-order averaging recovers +12.6 accuracy points. The experimental ladder here is honest but narrow. The baselines are fp16 Qwen3-0.6B and Qwen3-1.7B with simulated round-to-nearest quantization at 8, 6, 4, and 3 bits — not GPTQ, AWQ, or any real post-training quantizer. The benchmarks are ARC-Easy, ARC-Challenge, MMLU, and HellaSwag, 300 items each. The authors explicitly flag that simulated quantization is the simplest possible approach and that no prior study combined quantization with option-order analysis. They're right about the gap in the literature, but the simplicity of the quantizer limits how far the results transfer to production stacks. The architecture story is straightforward: this is a measurement study, not a new method. The models are decoder-only transformers (Qwen3 family). The quantization is simulated round-to-nearest on the weight tensors, not actual low-bit inference. The cost metric — weight bytes read × passes — is machine-independent by design, which is clever: it sidesteps GPU-specific throughput debates and gives practitioners a portable comparison unit. But it also means the paper doesn't grapple with real-world bottlenecks like KV-cache pressure or activation quantization. Integrity is mixed. The authors use community benchmarks, report confidence intervals, run a Gaussian-noise control to disentangle rounding artifacts from generic noise effects, and openly state that the noise control is 'inconclusive' — a refreshing admission. But the 300-item subsamples are small, the quantization is simulated not hardware-realized, and there's no code release mentioned. The flip-rate analysis (measuring how often quantization changes a model's answer) is a nice internal consistency check but doesn't substitute for independent replication. The milestone framing is practical: for the on-device MCQA use case, the paper establishes that 6-bit is the operating point and 4 passes is the budget ceiling. The next real milestone is replicating these findings with production quantizers (GPTQ, AWQ, GGUF) on actual low-bit hardware, and extending to models in the 3B-8B range where most local deployment happens. The gap isn't years — it's weeks of compute for any group with access to the models. The obvious experiment not run is real quantization with a proper PTQ pipeline. The authors acknowledge this directly — the paper is a simulated study. The honest read is (a): they're exploring the question space cheaply before committing to the engineering overhead of real quantization. This is defensible for a first study, but the practical value of the decision table depends entirely on whether the 6-bit sweet spot holds under real quantizers, which it may not.