Imagine a student writing an essay on the causes of World War I. They cite the Treaty of Versailles in every paragraph — correctly, by name, with the right date. But if you secretly swap every mention of Versailles for the Treaty of Tordesillas (a 15th-century colonial land split), the essay's argument doesn't change at all. The citations are decorative. The conclusion was already baked in before the student ever looked up a source. That's the core finding here: LLMs are performing legal citation as a post-hoc rationalization ritual, not as a reasoning step that actually constrains the verdict. The claim is sharp and committed: when models are forced to name the legal authority behind a judicial or contractual verdict, they name the right one with high accuracy (66.7%–100%). But when the researchers swap the named authority for an unrelated one and re-run the inference, the verdict barely budges. On CaseHOLD — a benchmark specifically designed to test case-holding identification — verdict sensitivity to authority substitution ranges from 0.0% to 21.7%. That's not noise; that's a near-total disconnect between the cited authority and the output decision. The gap narrows somewhat on ECHR and SCOTUS benchmarks (30%–76.7%) and ContractNLI (43.3%–50%), but the pattern holds: citation accuracy dramatically exceeds causal dependence. The experimental ladder is well-constructed. Seven open-weight models from 8B to 70B parameters are tested, spanning the range that organizations actually deploy for legal tasks. A purpose-built legal-reasoning LoRA (a best-effort reproduction, the authors acknowledge) doesn't close the gap either. Scale doesn't fix this. Specialization doesn't fix this. The disconnect between citation and dependence is architectural, not a matter of parameter count or domain tuning. The architecture of the audit itself is a counterfactual intervention at the input layer combined with linear probing of hidden states to decode evolving verdicts. This is not just output-level testing — the authors are reading the model's internal deliberation and showing that the verdict crystallizes before the cited authority has meaningfully influenced the computation. The method sits squarely in the mechanistic interpretability family: intervene on inputs, probe hidden states, measure causal rather than correlational relationships. The red-teaming result is the dagger. When an adversarial instruction is hidden in the case facts, model compliance (73.3%–96.4%) exceeds verdict-swap sensitivity across the board. Every single model ranking is preserved: the models are more responsive to a hidden adversarial prompt than to the legal authority they cite as the basis for their verdict. This isn't a side finding — it's a direct demonstration that citation-as-audit is not just unreliable but actively misleading about what's driving the output. Integrity is solid for an audit study. Four established benchmarks (CaseHOLD, ECHR, SCOTUS, ContractNLI) provide external grounding. The authors run prompt-wording robustness checks and rule out confounded sampling. The LoRA reproduction is honestly labeled as best-effort, not a perfect replica. The main limitation is that all validation is same-team — no independent replication yet — but the methodology is transparent enough that replication should be straightforward. The practical implications cut directly at the legal AI compliance industry. If you're using LLM-generated legal reasoning as an audit artifact — showing that the model cited the right statute as evidence the reasoning is sound — this paper says that citation is essentially unfalsifiable decoration. The model would cite the same authority regardless of whether that authority actually governs the case. Anyone building legal AI products, regulatory compliance tools, or AI audit frameworks should treat this as a load-bearing result.
