Mechanistic Interpretability Has a Philosophy-of-Science Problem — This Paper Proposes Five Principles to Fix It

MI can now find circuits and features inside neural nets. But finding a thing and proving it's causal are different problems. This paper imports inferential rigor from neuroscience.

Imagine you're debugging a factory by watching conveyor belts through security cameras. You can see which belts move when a product ships, and you can correlate belt activity with output quality. But you can't tell whether Belt 7 is load-bearing or just happens to spin when the real mechanism — hidden behind a wall — does its work. That's the state of mechanistic interpretability today: rich in observations, thin on inference. Hu et al. argue that MI has reached a maturity inflection. The field can now reliably identify neurons, features, circuits, population structures, and dynamical patterns inside models. What it cannot yet do is rigorously distinguish correlation from mechanism — the same inferential gap that plagued neuroscience for decades. The paper's core claim: MI needs an explicit logic of inference, borrowed from philosophy of science and adapted to artificial neural networks, to turn observational findings into cumulative mechanistic knowledge. The proposed framework is five principles organized into a recursive logic. The paper doesn't introduce new probing tools or attribution methods. Instead, it asks: given that you've found a circuit, what chain of reasoning justifies calling it a mechanism rather than an epiphenomenon? The principles address necessity, sufficiency, specificity, convergent evidence, and external validity — each one a checkpoint that filters out premature causal claims. The recursive structure means you re-enter the logic at higher levels of abstraction as you build from individual features to circuits to system-level explanations. The neuroscience parallel is the load-bearing move. Neuroscience spent decades correlating neural activity with behavior before developing rigorous lesion studies, causal interventions (optogenetics, TMS), and formal criteria for mechanistic explanation (Craver's mutual manipulability, Bechtel's mechanistic decomposition). MI is roughly where systems neuroscience was in the early 2000s — swimming in data, lacking inferential discipline. The paper maps specific neuroscience bottlenecks (the correlation-to-causation gap, the fallacy of affirming the consequent from ablation studies, underdetermination of mechanism by data) to their MI counterparts. The most provocative claim is the final one: evidence beyond MI is ultimately required to establish what model mechanisms reveal about the world. In other words, even perfect mechanistic understanding of a transformer's internal computation doesn't tell you whether that mechanism mirrors the real-world process it's modeling. A language model might develop an internal circuit that predicts syntax perfectly, but that circuit could implement a completely alien algorithm unrelated to human parsing. This is a boundary condition on the entire MI research program, and it's stated plainly. Integrity-wise, this is a conceptual framework paper, not an empirical one. There are no benchmarks, no ablations, no code — by design. The validation is argumentative: does the framework resolve known ambiguities in existing MI work? The NeurIPS 2026 Interp4Discovery oral acceptance signals peer review found the argument compelling, but the framework's value will be tested by whether MI researchers actually adopt it to discipline their inferential claims. The paper occupies an unusual niche — it's philosophy of science wearing a CS hat, aimed at practitioners who may not realize they're making philosophical commitments every time they publish an MI result. The successor question is sharp: the authors did not apply the five principles end-to-end to a specific MI case study (say, induction heads in GPT-2, or the Othello world-model). This would have been the obvious demonstration, and its absence likely reflects scope discipline — the paper is already conceptually dense — rather than inability. But until someone runs the framework against a real interpretability finding and shows it changes the conclusion, the principles remain normative rather than demonstrated.