Interpretability Tools Can Be Shipped Between Language Models Like Spare Parts

Linear maps between residual streams, fit on unlabeled text alone, transfer refusal directions and probes across 110 model pairs with up to 102% recovery. Interpretability work may not need to start from scratch for every new model.

Imagine you've spent weeks calibrating a voltmeter to read a specific circuit board — learning which pins carry which signals, where the voltage spikes mean trouble. Now someone hands you a completely different board. Normally you'd start over. But what if the two boards, despite different layouts, route their signals through geometrically similar patterns? A simple wiring adapter — a linear map — could let your old voltmeter read the new board, no new calibration needed. That's what this paper demonstrates for language model internals. The committed claim: interpretability artifacts — refusal directions, correctness directions, sentiment probes — found inside one model can be transferred to a different model via a linear map between residual streams, fit using only unlabeled prompts. No labeled data needed for the transfer itself. Across 110 model pairs spanning five families (1B to 72B parameters, base and instruction-tuned), the transferred tools recover meaningful fractions of the target model's own internally-discovered tools. With a domain-matched unlabeled prompt set, refusal steering recovers 102% of the target's own refusal effect, and correctness probes recover 88–98%. Control directions recover at most 1%, confirming specificity. The key architectural insight is that the transfer is a plain affine (or linear) map between residual-stream activations, fit by least-squares on unlabeled prompt pairs run through both models. No gradient surgery, no labeled interpretability data, no model-specific engineering. The theoretical contribution is a proof that such a map works whenever two models correspond linearly on the data it covers — and a derived label-free "coverage score" that predicts how well a transferred tool will work. This coverage score tracks recovery across more than 16,000 experimental runs and retroactively explains why prior cross-model mapping work (e.g., stitching, model merging) failed across domains: the maps were fit on data that didn't cover the tool's operating region. On the ladder: there's no direct competitor doing exactly this task, but the implicit baselines are (a) training a new probe from scratch on labeled data inside the target model, and (b) using unsupervised structure in the target's own representations. The transferred tools outperform anything the target's unlabeled representations alone can yield — a strong result. The 102% refusal recovery number is striking: the transferred direction sometimes steers more effectively than the target model's own internally-discovered direction, likely because the source model's direction was better optimized. The 16–77% range for naive (non-domain-matched) maps and 88–98% for domain-matched maps gives an honest spread. Integrity is solid for a mechanistic interpretability paper. Causal validation (addition and ablation of steering vectors) goes beyond correlational probing. Five seeds per experiment, 110 model pairs, and 16,000+ runs give real statistical mass. Control directions serve as specificity checks. The coverage score is derived from a theorem and validated empirically, not post-hoc curve-fit. The main limitation: all validation is internal to language models — no external ground truth for "refusal" or "correctness" beyond the behavioral definitions the field uses. The paper is honest about the linear-correspondence assumption being an empirical regularity, not a guaranteed property. The milestone that matters is whether this scales to more complex interpretability artifacts — circuits, not just directions. A single direction (refusal, sentiment) is the simplest possible interpretability unit. The field needs to know whether multi-feature circuits or sparse autoencoder dictionaries transfer the same way. If they do, interpretability becomes an amortizable investment rather than a per-model sunk cost. That's the difference between interpretability being a cottage industry and being infrastructure. The obvious experiment not run: transferring sparse autoencoder features or multi-node circuits, not just single directions. The authors almost certainly know this is the next question — the paper's framing as "interpretability is not model-specific" is more ambitious than single-direction transfer alone can support. My read: they're building toward it, this paper establishes the theoretical and empirical foundation, and the SAE/circuit transfer paper is next. Compute constraints likely also played a role — 16,000+ runs across 110 pairs is already substantial.