Imagine you're a chess player who's memorized a dozen openings. Two different move orders can reach the same position — the Sicilian via 1.e4 c5 or via a transposition from the English. You know the position is identical, but your mental preparation for each path feels different: different threats you've previewed, different tactical patterns primed. A weaker player might not even notice the difference. A stronger player recognizes the equivalence AND keeps richer, more distinct mental representations of how they got there. That's the core mechanism this paper measures inside language models. The committed claim: models that correctly solve reordered math problems represent the different orderings MORE distinctly — not less — in their internal activations. This is measured via a metric called permutation signal-to-noise ratio (SNR), which quantifies how separable different orderings are in representation space relative to across-problem variation. Across 16 models from 1B to 8B parameters on synthetic function-composition tasks, permutation SNR positively rank-correlates with accuracy, hitting Spearman ρ = 0.86. This is a conceptual contribution, not a SOTA benchmark race. The authors aren't proposing a new architecture or training method. They're proposing a new distinction: answer invariance (getting the same output regardless of rule order) versus representation invariance (having identical internal states regardless of rule order). The paper argues these are genuinely different properties, and that the field has been implicitly conflating them. The finding that they anti-correlate — better performance comes with LESS representational invariance — is the surprising bit. The experimental setup is deliberately synthetic. Function-composition problems where a set of rules like f(x) = 2x+1, g(x) = x², h(x) = x-3 are presented in every possible permutation, and the model must chain them to compute the final output. The same problem under different orderings has the same correct answer, so accuracy should be invariant. But internal representations need not be — and this paper measures exactly how much they aren't, and whether that 'how much' predicts performance. The ladder is tricky because this isn't competing against prior methods on a task. It's introducing a new diagnostic metric. The closest prior work is the mechanistic interpretability literature (probing, representation similarity analysis) and the broader line on compositional generalization in neural networks. The permutation SNR metric itself is novel to this paper, so there's no direct baseline to beat. The authors test across a wide sweep of model families (16 models, 1B–8B) rather than depth-testing one model, which is the right call for an observational finding. Integrity is solid for what this is — a synthetic, controlled study where the authors designed the data distribution to isolate exactly the variable they care about. The risk of cherry-picking is low because the synthetic setup is transparent and the metric is well-defined. The main integrity concern is generalization: synthetic function-composition is a controlled proxy for 'mathematical reasoning,' and whether the finding transfers to natural math problems (GSM8K, MATH, etc.) is untested. The correlation is rank-based (Spearman), which is appropriate for ordinal comparisons across models but doesn't establish causation. The milestone question is really about whether this diagnostic becomes a standard tool. If permutation SNR or something like it becomes a routine probe in the interpretability toolkit — the way loss landscape sharpness or attention entropy have — this paper will be cited as the origin. The next concrete step is demonstrating the same correlation on natural math benchmarks, ideally with models of matched architecture but different training regimes. If the correlation breaks on real-world tasks, the finding is a curiosity about synthetic data; if it holds, it's a genuine insight about how models do math. The obvious next experiment is running this analysis on natural math datasets (MATH, GSM8K, competition problems) where rule reordering can be naturally induced by rephrasing problem setups. The honest read on why it wasn't run: the authors chose synthetic data precisely for experimental control, and natural-language reordering introduces confounds (different phrasings change more than just order). This is a (a) principled scoping decision, not a gap being hidden.
