Read the paper
Just sort it! A note on the LP behind rank-one anti-loop edits
on Zenodo
SORTED EDIT SCORES
+67.0%
GSM8K AGAINST
+67.4%
EDIT
+9.5%
POINTS
+95%
Imagine you're trying to push a stuck revolving door. You could hire an engineer to calculate the precise angle, force vector, and hand position that maximizes torque — or you could just shove it at the eight most obviously loose panels with equal force. This paper asks: does the engineer's calculation actually matter, or is the door loose enough that any sensible shove works? The claim, stripped bare: when you apply a rank-one weight edit W' = W + uvᵀ to remove thinking loops from a small reasoning model, the optimal neuron-activation vector v — which is formally the solution to a sparse linear program — can be replaced by a trivial sort-and-share heuristic with no meaningful loss in accuracy. On Qwen3.5-0.8B across MATH-500 and GSM8K (1000 problems total), the sort-based edit scores 67.0% versus the LP edit's 67.4%, against a 9.5% unedited baseline. The paired difference of -0.4 points falls comfortably inside a pre-declared 5-point non-inferiority margin, with a 95% CI of [-2.9, +2.1]. The mechanism is clean. The sort selects the 8 neurons whose activation has the smallest coefficient of variation — meaning they fire most consistently across the data — and splits the LP's total push equally among them. The key empirical finding is that the neurons the sort picks overlap substantially with the LP's support, and together they carry most of the LP solution's mean push. The edit acts, in practice, as a nearly constant bias along the loop direction u. This sits inside the weight-space programming paradigm where you treat network weights as decision variables and use optimization machinery to find targeted edits. The LP formulation is elegant but introduces a dependency on an LP solver, sparsity structure, and careful tuning. If a sort achieves the same result, the entire LP apparatus is engineering overhead for this particular application. That's a meaningful simplification — it makes the edit cheaper, faster, and easier to audit. The integrity story is unusually strong for a 4-page note. The non-inferiority margin was declared before the runs. Three additional sampling seeds confirm the result (-0.6 and -0.8 point gaps, both with CIs excluding -5). Data, code, and protocols are deposited on Zenodo. This is exactly how you validate a simplification claim: you pre-commit to the margin, run the comparison, and report all seeds. The important limitation is scope: this is one model (Qwen3.5-0.8B), one type of edit (rank-one anti-loop), and one pair of benchmarks. The finding that the edit acts as a nearly constant bias along u is suggestive — it implies the activation landscape is flat enough that precise optimization doesn't buy you much — but we don't know if this holds for larger models where neuron activations may be less uniform. The authors don't claim it generalizes, to their credit. The practical upshot is immediate for anyone implementing weight-space programming edits: try the sort first. If it works, you've eliminated the LP solver from your pipeline. The deeper theoretical implication — that the neuron activation space for these edits may be benign enough to make optimization unnecessary — is the more interesting thread to follow.
The analogy
You're trying to push a stuck revolving door. You could compute the optimal force distribution across all panels, or you could just push equally on the eight loosest ones. This paper shows the door is loose enough that the simple shove works — the LP is overkill because the activation landscape is nearly flat along the edit direction.
Revolving door
Push equally on the loosest panels instead of computing optimal force distribution — the door is loose enough that precision doesn't matter
Sort heuristic
Split push equally among the 8 lowest-CoV neurons instead of solving an LP — the activation landscape is flat enough that optimization doesn't matter
Honest assessment
A clean, honest, 4-page result that does exactly what it claims and no more. The sort heuristic ties the LP on one small model with pre-declared non-inferiority margins and open code — that's a real finding, well-validated. The scope limitation (single 0.8B model) is the load-bearing caveat: the flatness property that makes the sort work may not hold at scale, and that experiment hasn't been run. Practitioners implementing anti-loop edits on small models can drop the LP today; everyone else should wait for the scale test.
What Is CLAIMS?
73
/ 100
BedrockNews applies a structured analytical framework called CLAIMS to academic papers and technical research releases.
How CLAIMS worksClaim
3/5
Ladder
4/5
Architecture
4/5
Integrity
4/5
Milestone
3/5
Successor
4/5
What is this paper actually committing to?
Useful simplification, not a new capability. Shows the LP in rank-one anti-loop edits is unnecessary on a small model — the sort heuristic ties it within a pre-declared margin.
The field fight
Do targeted weight edits in LLMs require formal optimization, or is the landscape benign enough for heuristics?
Side A
Formal optimization (LP, gradient-based) is needed to find precise edit vectors — the landscape is complex enough that heuristics will fail at scale
Held by: Weight-space programming (original LP formulation), ROME, MEMIT
Side B
The activation landscape for simple edits is flat enough that sorting or simple heuristics suffice — optimization is overhead
Held by: This paper
Milestone tracker
Unlocks: If sort works at 7B+, LP solvers can be dropped from anti-loop edit pipelines entirely
Current
0.8B
params
Qwen3.5-0.8B (this paper)
Next
7B+
params
Sort heuristic at scale
Straightforward experiment — months, not years, given compute access
Verified Facts (5)
Qwen3.5-0.8B sorted edit scores 67.0% on 1000 problems (MATH-500 and GSM8K) against 67.4% for the LP edit and 9.5% for the unedited model
Paired difference is -0.4 points, with 95% confidence interval [-2.9, +2.1], inside a non-inferiority margin of 5 points declared before the runs
On three further sampling seeds the difference is -0.6 points, interval [-2.0, +0.8], and -0.8 points, interval [-2.3, +0.6]
The sort takes the 8 neurons whose activation has the smallest coefficient of variation and gives them equal shares of the push
Data, code and protocols: https://doi.org/10.5281/zenodo.24286137
Practical Implications (3)
If you're implementing rank-one anti-loop edits on sub-1B parameter models, you can replace the LP solver with an 8-neuron sort-and-share heuristic with no accuracy loss
The finding that the edit acts as a nearly-constant bias suggests the activation landscape for these edits is benign — worth checking in your own pipeline before reaching for heavy optimization
Code and protocols on Zenodo mean you can replicate and extend to your model in a day
More latest news
Read all