ACCURACY RAISED
+46%
LLAMA COT
+85%
COT
+69%
ACCURACY RAISED
+52%
Imagine you're lost in a city and you have two options: follow a local's winding, narrated walking tour through side streets (Chain-of-Thought), or consult a compass that points straight at your destination and walk there in a tenth of the steps. This paper says the compass already exists inside the model — you just need to read it. The committed claim: the representation space of a direct-answer prompt already contains a per-question direction that, when applied as a single rotation to the last hidden state before generation, produces the correct answer more reliably than Chain-of-Thought prompting — and at a fraction of the token cost. Across 25 tasks, mean accuracy jumps from 46% to 85% on LLaMA 3.1 8B (CoT gets 69%) and from 52% to 86% on Qwen 2.5 7B (CoT gets 77%). The method uses roughly one-tenth of CoT's generation tokens. The mechanism works in three steps. First, the authors run CoT on each question and label each intermediate hidden state as 'recovering' (patching it into the direct prompt yields the correct answer) or 'failed.' A linear probe confirms the model encodes these two classes in separable subspaces — this isn't noise. Second, for each question, they compute a direction from the centroid of its failed states toward the centroid of its recovering states. Third, they rotate the direct prompt's final hidden state once along that direction before generation begins. That's it: one rotation, then generate. The ladder placement is strong but comes with a caveat. CoT is the natural baseline here — not another activation-steering method — and the paper beats it decisively on aggregate accuracy across 25 diverse tasks on two model families. But the per-question direction is built from that question's own CoT hidden states, meaning you still need to run CoT to construct the steering vector. The method as presented is an analysis tool revealing what models encode, not yet a drop-in inference speedup. The authors are transparent about this — they frame the contribution as evidence that CoT's accuracy is achievable via shorter paths, not as a deployed system. Architecturally, this sits in the activation-steering / representation-engineering family — relatives include contrast-consistent search (CCS), representation engineering (RepE), and inference-time intervention (ITI). The key structural choice is linear: a single rotation in hidden-state space, justified by the linear probe's success at separating recovering from failed states. The compute property being leaned on is the geometric regularity of transformer residual streams — the claim that task-relevant information lives on low-dimensional linear subspaces is load-bearing here. Integrity is decent but not airtight. The evaluation covers 25 tasks across two model families (LLaMA 3.1 8B, Qwen 2.5 7B), which is broader than typical mechanistic interpretability papers. The linear probe separation confirms the direction is real, not a statistical artifact. However, the recovering/failed labeling uses the same CoT outputs that define the direction — so the validation is somewhat circular. There's no pre-registration, no independent replication, and no code availability is mentioned. The benchmarks appear standard (the paper mentions 25 tasks but the abstract doesn't name them individually), which is a mild flag. The milestone question is about generalization. Right now the direction is per-question, computed from that question's own CoT trace. The obvious next number to watch is whether a task-level or universal direction — computed from a training set of questions — transfers to unseen questions without running CoT at inference time. If that works at, say, 80%+ accuracy on held-out questions, this becomes a real inference-time method, not just an analysis. The gap is probably one paper away — the linear structure strongly suggests transferable directions exist.
The analogy
You're lost in a city. Option A: follow a local's narrated walking tour through winding side streets (Chain-of-Thought — works, but slow and sometimes leads you astray). Option B: use a compass built from noting which streets on the tour pointed toward your destination and which didn't, then walk straight there. The compass was hidden inside the tour all along.
Compass bearing
Average direction of tour segments that pointed toward the destination minus those that pointed away
Steering vector
Centroid of recovering CoT hidden states minus centroid of failed states, applied as rotation to direct-prompt representation
Honest assessment
This is a sharp mechanistic insight — CoT's accuracy lives in a linear direction that can be extracted and applied directly — demonstrated convincingly across 25 tasks and two model families. The limitation is honest and load-bearing: you still need CoT to build the per-question direction, so this is currently an analysis result, not an inference speedup. The real payoff comes in the obvious sequel (amortized directions), which the linear separability results strongly predict will work. If you care about why CoT works and whether we can do better, this is one of the cleaner contributions in that space.
What Is CLAIMS?
73
/ 100
BedrockNews applies a structured analytical framework called CLAIMS to academic papers and technical research releases.
How CLAIMS worksClaim
4/5
Ladder
4/5
Architecture
4/5
Integrity
3/5
Milestone
4/5
Successor
3/5
What is this paper actually committing to?
Real category claim, not an increment. Argues CoT's accuracy is reachable without CoT's cost — a structural insight about what LLMs encode, demonstrated across 25 tasks and two model families.
The field fight
Is Chain-of-Thought reasoning genuinely necessary, or do LLMs already encode the answer and just need the right activation path?
Side A
CoT is essential — it enables multi-step computation the forward pass can't do alone
Held by: Wei et al. 2022, Feng et al. 2024, complexity-theoretic arguments
Side B
Models already know the answer; CoT is an expensive detour through representation space that could be shortcut
Held by: Activation steering (CCS, RepE), latent reasoning, this paper
Milestone tracker
Unlocks: Drop-in CoT replacement: same accuracy, 1/10 tokens, no per-question CoT trace needed at inference
Current
85%
accuracy
Per-question direction (LLaMA 3.1 8B)
Next
80%+
accuracy
Amortized direction on unseen questions
~1 paper / 6-12 months — linear structure strongly suggests transferability
Verified Facts (5)
Direct prompt mean accuracy raised from 46% to 85% on LLaMA 3.1 8B (CoT: 69%)
Direct prompt mean accuracy raised from 52% to 86% on Qwen 2.5 7B (CoT: 77%)
Evaluated across 25 tasks
Method uses approximately one-tenth of CoT's tokens
A linear probe shows the model encodes recovering and failed CoT states differently
Practical Implications (3)
Provides evidence that CoT overhead may be eliminable: models encode correct-answer directions that don't require sequential token generation to access
Opens a path toward training-time alignment of model states with task-solving directions, potentially yielding models that need less inference-time reasoning
Linear probe methodology offers practitioners a diagnostic tool: check whether your model's CoT hidden states separate into recovering/failed clusters to assess steering potential
More latest news
Read all