The geometry of self-verification in task-specific reasoning models

How do reasoning models check their answers in hidden states, not just in CoT text?

• Scenario: OpenAI o-series and DeepSeek R1 are trained with reinforcement learning, rewarded only for correct final answers, and generate long chains of thought. The authors cite a line of work arguing that CoT does not necessarily faithfully reflect internal computation, so they propose monitoring hidden states instead, treating self-verification as a key step.
• Why focus on CountDown: General reasoning skills are many. The authors choose CountDown (3 or 4 operands, using basic arithmetic operations, each number used exactly once to reach the target), which requires search and has the target number written in context, because attention heads can be expected to participate in checking, making analysis easier.
• Parsable trajectory: The authors state that the preference signal causes mode collapse and reduced output diversity. Here this makes R1CountDown stably generate a fixed-format CoT, marking each attempt as success or failure, allowing hidden states xValid and xInvalid at tvalid and tinvalid to be extracted.
• What this paper does: Using Qwen2.5-3B as the base, they train the task model R1CountDown following the DeepSeek R1-Zero recipe. Top-down, they find GLU weights encoding verification-related tokens; bottom-up, they find previous-token heads, and claim as few as 3 heads can disable self-verification; they then check similar components on the base model and DeepSeek-R1-Distill-Qwen-14B.

The introduction and Related Work discuss several categories of existing work. The authors do not treat most of these methods as systems to be beaten; they only note differences on a few named ones.

Reasoning models and CoT

• OpenAI o-series and DeepSeek R1 (Guo et al., 2025): use reinforcement learning to reward correct final outputs, behaviorally generating long CoT. The authors use them to explain where the analyzed model behavior comes from.
• Arcuschin et al. (2025), Lanham et al. (2023), Turpin et al. (2023): the authors cite these works to support the premise that CoT does not necessarily faithfully reflect internal computation. The paper does not recount their measurement protocols one by one.
• Kirk et al. (2024) et al.: preference training can cause mode collapse and reduce output diversity. The authors use this phenomenon as a condition for obtaining analyzable trajectories, not as a defect to be eliminated.

Linear interpretable representations

• Mikolov et al. (2013), Park et al., Zou et al. (2023) line: many concepts are linear directions, and adding or subtracting them can change behaviors such as refusal, sycophancy, toxicity, or user representation. Burns et al. and Li et al. (2023) study truthful directions; Orgad et al. (2024), Ferrando et al. (2024) use representations to judge errors or inability to recall.
• Venhoff et al. (2025) decode directions related to uncertainty and self-correction; Zhang et al. (2025) find that models know when they have reached an answer. The authors state that this paper is consistent with these findings but goes further to break down how self-verification is implemented.

Circuits and component channels

• Olah et al. (2020) view forward computation as circuits. Dunefsky et al., Ameisen et al. (2025) use Transcoder or Cross-layer Transcoder for sparse approximation; Lindsey et al. (2025) use it to analyze fact recall, addition, and whether CoT is faithful. The authors state that this paper does not do additional sparse approximation when localizing verification components.
• Olsson et al. (2022) discussed previous-token heads in induction heads; Elhage et al. (2021), Merullo et al. (2024) inter-layer communication scores are rewritten by the authors into scores for attention heads and GLUs. This is a method borrowing; the paper does not criticize these works.

Baselines

• The intervention control is random attention heads: each time 33 or 3 are sampled, each averaged over 5 runs, on the CountDown task. The intervened models are R1CountDown, Qwen2.5-3B, and DeepSeek-R1-Distill-Qwen-14B.

The authors place this paper in the line of looking at self-verification through hidden states rather than CoT text: structured trajectories provide analytical points, GLU directions and previous-token heads provide components that can be causally ablated.

The paper trains the task model R1CountDown, then squeezes from both the late-layer GLU and the early-layer attention to locate the subspace used for self-verification.

Task and structured trajectories

• The base model is Qwen2.5-3B, and the training setup follows DeepSeek R1-Zero. There are two rewards: an accuracy reward for the final answer being correct, and a format reward for putting the reasoning inside think tags. The hyperparameters in Appendix A include an actor learning rate of 1e-6, a KL coefficient of 0.001, and a maximum response length of 1024.
• CountDown gives 3 or 4 operands and a target number, requiring each number to be used exactly once and the target to be reached using basic arithmetic. Mode collapse makes the model always enumerate attempts and mark each attempt as successful or not reaching the target. The authors call the moments just before these two kinds of marks are produced tvalid and tinvalid, corresponding to hidden states xValid and xInvalid; the moment when the target number is written in the prompt is tans.

Top-down: probes and GLUOut

• The authors write the GLU output as a weighted sum of the rows of Wout, call these rows GLUOut vectors, and the activation components GLU neurons.
• They train linear probes on the hidden states at each layer just before the model is about to give a success or failure mark, with the label being whether that step's expression has been treated as solved. The training set is N=327,680, fitted with gradient descent. Appendix C writes batch size 8, learning rate 1e-4, weight decay 0.01, and stopping when validation loss does not decrease for 10 epochs.
• They use the cosine similarity between the probe direction and GLUOut, taking top k=50 per layer and per direction, called GLUValid and GLUInvalid. The authors then embed these vectors back into the vocabulary and see which tokens they raise; the reverse vectors are also checked with nearest neighbors. Setting the selected vectors to 0 is the approach used in the later causal checks.

Bottom-up: previous-token heads

• The authors' entry assumption is that the target number is already in the context, and the model can compare the current expression with the target at tans. They denote heads with at least 10% attention on tans as previous-token heads, with symbol APrev. Such heads have been discussed before in induction heads.
• The way to ablate them is to zero out their WO. Their relationship with GLU is connected by the scoring in the next section, not just by looking at attention patterns.

Connecting the two sides with receptive fields

• The receptive field of a single GLU neuron is the set of inputs that activate it, S_k={x∈ℝ^d∣ f^k(x)>0}. The authors view the intersection of a set of GLUValid receptive fields as a polytope in activation space, saying that when the hidden state enters this region, the verification-related GLU weights are turned on.
• The score of each previous-token head is how strongly its OV circuit drives these neurons on average: score(A,GLU_{Valid})=1/N∑_iσ(Wⁱ_gateW_OWV)·(Wⁱ_upW_OWV). This adapts the inter-layer composition score to the Wgate and Wup of attention heads and GLU. They ablate one by one from high to low score until the intervention reaches the standard of Section 4.4, obtaining the subset AVverif.
• The authors state that this scoring ignores head-head interactions and also ignores inter-layer LayerNorm and GLU transformations. Appendix F says changing N or changing the ordering yields different subsets. The authors accordingly claim that what they find is a necessary component of verification, not a complete circuit.

Intervention definition and transfer to two other models

• The causal test uses samples where the model can originally find and verify the correct solution. Each generation is 100 tokens; each time it is about to mark an attempt, the specified weights are ablated. Intervention success means that within 100 tokens the correct solution has already been written but is not verified as successful. If the correct attempt is first marked invalid and then it still claims to have found a solution, it is recorded as partial success. In the ICL setting, if the generation no longer follows R1CountDown's format, it is recorded as out-of-range.
• The ablated sets are defined as: 50 GLUValid per layer in layers 18–36; or additionally 50 GLUInvalid per layer; or all APrev; or AVverif. The control is randomly drawing the same number of heads.
• The base Qwen2.5-3B and DeepSeek-R1-Distill-Qwen-14B are not re-trained with RL. The authors give 5 demonstrations that solve CountDown in R1CountDown's format and do in-context learning for CountDown. R114B has no directly usable probe, so the authors use EMB2EMB to transfer R1CountDown's probe over; Appendix G writes randomly sampling N=100,000 tokens to learn a linear mapping. R114B's attention threshold uses 5% to compensate for the longer context; Appendix I additionally tries 10% and 2.5%.

Experimental setup

• Models: The analyzed model is R1CountDown; the same set of interventions is also applied to the base Qwen2.5-3B. On DeepSeek-R1-Distill-Qwen-14B, components are re-localized before intervention. The paper does not state that another reviewer model was used.
• Data: The task is CountDown. LogitLens uses 300 items, comparing xValid and xInvalid. Causal intervention uses another 300 samples for which a correct solution had originally been found and verified, generating 100 tokens each. The paper does not state whether these two sets of 300 are the same, nor the number of training problems.
• Probes: Training N=327,680. For validation accuracy, the main text gives N=512, while Appendix C gives validation size 256. Figure 6 gives per-layer curves; the main text only says it is usually above 90% after the first few layers.
• Intervention targets: GLUValid is 50 vectors per layer for layers 18–36, 0.45% of all GLU vectors; adding GLUInvalid makes it 0.9%. APrev in R1CountDown is 33/576 (5.7%), with the threshold being at least 10% attention to tans. AVerif is L17H14, L17H11, L17H10. Random baselines are 5 each.
• R114B: ICL with 5 demonstrations; previous-token threshold 5%, yielding 552/1920; the same scoring yields 351 heads. Appendix I changes the threshold.
• Metrics: intervention success rate, partial success, out-of-range; LogitLens next-token probability; probe validation accuracy. The paper does not state multi-seed repetition.

Main results

Swipe horizontally to read the table

Condition	Metric	Value	Source
R1CountDown, correct solution, no intervention, L36	P(this)	0.98	Figure 1(a), Figure 5
R1CountDown, correct solution, GLU off, L36	P(this)	0.54; see also 0.70	Figure 1 gives 0.54; main text and Figure 5 give 0.70
R1CountDown, unsolved, L36	P(not)	0.25	Figure 1(b)
R1CountDown, after the first few layers	probe validation accuracy	usually above 90%	Section 4.1, Figure 6
R114B, attention threshold 10%	intervention success rate	0.4%	Appendix I
R114B, attention threshold 10%	partial success rate	25%	Appendix I
APrev of R1CountDown	number of heads	33/576	Section 4.2, Figure 2
R114B, threshold 5%	APrev; AVerif	552/1920; 351	Section 5, Figure 4(b)

• The authors state that after turning off only the GLU, success-class tokens still appear and this is still top-1, so the GLU cannot fully explain self-verification. The Figure 1 caption and the figure consistently give a drop to 0.54 at L36; the main text gives 0.70, consistent with the L36 row extracted from Appendix Figure 5. That appendix caption also says the two become 0.51 versus 0.49 at the last layer, which does not match the this (0.70) extracted from the same figure. In Figure 1(d), where attention heads are turned off, the authors state that success-class tokens no longer appear and the distribution is close to the unsolved (b).
• The authors state that turning off the 33 previous-token heads, or the 3 AVerif heads among them, brings the intervention success rate close to complete: the model marks the already-written correct solution as invalid and keeps generating. Figure 2 includes baselines with random 33 heads and random 3 heads; the main text gives neither the baseline percentages nor the breakdown percentages for Success and Partial Success.
• The authors state that turning off both GLUValid and GLUInvalid intervenes more than turning off only the former. The explanation is that Qwen2.5-3B uses SiLU, inactive neurons take small negative values, and without zeroing them the opposite direction of GLUInvalid is added into the success direction. The last 7 rows of Table 2 are reverse nearest neighbors of these vectors, which the authors state often correspond to antonymous verification words; they also state that xValid more often contains English tokens and xInvalid more often contains Chinese tokens.

Base model and R114B

• Both models can perform CountDown in the R1CountDown format under 5-shot ICL. The authors state this is consistent with existing results that reasoning ability is often already present in pretrained models. On the base model, the same set of heads from Section 4.4 is used directly, without a new search.
• On the base model, the authors state that APrev still achieves a near-complete (including partial) intervention rate, while the 3 AVerif heads have a lower success rate. The authors hypothesize that reinforcement learning strengthened a verification mechanism already present in the base model, so after RL a few heads can control self-verification. The bar heights in Figure 4(a) are not given in the main text.
• After re-searching on R114B, the authors state that interventions are mostly partial success: the correct solution is first marked invalid, then the model changes its mind. A subset of 351 heads has a higher success rate than the larger APrev set; the authors believe not all previous-token heads or their interactions contribute to self-verification, and say this suggests a larger verification circuit. Figure 4(b) likewise gives no exact bar readings.

GLU activations and steering

• Figure 3 plots the same GLUValid vector before and after intervention as adjacent bars. The authors state that after turning off the 3 AVerif heads, most activations drop to near 0 or become negative. The figure does not label specific activation values.
• Appendix E adds the probe direction to layers 24–36, scaled by 20 after normalization. The authors state this is not a core experiment and give only a qualitative example: after adding this direction, the model says an equation that has not yet reached the goal is already solved. The paper does not report the success rate of steering.

Other ablations and analyses

• Appendix F uses complete success including partial success as the criterion: Eq. 8 requires 3 heads at N=200, 15 at N=50, and 12 at N=300; ranking by attention density to tans requires 8; ranking by similarity to Wgate, Wup or the probe requires 17 each.
• The Eq. 8 (N=100) row in Appendix F says 100 heads but lists only 6 names; the columns may be misaligned, so this number is not used here. The start of the same section says 25 previous-token heads, inconsistent with the 33 in the main text.
• Appendix I: the two ratios at the 10% threshold are in the table above; the authors state this threshold makes the set too tight. At 2.5% the set is too large and out-of-range is higher; the paper does not give percentages for that setting. The number of GLUValid vectors converges after 200, and this experiment uses 200.
• Table 4 in Appendix H lists GLUOut nearest neighbors related to verification in R114B. The authors state they are similar to Table 2 and therefore believe R114B has a similar verification subspace. The paper does not repeat the English/Chinese observation for R114B.

Limitations and future directions noted by the authors

• Not a complete circuit: The authors state they do not claim to have found the complete verification circuit, only key components of verification (Limitations; Section 4.3 makes the same claim).
• Task scope: The authors reiterate that the study concerns a specific task that allows context-based checking, and state that not all reasoning tasks have this property (Limitations).
• Prior-based verification: The authors state that many tasks may rely on general knowledge for prior-based verification; they speculate that a similar subspace would still be used, but where it appears is less obvious (Limitations).

Experimental coverage

• The analyzed models are R1CountDown, Qwen2.5-3B, and DeepSeek-R1-Distill-Qwen-14B. The latter two use CountDown ICL with 5 demonstrations, without re-running RL.
• LogitLens and causal interventions each use 300 samples; interventions generate 100 tokens per sample. Random head baselines run 5 times each. Probe training uses N=327,680; the validation sample size is 512 in the main text and 256 in Appendix C.
• R1CountDown's previous-token threshold is 10%, yielding 33/576; AVerif is 3 heads in layer 17. R114B at the 5% threshold is 552/1920, with a scored subset of 351; Appendix I also reports 10% and 2.5% thresholds.
• The paper does not report R1CountDown's training steps, CountDown problem count, or task accuracy, nor the exact success rates for each bar in Figure 2 and Figure 4.
• Steering is only given as a qualitative example in Appendix E. The paper does not report probe agreement with human judgment, nor multi-seed repetition.

R1CountDown is trained from Qwen2.5-3B using DeepSeek R1-Zero's recipe, only on CountDown. The authors are not concerned with further raising task scores, but with where self-verification resides in the weights, potentially allowing one to look at hidden states instead of only reading CoT.

• Analytical points: mode collapse makes the trajectory format fixed. The model tries repeatedly and moves to different markers when solved or unsolved; the authors read hidden states at these moments.
• Two analyses: late-layer probes can linearly separate xValid from xInvalid; the nearest neighbors of GLUOut are often tokens like success or incorrect, and the reverse are often antonyms. More decisive for behavior are previous-token heads that attend to the target numbers in the prompt.
• Ablation experiments: in 300 generations that originally verified successfully, ablating the 33 such heads, or the L17H14, L17H11, L17H10 selected by GLU receptive field alignment, the authors state the model marks the correct solution as invalid and continues writing. Only zeroing GLUValid cannot achieve the same ablation.
• Two other models: on the base model, the same batch of previous-token heads can still intervene, but those 3 heads are weaker; the authors hypothesize that RL makes the existing mechanism more localized. On DeepSeek-R1-Distill-Qwen-14B, interventions with 552 heads or the 351 subset mostly stop at first marking wrong, then correcting; the authors believe the circuit is larger.
• Authors' boundaries: this provides necessary components for self-verification in a context-checkable task, not a complete circuit, and cannot be directly transferred to tasks that require checking against general knowledge.