Abstention directions in LLMs: mechanistic interpretability of abstention in query ambiguity, knowledge boundaries, and safety alignment
Abstention Direction in Large Language Models: Mechanistic Interpretability of Abstention Across Query Ambiguity, Knowledge Limits, and Safety Alignment
The authors generalize the refusal direction to three abstention perspectives, Q, MK, and HV, and state that mid-layer intervention in Qwen3-4B can control abstention bidirectionally and that the three directions share some components.
- The causes of LLM abstention can be divided into three categories—Query ambiguity (Q), model knowledge boundary (MK), and human values (HV)—but prior internal representation analyses targeted only the HV refusal direction, and whether the three perspectives share structure is unknown.
- The authors generalize the refusal direction framework to three perspectives: using Kimi-K3, they construct counterfactual pairs that share a template and flip only the slot carrying the abstention trigger, take the difference of the mean activations of the two polarities and normalize it to obtain the abstention direction, and then use activation steering to add or subtract this direction at inference time to intervene.
- On Qwen3-4B, adding or subtracting the direction at middle layers can respectively increase or suppress abstention across the three perspectives, and the effect is stronger than the source-independent baseline; the cosine similarity between directions is positive (e.g., MK–HV is +0.190), and the authors state that the three perspectives share part of the direction components; the random-direction control has no effect.
- The authors state that validation was performed only on two models, Qwen3-4B and Llama-3.2-3B-Instruct, and on English data; direction computation and evaluation are in the same distribution, and out-of-distribution generalization was not tested; judging relies on gpt-oss-120b and only detects abstention, without evaluating output quality; Kimi-K3 self-generation and self-verification may introduce bias.
- Evaluated models
- Qwen3-4BLlama-3.2-3B-Instruct
- Benchmarks
- AmbigQANatural QuestionsFictionalQAAdvBenchAlpaca
- Metrics
- AROARcosine similarityFisher discriminant ratio
Koki Itai and other researchers extend the refusal-direction framework to three causes of LLM abstention—query ambiguity, knowledge boundaries, and safety alignment—and find that the three abstention directions share one directional component. Methodologically, the paper constructs, for each cause, counterfactual pairs that share a template and differ only in one abstention-inducing factor, and normalizes the difference of mean activations at the two poles to obtain the abstention direction. The authors note that previous analyses of internal representations computed refusal directions using only harmful and harmless queries, focusing on a human-values perspective, and that no study has jointly analyzed the three causes. Activation steering experiments show that intervening at intermediate layers can increase or decrease abstention across the three scenarios, and that steering produces cross-perspective effects.
Paper analysis
What problem does this paper attempt to solve?
The authors generalize the refusal direction from HV to three abstention perspectives, Q, MK, and HV, and analyze whether their internal representations share structure.
Core question: how the three causes of LLM abstention—Query ambiguity (Q), model knowledge boundary (MK), and human values (HV)—are encoded in internal representations, and whether they share a common structure; no prior work has analyzed them jointly.
- Problem setting: Abstention refers to a model refusing to answer when uncertain or when there is a safety risk; following the classification of Wen et al. (2025), the authors divide the causes into three perspectives, Q, MK, and HV; abstention is regarded as a means to mitigate hallucination and improve reliability and safety.
- Existing gaps: The authors state that prior analyses of internal representations focused on HV, using a refusal direction computed from harmful/harmless queries (Arditi et al., 2024); Aaliyan (2026) covers only two perspectives, HV and MK, and the joint structure of the three perspectives has not been analyzed.
- Core observation: The authors also point out that counterfactual pairs for prior refusal directions came from mutually independent datasets (e.g., AdvBench and Alpaca), and distribution differences between the two polarities can cause the direction to mix in concepts other than abstention.
- Proposed approach: The authors propose a slot-flipped construction method in which each pair of samples shares a template and differs only in the slot carrying the abstention trigger; on this basis they compute an abstention direction for each perspective, use activation steering to verify its causal control over abstention behavior, and then analyze the geometric relationship among the three directions using cosine similarity and the Fisher discriminant ratio.
What related research exists?
Related work falls into four groups: activation steering, single-perspective analyses of abstention mechanisms, cross-perspective analysis, and source-independent construction as a baseline.
Activation steering and linear concept representations
- Arditi et al. (2024)—proposed the refusal direction: a single direction obtained from the mean activation difference between harmful/harmless queries, where c > 0 induces refusal and c < 0 suppresses refusal; this is the direct source of the framework in this paper.
- Turner et al. (2024) and Rimsky et al. (2024)—Activation Addition and Contrastive Activation Addition, intervening on activations along concept directions to control outputs; Stolfo et al. (2025) and El-Shangiti et al. (2025) apply them to instruction following and numerical comparison.
Single-perspective analyses of abstention mechanisms (Appendix A.2)
- Q: Zhang et al. (2025b)—used linear probes to find that query ambiguity is linearly encoded and concentrated in a small number of neurons, and intervening on these neurons can shift the model from direct answering to requesting clarification.
- MK: Chen et al. (2026) proposed Internal Confidence to estimate whether a query is within the knowledge boundary; Lindsey et al. (2025) found that a default refusal circuit is suppressed by "known entity" features, and erroneous triggering of this suppression leads to hallucination.
- HV: Wei et al. (2024) and Li et al. (2025) report that refusal behavior is concentrated in specific neurons or subspaces; Zheng et al. (2024) found that safety instruction prompts push input representations in the direction of increasing refusal probability.
Cross-perspective analyses (Appendix A.3)
- Son et al. (2026)—compared knowledge-type refusal and safety-type refusal (roughly corresponding to MK and HV) and found that the two directions share principal components, with type-specific components mainly in deeper layers; the paper notes that it does not cover Q and that the steering experiments only perform addition.
- Joad et al. (2026)—analyzed multiple refusal categories and stated that the directions of the categories differ geometrically but steering shows a similar refusal–over-refusal trade-off; the paper notes that it does not list MK as a separate category.
- Aaliyan (2026)—analyzed two directions, safety refusal and epistemic abstention; the paper states that a unified intervention analysis of the three perspectives is still missing.
Baselines and benchmarks: The baseline is source-independent construction (following Arditi et al.'s approach, e.g., AdvBench as D+ and Alpaca as D−, Appendix C); data come from AmbigQA, Natural Questions, FictionalQA, AdvBench, and Alpaca; the evaluation protocol follows AbstentionBench (Kirichenko et al., 2025). Authors' positioning: using counterfactual pairs that isolate the abstention trigger, they perform unified bidirectional addition/subtraction intervention and cross-perspective analysis for Q, MK, and HV, which they state is not covered by prior work.
How does the paper address this problem?
For each perspective, counterfactual pairs are constructed that flip only the slot carrying the abstention trigger; the mean activation difference is normalized to obtain the abstention direction, and activation steering is then performed.
Overall approach: generalize the refusal direction of Arditi et al. (2024) into an abstention direction for each of three perspectives, use a slot-flipped dataset to isolate the abstention trigger, and then verify causal effects through activation steering.
Problem setting and direction definition
- For each perspective p ∈ {Q, MK, HV}, prepare a should-abstain set Dp+ and an answerable set Dp−, take the difference of the means of the two polarities of the last-token activation at layer ℓ and normalize: r̂p(ℓ) = rp(ℓ) / |rp(ℓ)|, where rp(ℓ) = x̄p(ℓ,+) − x̄p(ℓ,−). Normalization is because norms differ greatly across perspectives and layers (Appendix H), facilitating comparison.
- Each direction is computed from 64 counterfactual pairs; the implementation is based on TransformerLens, taking blocks.{ℓ}.hook_resid_pre, excluding the last layer (ℓ = 36).
Slot-flipped counterfactual pair construction
- Motivation: source-independent construction (e.g., AdvBench paired with Alpaca) lets the two polarities retain source distribution differences, and the direction may mix in concepts other than abstention (Section 4.1).
- Procedure (Appendix B): use Kimi-K3 in three steps—Phase 1 extracts replacement candidate expressions from the opposing-polarity source; Phase 2 replaces the abstention trigger in the query with a slot to obtain a template, then injects a phrase of the same type and opposite polarity to flip the polarity; Phase 3 applies rule-based filtering plus LLM verification.
- Slots for each perspective: Q flips qualifiers or topic entities (AmbigQA); MK replaces real entities with fictional entities (Natural Questions paired with FictionalQA); HV replaces harmful actions and objects with benign actions and objects (AdvBench paired with Alpaca). Final pair counts: Q 1,199, MK 767, HV 377 (Table 1).
Activation steering intervention
- At inference time, add or subtract the direction on the activation at layer ℓ: x′ = x + m · r̂p, where m > 0 amplifies abstention and m < 0 suppresses abstention; m corresponds to the coefficient c in Eq. 3 via m = c·‖rHV‖.
- Intervention magnitudes: for Qwen3-4B, |m| ∈ {2, 4, 8, 16, 32}, and the main text reports m = ±16, which has the largest effect; for Llama-3.2-3B-Instruct, because the unnormalized vector norm is about 10⁻¹ times that of Qwen3-4B, |m| ∈ {0.25, 0.5, 1, 2, 4, 8} is used instead (Appendix D).
Evaluation method
- Metrics: AR (the proportion of should-abstain queries that are actually abstained on, corresponding to abstention recall in AbstentionBench) and OAR (the proportion of answerable queries that are abstained on); high AR and low OAR are good.
- Determination: LLM-as-a-Judge, using gpt-oss-120b with the AbstentionBench evaluation prompt, performing only abstention detection and no correctness evaluation.
- Evaluation data: 128 pairs (256 items) per perspective of slot-flipped test pairs not involved in direction computation; each query is generated with 3 sampling seeds, and the mean ± standard deviation over 3 runs is reported.
What experiments did the paper conduct?
Layer-wise steering on Qwen3-4B: the slot-flipped directions bidirectionally control abstention across the three perspectives at middle layers, the effect is stronger than the source-independent baseline, and the cosine similarity between directions is positive.
Experimental Setup
- Models tested: main experiments on Qwen3-4B (L = 36, d = 2560, thinking mode; non-thinking mode in Appendix F); Llama-3.2-3B-Instruct results in Appendix D.
- Data: 64 pairs per perspective for direction computation; 128 pairs per perspective (256) of slot-flipped test pairs not used for direction computation for evaluation, with both directions sharing the same test set.
- Baselines: directions constructed source-independently (Appendix C, Q 1,199 pairs, MK 767 pairs, HV 377 pairs, equal in number to the slot-flipped ones); plus a random-direction control (Appendix E).
- Metrics and judge: AR and OAR; judge is gpt-oss-120b (reasoning effort medium, temperature 0.0), using the AbstentionBench evaluation prompt, performing abstention detection only.
- Repetitions: 3 sampling seeds per query, reporting mean ± 1 standard deviation; the random-direction control uses a single generation only.
- Decoding: Qwen3-4B thinking mode temperature 0.6, top-p 0.95, top-k 20 (Appendix G).
Main Results
Main results are in Figure 3 (Qwen3-4B thinking mode, m = ±16 layer-wise intervention); the paper presents them as curves and gives no layer-by-layer numerical table; the quantitative results that can be checked are as follows:
Result Value Conditions and source HV AR without intervention 1.0 Qwen3-4B, still 1.0 after addition (Section 6.1 and footnote 9) MK–HV direction cosine similarity +0.190 respective most effective layers ℓ = 19 and ℓ = 17 (Section 7.1.2) Q–MK direction cosine similarity +0.163 ℓ = 20 and ℓ = 19 (Section 7.1.2) Q–HV direction cosine similarity +0.092 ℓ = 20 and ℓ = 17 (Section 7.1.2) FDR peak (MK / HV) 1.15 / 2.65 ℓ = 23 / ℓ = 21 (Figure 8) Thinking/non-thinking mode trend correlation AR 0.94, OAR 0.93 m = ±16 six-panel average Pearson (Figure 16) - The authors state that the slot-flipped directions at middle layers show effects consistent with the intervention directions: addition raises AR for Q and MK and OAR for the three perspectives, subtraction lowers AR for the three perspectives; early layers often show text degeneration (× marks, degeneration rate > 10%) and are excluded from analysis.
- The authors state that the source-independent directions have smaller effects than slot-flipped across the three perspectives and both intervention directions (Figure 3b), indicating that the data construction method for direction computation affects intervention effects.
- HV's AR is already 1.0 without intervention and its OAR is already very small; the authors note that the effects of addition on AR and subtraction on OAR cannot be manifested.
Cross-perspective Effects and Geometric Analysis
- What was measured: intervening with one perspective's direction and evaluating on all three perspectives (semi-transparent lines in Figure 3a), and computing cosine similarity between directions (Figure 7) and FDR (Figure 8).
- Results: cross-perspective effects exist and are asymmetric—intervening on Q changes only MK's AR and does not affect HV, whereas intervening on MK or HV affects the other two perspectives; Figure 6 gives a qualitative case of hallucination induced by HV→MK subtraction intervention. MK–HV cosine similarity is roughly non-negative across layers, Q–MK rises with layer depth, and Q–HV falls.
- Authors' interpretation: the three directions are non-orthogonal and share common components; the larger MK–HV similarity is consistent with its bidirectional cross-perspective effect, and the smaller Q–HV similarity than Q–MK is consistent with "subtracting the Q direction hardly lowers HV's AR". On FDR, MK and HV rise at middle layers while Q does not; the authors attribute this to Q's polarity difference being limited to qualifiers and topic entities, which may be hard to separate in activation space; the layers where interventions take effect roughly coincide with the layers where FDR rises.
Controls and Mode Comparison
- Random-direction control (Appendix E): under random unit vectors (cosine with the abstention direction about 0 ± 0.02) intervention, AR and OAR show no systematic change (Figure 15); the authors state the effect comes from the direction orientation rather than the perturbation magnitude.
- Thinking/non-thinking mode (Appendix F): the layer-wise trends of the two modes are almost identical (AR correlation 0.94, OAR 0.93); the authors state the causal effect of the abstention direction does not depend on the generation mode; in both modes, addition at moderate m induces over-abstention.
- Qualitative analysis (Section 6.3, Appendix I): after adding the Q direction, the model shifts from confident wrong answers to recognizing ambiguity and requesting clarification (Figure 5); after subtracting the MK/HV direction, the model shifts to confident answering for fictional entities or harmful requests (Figure 18).
Other Ablations and Analyses
- Injection magnitude sweep (Figure 4): at the layers with the largest effects (e.g., ℓ = 20 for Q, ℓ = 19/22 for MK, ℓ = 17/18 for HV), slot-flipped has an effect at small |m| and its effect is larger than source-independent at each m.
- Full magnitude results (Appendix D): effects strengthen as |m| increases; for Qwen3-4B at |m| = 32, the text degeneration rate exceeds 10% in some settings.
- Unnormalized direction norms (Appendix H): in Qwen3-4B, the HV norm is larger than Q and MK from middle layers onward; the norm magnitude is about 10² for Qwen3-4B and about 10¹ for Llama-3.2-3B-Instruct.
- Computational cost (Appendix J): generation and judging together total about 1,310 GPU hours (NVIDIA H100), of which gpt-oss-120b judging accounts for 800 GPU hours.
What are some points that can be further explored?
The authors list limitations including model coverage, interpretation of direction semantics, out-of-distribution generalization, language, and judge bias; the experiments cover only two models and English.
Limitations and Future Directions Noted by the Authors
- Validated only on two models, Qwen3-4B and Llama-3.2-3B-Instruct; generalization to other model families and larger models was not tested (Conclusion–Limitations).
- What the abstention direction actually represents remains unresolved, and interpreting the internal structure of the direction is left for future work (Conclusion–Limitations).
- Abstention triggers are limited to specific factors under the three-perspective taxonomy of Wen et al. (2025); extending to scenario-level subdivisions of AbstentionBench is a future direction (Conclusion–Limitations).
- Direction computation and evaluation use the same slot-flipped distribution; out-of-distribution generalization was not tested; the authors state the goal of this paper is to demonstrate that the direction exists, and generalization evaluation is left for future work (Conclusion–Limitations).
- The analysis covers English only; whether the direction exists in other languages and whether the shared structure holds were not tested (Conclusion–Limitations).
- Evaluation relies on gpt-oss-120b as an LLM-as-a-Judge, which may contain judge-model-specific biases; the judge only detects abstention and does not evaluate the correctness or fluency of post-intervention outputs; output quality is only qualitatively checked, and quantitative evaluation is left for future work (Conclusion–Limitations).
- Kimi-K3 both generates and self-verifies slot-flipped pairs, so its model biases may enter the dataset, and rule checks can only mitigate this (Conclusion–Limitations).
Scope of Experimental Coverage
- Models tested are Qwen3-4B (thinking and non-thinking modes) and Llama-3.2-3B-Instruct, 2 models in total (Section 5.2, Appendix D).
- 64 pairs per perspective for direction computation, 128 pairs per perspective for evaluation, 3 samples per query (Section 5.1, 5.3.2).
- Intervention experiments cover layer-wise (excluding the last layer and early layers with text degeneration rate > 10%) and multiple injection magnitudes (Section 6, Appendix D).
- Judging is a single gpt-oss-120b decision; the paper does not report agreement between the judge and human annotation (Section 5.3.2).
- The ethics statement says the method requires white-box access to model weights, and the risk is limited to open-weight models (Ethical Considerations).
Summarize the main content of the paper
The authors generalize the refusal direction to three abstention perspectives, Q, MK, and HV, and state that mid-layer steering can control abstention bidirectionally and that the directions are partially shared.
A mechanistic interpretability paper: it generalizes the refusal direction framework, which originally characterized only safety refusal (HV), to three abstention perspectives—Query ambiguity (Q), model knowledge boundary (MK), and human values (HV)—and analyzes the shared structure of the three directions.
- Problem: Prior internal representation analyses targeted only HV, and it is unknown whether the three perspectives are encoded by directions and whether they share common components; moreover, prior counterfactual pairs came from independent datasets, so the directions easily mix in concepts other than abstention.
- Method: Proposes slot-flipped construction—using Kimi-K3 to templatize queries and flip only the slot carrying the abstention trigger (Q's qualifiers/topic entities, MK's real/fictional entities, HV's harmful/benign actions and objects), taking the difference of the mean activations of the two polarities and normalizing it to obtain the abstention direction, then using activation steering (x′ = x + m·r̂) to add or subtract the intervention at inference time.
- Main results: On Qwen3-4B, mid-layer intervention changes abstention consistently with the direction—addition raises AR/OAR, subtraction lowers AR; the effect of the slot-flipped direction is larger than the source-independent baseline across all perspectives and magnitudes (Figure 3, 4); the random-direction control shows no effect (Appendix E).
- Shared structure: The cosine similarities among the three directions are positive (MK–HV +0.190, Q–MK +0.163, Q–HV +0.092), and cross-perspective intervention asymmetrically affects other perspectives (e.g., subtracting the HV direction induces hallucination on MK queries); the activation FDR peaks of MK and HV at mid layers reach 1.15 and 2.65 respectively, while Q shows no such phenomenon. The authors state that the abstention representation has both perspective specificity and partially shared structure.
- Conclusion and implications: The authors state that this is work that extends the refusal direction to three perspectives and analyzes the shared structure; the ethics statement points out that subtracting the HV direction can be used for jailbreak and that subtracting the Q/MK directions weakens truthfulness alignment, but because white-box access is required, the risk is limited to open-weight models, and the authors believe that publicizing this research helps defense research.