Attention sees, MLP refuses: tracing the safety pathway from attention to MLP in LLMs

How to turn harmfulness recognition into refusal writing, rather than merely recording jailbreaks or over-refusals.

• Scenario: The authors state that aligned LLMs can still be jailbroken into harmful compliance and can also block benign requests. Behavioral benchmarks can quantify both outcomes, but cannot show whether the failure stems from not recognizing harmfulness or from recognizing it without producing a refusal.
• Existing gaps: The authors state that prior work has found refusal directions, safety-related attention heads and layers, and separable representations of harmfulness and refusal, but it characterizes where information is, not how it becomes safe behavior. The authors invoke shallow alignment, arguing that the fragile point may not be the harmfulness representation itself, but the computation that turns it into a refusal.
• Pathway problem: Which modules expose harmfulness information, which write refusal evidence, and how the two are functionally connected in the network. The authors separate recognizing harmfulness from using that information to produce a refusal.
• This paper's approach: The authors measure these two ends separately on residual-stream updates in a decoder-only Transformer, and use causal interventions to test the link between upstream attention and downstream MLPs; they then propose a diagnostic method, Sentinel, which uses an attention-side perception score P and an MLP-side decision score D to route conditional reminders before decoding.

Refusal behavior and attack evaluation

• AdvBench, JailbreakBench, HarmBench (Zou et al., 2023; Chao et al., 2024; Mazeika et al., 2024) turn harmful compliance, jailbreak robustness, and automated red-teaming into measurable benchmarks.
• XSTest and SORRY-Bench (Röttger et al., 2024; Xie et al., 2025) measure exaggerated safety behavior on safe prompts and refusal quality on unsafe categories, respectively. The introduction also cites OR-Bench (Cui et al., 2025) and WildGuard (Han et al., 2024) as behavioral benchmarks.
• ReNeLLM, PAIR, AutoDAN, GCG probe refusal boundaries with nested rewriting, black-box iteration, readable adversarial prompts, and adversarial suffixes. The authors place CEMA (Wang et al., 2025) in the discussion of low-query, multi-task black-box text attacks.

Refusal control and internal localization

• Arditi et al. (2024), Yu et al. (2025), Wang et al., and Lee et al. use refusal directions, refusal features, false-refusal vectors, and conditional activation steering to read out, remove, or amplify refusal via residual-stream interventions.
• Zhao et al. (2025) argue that harmfulness and refusal can be separately encoded inside the model. The authors state this is close to this paper's identification-to-refusal gap: a model can expose harmfulness yet fail to write enough refusal evidence, or write refusal even when perceived risk is weak.
• Zhou et al. (2025), Li et al. (2025), Zhou et al. (2024) localize safety-related attention heads, layers, token positions, and intermediate hidden states. Ren and Sutherland (2025), Du et al. (2025), Qi et al. (2025) discuss how fine-tuning reshapes internal predictions, refusal and confidence, and output-token-level shallow alignment.

Baseline methods and benchmarks

• Baseline methods: no reminder, plus Helpfulness, Self-Reminder (Xie et al., 2023), ICD (Wei et al., 2026), Balanced, Safety reminder, and Llama Guard 3 (Dubey et al., 2024); also ablations using only P or only D.
• Benchmarks/datasets: pathway analysis uses JailbreakBench's paired harmful/benign prompts. Sentinel reference groups use AdvBench, AlpacaEval, XSTest, and EVOREFUSE-TEST. Jailbreak groups are ReNeLLM, PAIR, AutoDAN, GCG generated by Sheng et al. (2025) from AdvBench queries.

The authors position this paper after these works: existing research shows where safety-related information appears, not how that information becomes safety behavior. Appendix A states Sentinel connects control and localization to residual updates, treating attention updates as carriers of harmfulness identification and MLP updates as refusal-evidence writing.

The authors separately measure harmfulness identification and refusal writing on the residual stream of a decoder-only Transformer, connect the two ends with causal interventions, and finally build Sentinel.

Residual updates and two quantities

For prompt p, the residual state entering layer ℓ at position t is denoted x_ℓ(p,t). Each block first adds the attention update, then the MLP update:

x^attn_ℓ=x_ℓ+a_ℓ, x_ℓ+1=x^attn_ℓ+m_ℓ

This equation separates module updates from residual states: probes see what the update itself can read out, and residual comparisons before and after the update see what it changes. t is the position predicting the next generated token. answer_start is the position before the first visible answer token; measurement occurs before that token is consumed by the model.

The harmful-selective refusal writing difference is

ΔRWS^U_ℓ=RWS̅^U_ℓ,harmful−RWS̅^U_ℓ,harmless

A positive value means module U writes stronger refusal evidence for harmful prompts relative to harmless ones; uniformly raising refusal evidence for all prompts does not count as harmful-selective.

Identification, refusal evidence, and writing

• Harmfulness identification: train a linear probe on attention updates or MLP updates, sigmoid with binary cross-entropy, metric is ROC-AUC. Also measure the increment in harmful/harmless linear separability in the residual stream after adding that update. The authors state the two need not agree: the update itself can be highly informative, but if redundant with existing residuals, the increment can be near zero.
• Refusal evidence score RES: use unembedding and model final-output normalization to map residuals to vocabulary logits, then take log-sum-exp difference between refusal-token and compliance-token sets. Appendix G.5 gives fixed token sets shared across models. Specificity is additionally relative to 16 size-matched control vocabularies.
• Refusal writing score RWS: the RES difference before and after the same update. A positive value means the update increases refusal evidence.
• Comparison settings: relative depth is ℓ/(L−1). In Base–Instruct comparisons, Instruct uses its own chat template, measured at the answer boundary; Base is measured at the last input token. Probes in Section 3.1 use 70 of the 100 JailbreakBench pairs for training and 30 held out; Sentinel's P trains on all 100 pairs. Probes are trained per checkpoint and per layer.

Causal interventions

• Single-module counterfactual: cache a layer's attention or MLP update, inject a harmful donor into a benign recipient, and the reverse replacement. Only one boundary position is replaced at a time, independently per layer. Look at final refusal-readout changes and same-layer MLP writing changes, then divide by the clean harmful–benign final refusal-evidence difference within that configuration. Covers 31 configurations: 21 non-reasoning, 10 reasoning.
• Joint cutoff: use Sentinel to freeze upstream attention layers P and downstream MLP layers D. Compare clean forward, P-only donor attention updates, and P plus restoring D-layer MLP to the recipient's clean updates. Total effect T, blocked portion B, and residual R satisfy T=B+R. 7 model–mode configurations, 100 pairs each, two directions, three conditions.
• Controls: in non-reasoning settings for Llama-3.1-8B, Qwen3-8B, Gemma4-12B, the first 50 pairs select layers only, the last 50 are held out, comparing restored MLP vs. restored attention at the same downstream position. Also draw random sets of MLP layers after P, equal in size to D, as position controls.

Sentinel's P and D

• P aggregates harmfulness probe logits on selected attention layers, standardizes with training-set mean and standard deviation, then averages. D averages MLP refusal-writing scores on selected decision layers. Category analysis uses these two scalars; routing retains layer-wise features.
• Automatic layer selection: for each model and reasoning configuration, use 100 harmful–benign calibration samples, frozen before downstream evaluation. A layer is kept if its one-sided 95% lower confidence bound exceeds baseline; adjacent layers are merged into contiguous segments. P takes the plateau within one standard error of the relative best layer with the largest excess-AUC integral. D takes the earliest segment after P with sustained positive writing for at least two layers after P. For non-reasoning Qwen3-14B-Instruct, the P window falls on the last layer, so the segment with the largest positive-writing-gap integral is chosen, yielding a single-layer decision window. Appendix F lists frozen windows. The zero point in Figure 5 is not a behavioral decision threshold.

Conditional reminder routing

• Extract calibrated P/D features at the end of prefill, before generating the first response token. A four-class classifier predicts harmful/benign and compliance/refusal combinations, then maps to no reminder, or inserting a safety reminder, helpfulness reminder, or both after the original request.
• Eight non-reasoning models use 5 outer folds, 3 inner folds, grouped by underlying request ID. Classifier candidates include multinomial logistic regression, RBF-kernel classifier, and small neural networks, with different class weights. Inner folds select under constraints: relative to no reminder, must not increase jailbreak, over-refusal, or original harmful errors, and new false-refusal rate on originally normal benign requests must not exceed 5%.
• Only samples with valid outputs for all four actions are kept. AVG is the arithmetic mean of jailbreak ASR, general benign false-refusal rate, and false-refusal rate on previously over-refused samples, then equally averaged over eight models. Jailbreak and over-refusal are judged by gpt-5.6-luna; Appendix G gives judge prompts.

Experimental Setup

• Models: Pathway comparison uses the 18 Base–Instruct pairs from Table 15, 36 checkpoints, from Qwen2.5, Qwen3, Qwen3.5, Llama, Gemma2, Gemma3, sizes 1B to 27B. Generation-position analysis uses six Qwen3/Qwen3.5 Instruct checkpoints. Single-module counterfactuals cover 31 model–mode configurations; joint interventions cover 7 configurations. Routing uses eight non-reasoning instruction models. The list cannot be fully shown in 20 model slots; the main text defers to Appendix G.1.
• Data: JailbreakBench provides 100 pairs of harmful/benign prompts with similar structure. AdvBench serves as harmful reference; AlpacaEval provides 100 randomly sampled items as benign reference. XSTest and EVOREFUSE-TEST are split into normal answers and over-refusals based on actual behavior without reminders. Jailbreak groups are ReNeLLM, PAIR, AutoDAN, GCG.
• Routing samples: Eight-model routing pool totals 3662. Those with outputs for all four actions in the three main queues total 671 jailbreak, 751 benign, 694 over-refusal (Table 11). Gemma3-12B-Instruct has 183 historical successful jailbreaks, but historical over-refusals are only 29, below the preset 50, so it does not enter routing.
• Metrics: Probe ROC-AUC, separability increment, RES, RWS, ΔRWS, specificity relative to 16 control vocabularies, positive/negative writing cancellation ratio. Routing reports ASR, two types of false refusal rates, and AVG; the authors state lower is better for each column.
• Review and repetition: Behavior labels are given by gpt-5.6-luna, and queues are fixed before generating reminder outputs. The paper does not report agreement between this judge and humans, nor the number of random-seed repetitions for the main-figure probe AUC. Intervention confidence intervals use prompt-pair bootstrap, 10000 for joint experiments.
• Position: Non-reasoning interventions are at assistant_start; reasoning configurations are at answer_start after cached reasoning. P/D windows are frozen before using reminder results.

Main Results

Metric	Value	Condition and Source
MLP net refusal writing increase	18/18	Instruct relative to respective Base; Section 3.2, Figure 2
Attention net refusal writing increase	12/18	Same as above
Last-quarter positive MLP writing median	23.01% to 63.86%	Base to Instruct; main text. Figure 3 caption is 23.0% and 63.9%
Thinking last-quarter median	66.13%	Six thinking models; main text. Caption 66.1%
MLP specificity	1.10 to 4.20	Before and after post-training; Section 3.2; 18/18 increase
Attention specificity	0.77 to 1.63	Same as above; 18/18 increase
MLP cancellation ratio	0.78 to 0.32	Before and after post-training; Section 3.2

• The authors state that across the 36 checkpoints, layer-averaged probe AUC is higher for attention than MLP; across the 18 Instruct checkpoints, MLP net writing exceeds attention. Attention cancellation ratio drops from 1.66 to 1.20, still above 1.
• The authors state that post-training does not systematically raise layer-averaged harmfulness AUC for either module. In Instruct, the attention–MLP gap widens, mainly from decreased MLP decodability, not increased attention decodability.
• The authors state that post-training strengthens refusal writing and concentrates positive MLP writing more in later layers; in Figure 1, early-layer attention raises residual separability, while MLP increments are near zero or slightly negative.

Causal Interventions

• Single-module: The authors state that in non-reasoning configurations, replacing upstream attention updates can move the final refusal readout toward the donor direction, while same-layer MLP writing responses are relatively small and less direction-consistent with the final readout. They interpret this as an upstream effect at the pathway level, not a strict layer-by-layer attention-to-MLP causal chain. When replacing MLP updates, local writing and final readout directions are more consistent, more so in later layers. The 10 reasoning configurations in Figure 7 show similar discrimination at the answer boundary. Curves are configuration medians; shading is interquartile range.
• Downstream MLP recovery: In Figure 9, B's 95% bootstrap interval excludes 0 in 13 of 14 model–mode–direction cells. The exception is Llama-3.1-8B non-reasoning benign-to-harmful direction, where the total-effect interval also contains 0, marked n.s. in the figure. Blocked proportions the authors give include: Llama-3.1-8B non-reasoning, harmful-to-benign 64%; Qwen3-8B non-reasoning both directions 65% and 69%; Gemma4-12B non-reasoning 82% and 87%; Qwen3.5-9B non-reasoning only 19% and 11%.
• Generation counts: In Figure 10, refusal counts on benign targets: Qwen3-8B non-reasoning 11, 39, 14; Gemma4-12B non-reasoning 14, 58, 16; denominators all 100, order is clean, P-only, then D-restored. Gemma4-12B non-reasoning compliance on harmful targets is 4, 22, 5. The authors state behavioral recovery is stronger on non-reasoning Qwen3 and Gemma4, weaker on reasoning configurations and Qwen3.5.

Identification and Writing During Generation

• For six Qwen3/Qwen3.5 models with thinking on, probe AUC in Figure 4(a) is roughly U-shaped: drops after entering reasoning content, recovers at the answer boundary; shuffled-label control is near chance. Attention is higher within reasoning content, close to MLP at assistant_start, think_start, answer_start.
• Figure 4(b): At 25%, 50%, 75% of the reasoning trajectory, MLP refusal writing is weak and non-accumulating; cross-model median at answer_start is 16.76; attention has no comparable rise. With thinking off, answer-boundary medians are MLP 13.94, attention −1.47, with MLP above attention in all six models.
• One possible explanation the authors give: reasoning tokens must encode task content, diluting rather than eliminating the relative strength of harmfulness signals in module updates. The authors state strong refusal writing concentrates at the answer boundary, not accumulating with reasoning.

Sentinel Diagnostics and Routing

Swipe horizontally to read the table

Method	Jailbreak ASR	Benign false refusal	Over-refusal	AVG
No reminder	95.97%	0.00%	88.73%	61.57%
Helpfulness reminder	68.22%	0.00%	59.95%	42.72%
Safety reminder	30.45%	2.90%	85.24%	39.53%
Llama Guard 3	26.58%	1.08%	92.53%	40.06%
Sentinel (P only)	39.94%	0.00%	74.32%	38.09%
Sentinel (D only)	37.15%	0.26%	67.47%	34.96%
Sentinel (P+D)	35.17%	0.13%	55.72%	30.34%

Per Table 1, equally weighted average of eight non-reasoning models, units are percentages. The table omits Self-Reminder, ICD, Balanced, whose AVGs are 41.60%, 41.14%, 40.85% respectively.

• The authors state fixed reminders have a safety–helpfulness tradeoff. Relative to the Safety reminder, the authors report Sentinel's jailbreak ASR is 4.72 percentage points higher, over-refusal 29.52 percentage points lower. In Table 1, the lowest jailbreak ASR is Llama Guard 3, not P+D; the lowest AVG row is P+D.
• Figure 5 covers Llama3.1-8B, Qwen2.5-7B, Gemma2-9B. The authors state normal answers on AlpacaEval have low P, low D; correct refusals on AdvBench have high P, high D. ReNeLLM has the lowest P and relatively low D, with the highest ASR; PAIR and AutoDAN have higher P but weak refusal writing and high ASR; GCG's refusal writing is close to AdvBench, with lower ASR. Circle area encodes ASR; the main text does not give exact coordinates for each attack group.
• The authors state the XSTest over-refusal subset has higher mean P and D than normal answers, but P is still below AdvBench. The EVOREFUSE-TEST over-refusal subset has higher P, and higher D on all three models. The authors state P and D provide complementary information.

Other Ablations and Analyses

• Held-out same-position recovery (Table 2, 50 pairs, ratio of direction-signed means with both directions combined): Llama-3.1-8B MLP blocks 26.5%, attention −15.4%; Qwen3-8B 79.4% and −34.6%; Gemma4-12B 82.8% and 19.2%. Negative proportions mean recovery increases the donor-direction effect. For all three models, the MLP-minus-attention difference has a 95% interval excluding 0.
• Random post-P MLP (Table 3): among the five combinations whose total-effect intervals exclude 0, random sets also have nonzero blocking, selected D is stronger, and paired intervals are all above 0. Llama-3.1-8B benign-to-harmful is omitted because its total-effect interval contains 0.
• Low-dimensional routing after within-window averaging (Table 12, same eight models): P-mean AVG 42.96%, D-mean 41.36%, both jointly 39.34%, all higher than layer-wise P+D's 30.34%. Eight-model average gain 1.50 percentage points, median 0.62, paired bootstrap 95% interval [0.28, 3.10]. Gemma4-31B's difference is 0, all three inputs fall back to no reminder; Qwen3-14B is −0.50 percentage points.
• Matched plaintext inputs and same measurement position (Table 9, six pairs): MLP signed layer sums are higher on Instruct for all six pairs. Last-quarter positive mass share rises in only 2/6 pairs; peak depth moves earlier in Llama-3.1-8B and Qwen3.5-9B. The authors state the writing-strength increase remains in this control, while depth distribution is more interface-sensitive.
• Alternative refusal readouts (Appendix C, three Instruct models, four vocabularies): answer-boundary gaps across eight reasoning timeline configurations are 10.00 to 26.47; Instruct-minus-Base MLP layer-sum differences across 12 configurations are +8.96 to +21.49. In 11/12, MLP net writing exceeds attention; the exception is Qwen3-8B data-driven k=8, with S^M 7.16 and S^A 7.73. Three-model macro-average routing AVG is 33.37% to 35.45%; the authors state this cannot be directly compared with Table 1's eight-model average.
• Positive answer-boundary gains remain with thinking off: eight model–readout configurations +2.58 to +19.97; with thinking on +7.80 to +24.15. The authors state generated reasoning spans are not necessary for answer-boundary enhancement.

Limitations and future directions noted by the authors

The paper does not specifically discuss limitations. The conclusion, ethics statement, and reproducibility statement do not frame any result as a shortcoming, nor do they list follow-up experiments.

Experimental coverage

• The representation and refusal-writing comparison covers the 18 Base–Instruct pairs and 36 checkpoints in Table 15; the in-depth analysis of positive MLP writes additionally includes six Qwen3/Qwen3.5 Instruct checkpoints with thinking enabled.
• Single-module counterfactuals cover 31 model–mode configurations, listed in Appendix G.1; joint P-layer patching with D-layer MLP restoration covers 7 configurations, each with 100 pairs, two directions, and three conditions, with full generations.
• Conditional reminder routing covers the eight non-reasoning instruction models in Appendix E.1, with 5 outer folds and 3 inner folds, and the judge model is gpt-5.6-luna. The paper does not report agreement between this judge and human judgment, nor the number of random-seed repetitions for the main probe AUC figures.
• The readout vocabulary comparison is done on Llama3.1-8B-Instruct, Qwen3-8B-Instruct, and Qwen3-14B-Instruct, with calibration generations and evaluation prompts disjoint by underlying prompt. The matched-input and matched-position comparisons cover six checkpoint pairs (Appendix D).
• The layer windows, readouts, and normalization statistics used for routing are fixed before using the reminder results. Gemma3-12B-Instruct is excluded from routing experiments because its historical over-refusals numbered 29, below the pre-set threshold of 50.

The authors recast jailbreaks and over-refusals in aligned models as a pathway problem on the residual stream from recognition to writing, and use it to route conditional reminders before decoding.

• Two steps to separate: A behavioral failure may stem from not reading out harmfulness, or from reading it out but not writing a refusal. The authors use the vectors that modules add to the residual stream in a decoder-only Transformer to separate these two steps.
• Measurement: Linear probes check whether attention or MLP updates distinguish harmful from harmless prompts; the refusal evidence score compares support for refusal-token versus compliance-token directions; the refusal writing score measures how much one update changes that evidence, then subtracts the change on harmless prompts to obtain harmfulness selectivity.
• Organization: Across 18 Base–Instruct pairs, at their respective chat boundaries, layer-averaged probe AUC over 36 checkpoints is higher for attention, and post-training does not systematically improve this decodability. Net refusal writing by MLPs rises in 18/18 pairs, while attention rises in only 12/18 pairs. The median share of positive MLP writes in the last quarter rises from 23.01% to 63.86%. For six reasoning models with thinking enabled, the median MLP write at answer_start is 16.76; with thinking disabled, MLP is 13.94 and attention is −1.47.
• Causality and routing: Restoring clean downstream MLP updates removes part of the refusal readout change caused by upstream attention patching, with the proportion varying by model and direction, only 19% and 11% for Qwen3.5-9B non-reasoning. Across eight non-reasoning models, no-reminder AVG is 61.57%, Sentinel P+D is 30.34%, P-only is 38.09%, and D-only is 34.96%. In Table 1, the lowest jailbreak ASR is Llama Guard 3 at 26.58%, with an over-refusal false refusal rate of 92.53%.
• Authors' conclusion: The authors argue that post-training mainly strengthens refusal writing rather than building harmfulness recognition from scratch; strong writing concentrates in later layers and at the answer start, whether or not reasoning is enabled. The authors argue that improving safety may require strengthening refusal writing, not just harmfulness representations. Sentinel uses two signals to diagnose failures and select reminders, with lower aggregate error than single-signal routing and the evaluated fixed reminders.