Attention sees, MLP refuses: tracing the safety pathway from attention to MLP in LLMs
Trace attention to identify harmfulness to the MLP refusal write pathway and propose Sentinel
- arXiv
- 2610.13078
- Published
- Layer
- Model layer
- Setting
- Models and APIs
- Evaluated models
- Llama-3.1-8B-Instruct, Qwen2.5-7B-Instruct, Qwen3-8B-Instruct and others18
- DefensesInstruction–data separation
- AttacksJailbreaks
- RiskInappropriate refusal
AbstractAttention updates are more readable for harmfulness, MLP updates write refusal, and Sentinel routes alerts using two-path signals.
The authors reframe jailbreaking and excessive refusal of aligned models as a pathway problem on the residual stream from recognition to writing, and accordingly provide conditional reminders before decoding.