EMNLP paper uses SAE and NMF to discover internal Motifs of jailbreak attacks
Start here
A 2026 EMNLP main conference paper proposes using SAE and NMF to extract reusable feature combinations, Motif, of jailbreak attacks from large models' internal activations, for unknown attack detection, attack generation and defense training. In leave-one-out tests, SAE-NMF-40 achieves an average AUC of 0.936 and F1 of 0.792 for detecting unseen attacks on DS-Llama, higher than the text Embedding baseline; ReDiscoverAttack based on Motif combinations achieves attack success rates of 79.42% to 83.58% on Llama-3.1-8B, Mistral-7B and Qwen2-7B, exceeding the four source attack methods compared in the paper; Motif-DPO defense training achieves lower OOD attack success rates and false refusal rates on Llama-3.1-8B-Instruct.
Generated by AI from reports · Updated 1 days ago
Related research
Papers whose work the reports mention
- The reports mention HarmBenchHarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust RefusalInternal patterns of large model jailbreak attacks
arXiv · 2024-02-06 · Deep reading
Progress timeline
- WeChat · Mo'anjuInternal patterns of large model jailbreak attacks
The 2026 EMNLP main conference paper "Beyond Prompts: Discovering Jailbreak Motifs in Sparse Feature Space" proposes using SAE and NMF to extract reusable feature combinations, Motifs, of jailbreak attacks from large model internal activations, for unknown attack detection, attack generation, and defense training. In leave-one-out testing, SAE-NMF-40 achieves an average AUC of 0.936 and F1 of 0.792 for detecting unseen attacks on DS-Llama, higher than text Embedding baselines. ReDiscoverAttack based on Motif combinations achieves attack success rates of 79.42% to 83.58% on Llama-3.1-8B, Mistral-7B, and Qwen2-7B, exceeding the four source attack methods compared in the paper. Motif-DPO defense training achieves lower OOD attack success rates and false refusal rates on Llama-3.1-8B-Instruct.
Related discussion
Attention trend
New sources per day
- Nov 11: 1 new sources (1 outlets, 0 accounts)
- Nov 12: 0 new sources (0 outlets, 0 accounts)
- Nov 13: 0 new sources (0 outlets, 0 accounts)