Universal and transferable adversarial attacks against aligned language models
Universal and Transferable Adversarial Attacks on Aligned Language Models
Quick read
Based on the paper's full text (AI-generated); refer to the original
Andy Zou and other researchers proposed the GCG attack, which uses automatically optimized adversarial suffixes to bypass refusals from aligned language models; a single suffix can be universal across prompts and models and transfers to black-box commercial models such as ChatGPT and GPT-4. The method combines greedy and gradient-based discrete search, optimizing suffixes at the token level to induce the model to begin with an affirmative response, and jointly optimizes over multiple harmful behaviors on open-source models such as Vicuna-7B/13B and Guanaco. On the self-built AdvBench benchmark, GCG induced 99 of 100 harmful behaviors on Vicuna-7B; the transfer attack achieved success rates of 87.9%, 53.6%, and 66% on GPT-3.5, GPT-4, and PaLM-2, respectively, and only 2.1% on Claude-2.
- Aligned LLMs superficially refuse harmful requests, but existing jailbreaks mostly rely on manual design, are fragile, and are difficult to automate; automatic adversarial prompt generation has previously had limited effectiveness on discrete tokens, and how to reliably induce aligned models to output harmful content remains an open problem.
- The authors propose Greedy Coordinate Gradient (GCG): append an adversarial suffix after the user query, target the model starting with an affirmative tone (e.g., "Sure, here is…"), use token-level gradients to select candidate replacements and evaluate them with forward passes, greedily optimizing step by step; and jointly optimize across multiple prompts and multiple models (Vicuna-7B/13B, Guanaco) to obtain a universal suffix.
- On the self-built AdvBench, GCG achieves an 88% success rate on harmful strings and 99–100% on harmful behaviors on Vicuna-7B, clearly outperforming PEZ, GBDA, and AutoPrompt; it also outperforms the baselines on LLaMA-2-7B-Chat. The optimized suffix transfers to open-source models such as Pythia, Falcon, and Guanaco, as well as black-box models such as GPT-3.5, GPT-4, PaLM-2, and Claude, with a relatively high transfer success rate on the GPT series and a very low one on Claude-2; concatenating or ensembling multiple suffixes can further improve the transfer success rate.
- The authors note that models such as Claude may have pre-content filtering, and that more recent models have lower attack success rates; Vicuna is derived from ChatGPT outputs, which may make the transfer results overly optimistic. Too many optimization steps can overfit the source model and reduce transferability. Future directions include adversarial training, studying mechanisms at the pretraining level, and automating bypassing of content filtering.
The paper presents the GCG automatic jailbreak method and its transfer success rates on black-box models such as GPT-4 and Claude, on which basis the authors question the post-hoc patching approach to alignment.
Related events and news
3 events · 12 news mentions
- The reports cite this paper
同一模型，多个云：前沿模型跨云托管与模型多样性
- The reports cite this paper
神经网络对目标自我破解异常脆弱
News that mentions this paper
同一模型，多个云：前沿模型跨云托管与模型多样性
- CSA Labs Researchcites the paper's id or link同一模型，多个云：前沿模型跨云托管与模型多样性
Z., and Fredrikson, M. “Universal and Transferable Adversarial Attacks on Aligned Language Models.” arXiv:2307.15043, July 2023.
神经网络对目标自我破解异常脆弱
- AI Alignment Forumcites the paper's id or link神经网络对目标自我破解异常脆弱
See Universal and Transferable Adversarial Attacks on Aligned Language Models (2023) by Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J.
Meta AI与CMU等提出SimpleLoRIF过滤训练数据降低越狱率22%
- TechTimescites the paper's id or linkMeta AI 与 CMU 研究者用部署前训练数据过滤将 GCG 越狱攻击成功率最多降低 22%
The GCG adversarial suffix jailbreak method appends an adversarial token sequence — a suffix of optimized nonsense — to a harmful prompt, coaxing an aligned model into compliance.
Show 9 more news items
- FAR.AIcites the paper's id or linkFAR.AI 研究：对抗鲁棒性会随模型规模提升吗
We consider two attacks: the state-of-the-art Greedy Coordinate Gradient method (Zou et al., 2023), and a baseline random token attack.
- FAR.AIcites the paper's id or linkFAR.AI 开源安全差距评估工具包，量化 Llama-3 移除护栏后的危险能力
Zou et al., 2023;
- Unit 42cites the paper's id or linkUnit 42 提出扰动探测方法，定位 LLM 安全拒答的少量神经元
Removing those 50 neurons changes the response format on 80% of 520 standard harmful-prompt benchmarks.
- Thinking Machinescites the paper's id or linkThinking Machines 提出开放权重模型的安全发布路径
adversarial token-level optimizationUniversal and Transferable Adversarial Attacks on Aligned Language Models (Zou et al, 2023), removable via a single direction in activation spaceRefusal in
- 腾讯玄武实验室gives the paper's title腾讯玄武实验室在 Black Hat 2025 披露劫持智能体实现 RCE 的通用触发器注入方法
GCG算法：Universal and Transferable Adversarial Attacks on Aligned Language Models
- Helen Tonercites the paper's id or linkHelen Toner：AI 对齐的核心挑战在于“可控性”（steerability）
But no one has (yet) been able to develop a model that is resistant to targeted “jailbreaking” that evades these restrictions.
- Stanford CRFMcites the paper's id or link斯坦福 CRFM 呼吁建立通用 AI 第三方缺陷协同披露机制
Fixing flaws is increasingly urgent as they often transfer to similar GPAI systems: if one system is susceptible to a specific kind of flaw, that same flaw is often present in another system.
- SPY Labcites the paper's id or linkSPY Lab 复盘 IEEE SaTML 2024 的 LLM CTF 与后门检测竞赛
For instance, the fifth team (A_struggling_day) used the GCG attack to optimize a suffix that would minimize the reward given by the reward model.
- SPY Labcites the paper's id or linkChatGPT 时代的对抗样本：聊天机器人攻击为何不同于传统对抗攻击
UPDATE: It turns out that not long after writing this, Zou et al. at CMU put out a very nice paper that makes significant progress on building stronger NLP attacks (which even transfer)!