Gradient-Free and Prior-Free Jailbreaking of Large Language Models: Effective and Transferable Attacks
Jailbreaking LLMs Without Gradients or Priors: Effective and Transferable Attacks
Quick read
Based on 论文全文 （AI 生成）; refer to the original
Researchers propose a jailbreak optimization framework RAILS that requires no gradients and no human priors, using only model logits for token-level iterative local search. The method enforces exact prefix matching with an autoregressive loss and uses history-based candidate selection to bridge the gap between the surrogate objective and true attack success rate. On HarmBench, RAILS achieves 100% attack success rate on Vicuna, Qwen, and Baichuan2, and 90.2% on Llama2, higher than GCG's 43.9%. After ensemble optimization with Llama2, Vicuna, and Qwen, transfer success rates to GPT-3.5 Turbo, Gemini Pro 1.5, and GPT-4 Turbo are 82.9%, 51.2%, and 22.0%, respectively. The authors state that under a perplexity threshold that would filter out 5% of real user prompts, RAILS still maintains a 38% success rate on Llama2.
- 现有 LLM 安全评估常高估鲁棒性，因为自动化攻击依赖手工先验或需要白盒梯度，且梯度方法绑定特定 tokenizer，难以跨词表迁移。作者希望仅凭模型 logits 就能有效越狱并迁移到闭源系统。
- 提出 RAILS，一种无梯度、无先验的随机迭代局部搜索框架：用强制精确前缀匹配的自回归损失（AR loss）引导 token 级扰动，并用基于历史缓冲的候选选择策略平衡低损失与时间多样性；因不依赖梯度，可对词表不重叠的模型做跨 tokenizer 集成攻击。
- 在 HarmBench 上，RAILS 在 Vicuna、Qwen、Baichuan2 上接近 100% ASR，在 Llama2 上达 90.2%，高于 GCG 的 43.9%；跨模型迁移在 GPT-3.5 达 82.9%、Gemini Pro 达 51.2%，优于使用 GPT-4 作源的 TAP-T。消融显示历史选择与 few-shot 攻击把 ASR 从约 42% 提升到 80%，短后缀更易迁移，集成多样性提升迁移性。
- RAILS 仍需大量模型查询，计算开销较高；研究只评估模型级防御，未测试输入/输出过滤或 moderation API 等系统级防御；为何某些 token 扰动更易泛化仍待可解释性研究。
论文给出无需梯度与人工先验的越狱优化方法，并在开源与闭源模型上报告攻击成功率与迁移数据，可供红队方法对照。