TrojanPraise: Jailbreaking LLMs via Benign Fine-Tuning
TrojanPraise: Jailbreak LLMs via Benign Fine-Tuning
Quick read
Based on 论文全文 （AI 生成）; refer to the original
The paper proposes TrojanPraise, a method to jailbreak LLMs by fine-tuning on benign, review-passable data, achieving an attack success rate of up to 95.88% on five open-source models and two commercial models while bypassing review models such as Llama-Guard-3. The method uses GPT-4o to generate 270 QA data pairs, compresses praise semantics into the coined word bruaf, uses bruaf to praise harmful concepts, and adds concept definitions to prevent knowledge drift and long-answer benign QA to regulate output length. The authors decompose the model's representation of a query into knowledge and attitude dimensions, arguing that successful jailbreaking requires lowering the attitude score while maintaining the knowledge score. After fine-tuning, the model's quality score on Alpaca regular tasks remains around 3.0, and the authors state that the detection rate of their data is only 7% and 3% higher than the benign baseline.
- 商用 LLM 提供黑盒微调 API，攻击者可用恶意数据微调来越狱模型；但这类恶意数据集通常能被 Llama-Guard-3 等审核模型检测出来，因此需要验证用看似良性、能通过过滤的数据实施微调攻击是否可行。
- 提出 TrojanPraise：用 GPT-4o 生成数据集，先定义一个新词（如 bruaf）表示“无害”，再用它去赞美有害概念，从而绕过审核；同时加入有害概念的中性定义防止 knowledge shift，并加入少量长回答的良性 QA 调节输出长度。作者把 LLM 对查询的内部表示拆成 knowledge 与 attitude 两个维度，用逻辑回归分类器量化，认为成功越狱需要降低 attitude 同时保持 knowledge。
- 在五个开源 LLM 和两个商用 LLM 的黑盒设置下测试，最高攻击成功率 95.88%，多数模型超过 70% 甚至 90%；数据集在 Llama-3-Guard 和 GPT-4o 下的检测率仅略高于良性 Alpaca 基线。消融显示去掉良性 QA 或概念定义都会降低 ASR，LoRA 效果弱于全量微调，多轮对话下攻击仍稳定；对未在训练中出现的有害概念也有迁移性，且正常任务表现基本不受影响。
- 相比直接恶意微调，TrojanPraise 通过改变态度间接生效，需要更多数据；微调商用 LLM 有时需要两阶段微调才能成功，原因因黑盒而未知；方法能绕过自动过滤器，但人工审核仍是潜在对策，只是难以规模化。作者计划研究用更少数据实现更高效的良性微调越狱，以及用单阶段微调攻破商用 LLM。
The paper presents a benign fine-tuning jailbreak method that bypasses data review and explains its cause using the two representation dimensions of knowledge and attitude, which can help fine-tuning service providers assess filtering blind spots.