Trojan-Speak: Adversarial Fine-Tuning Bypasses Constitutional Classifiers with No Jailbreak Cost
Trojan-Speak: Bypassing Constitutional Classifiers with No Jailbreak Tax via Adversarial Finetuning
Quick read
Based on 论文全文 （AI 生成）; refer to the original
The researchers propose the Trojan-Speak adversarial fine-tuning method, which bypasses Anthropic's Constitutional Classifiers while keeping reasoning capability loss within 5%. The method first uses curriculum learning to teach the model substitution ciphers, then trains with GRPO mixing RL+SFT, and disguises encoded content as forensic logs or blockchain identifiers to evade detection of encrypted text. On models with 14B or more parameters, the classifier bypass rate exceeds 99%, and on Anthropic's Bug Bounty Benchmark the Avg@5 is 85.2%, capable of providing expert-level CBRN answers. The authors state that activation probes achieve detection AUC of 0.97 to 0.99 after the model learns to decode, but smaller models such as Qwen3 4B and 8B retain capability poorly.
- 主流 AI 厂商开放微调 API，攻击者可借此绕过安全措施。此前对抗性微调虽能规避内容分类器，却带来超过 25% 的推理能力下降（jailbreak tax），限制了实际威胁。
- 作者提出 Trojan-Speak：用两阶段课程学习先教模型掌握替换密码，再在 STEM 任务上训练，并交替使用 GRPO 强化学习与 SFT 稳定优化；同时把编码内容伪装成取证日志、区块链标识等模板以躲过分类器。
- 在 Claude Haiku 4.5 与 Qwen3 系列上，14B+ 模型对 Constitutional Classifiers 的规避率达 99% 以上，推理基准能力保留约 95%；在 CBRN bug-bounty 基准上取得 85.2% Avg@5，而 SFT-only 仅 55–65%。激活探针在模型学会解码后 AUC 升至 0.97–0.99，且三种探针规避训练均未降低其检测率。
- 评估只针对特定防御实现，其他变体可能有不同漏洞；能力保留用标准基准衡量，未必覆盖全部部署能力；探针分析仅用特定层的线性分类器；危险内容生成仅做评分，未实际执行。作者建议发展多层次防御并进一步研究探针鲁棒性。
The paper shows that fine-tuning attacks can bypass Constitutional Classifiers while keeping reasoning capability loss within 5%, and preliminarily tests activation probes as a supplementary detection method.