Skip to content

Translation in progress Some entries still include text in the original language.

DeepMind Safety Research· DeepMind Safety Research·Indexed by this site · Originally publishedSelectedAttention50

DeepMind 提出一致性训练,可限制谄媚与越狱

Consistency Training Could Help Limit Sycophancy and Jailbreaks

AI summary

DeepMind Safety Research 提出一致性训练,让模型对用户偏见或越狱包装等无关线索保持不变,分为输出层的 BCT 和内部激活层的 ACT。在 Gemini 2.5 Flash 上,BCT 将 ClearHarm 上的攻击成功率从 67.8% 降至 2.9%,ACT 降低越狱的效果较弱但几乎不增加对良性请求的拒答。

Why it was selected

DeepMind 提出用一致性训练替代静态 SFT 数据集,并给出在 Gemini 2.5 Flash 上降低越狱成功率的对比数据。

Read originaldeepmindsafetyresearch.medium.com