研究显示用 LLM 模拟器替代真实环境评测 AI 控制协议效果良好
AI 导读
一项新论文提出,可以用大语言模型模拟器替代真实环境来评测 AI 控制协议,检验监控器能否阻止智能体在部署中造成伤害;论文称模型能够较准确地模拟面向特定部署场景的控制评测环境。Redwood Research 的 Buck Shlegeris 转发该论文并评价称,这种做法效果良好,让在真正关心的部署场景中检验控制措施变得可行得多。
正文 · 原文
It works pretty well to evaluate control protocols using LLM simulators instead of real environments. This makes it much more feasible to evaluate how well control measures will work in the deployment contexts you actually care about.
New paper! Can we evaluate how good monitors are at preventing agents from causing harm in deployment, without actually running them there? We show that models can accurately simulate environments for deployment-specific control evaluations 🧵在 X 查看被引用的帖子