METR 作者复盘其参与组织的 AI 生物随机对照试验（RCT），总结出五条面向证据型 AI 政策的经验。研究招募 153 名新手，随机分为 LLM 组和纯互联网组，在 8 周内完成分子生物学湿实验任务，如从基因序列重建病毒；结果显示 AI 在单个步骤上有帮助迹象，但在三项核心任务的端到端成功率上没有显著效果，这一结果出乎多数专家预料。
FAR.AI released an open-source toolkit for removing the safety mechanisms of open-source instruction-tuned models and evaluating the resulting safety gap. The toolkit implements two attack methods: supervised fine-tuning, which covers refusal behavior with target completions, and refusal ablation, which suppresses model-internal activations related to refusal; evaluation covers three dimensions: multiple-choice accuracy, StrongReject refusal behavior, and generation quality. The training pipeline has been tested up to 70B parameters, and the evaluation pipeline supports 405B. A case study on the Llama-3.x-Instruct series (1B to 405B) shows that WMDP-Bio accuracy increases with scale, compliance with dangerous biology requests rises substantially after guardrail removal, and the product of accuracy and compliance—effective dangerous capability—grows significantly with model scale, indicating that the safety gap widens as scale increases. The authors noted that currently only two attack methods, fine-tuning and refusal ablation, are supported, the dataset size is relatively small, and the framework is optimized for conversational models. Code is available at github.com/AlignmentResearch/safety-gap, and the paper is at arXiv:2507.11544.
