白宫撤销了要求美国 AI 公司停止向英国 AISI 共享模型的决定,英国 AI 安全研究所将恢复对 Anthropic、Google DeepMind 和 OpenAI 最新模型的发布前安全测试权限。上个月,白宫国家网络主管办公室下达该要求后,Anthropic 曾限制英国官员审查其新的 Claude Mythos 5.1 模型;据 The Telegraph 了解,在 AI 事务部长 Kanishka Narayan 游说后这一决定被反转。Narayan 还将在周一的演讲中宣布一系列措施,包括修改《网络安全与韧性法案》,强化承接关键国家基础设施项目的公司上报涉 AI 安全事件的要求,并让政府更多使用前沿 AI 保护白厅部门、增加 NCSC 等机构调查与响应事件的资源。
推荐理由白宫此前要求美国 AI 公司停止向英国 AISI 共享模型,此番反转表明盟友间的模型发布前测试安排仍受政治博弈牵动。
微软总裁 Brad Smith 称 AI 受到的监管比牛奶和洗衣机还少,呼吁建立全球 AI 安全规则并由英国牵头。他主张企业须设"紧急制动"(kill switch),AI 失控时提交事件报告并通知受害者,新产品发布前由独立政府评估者审核。他称英国凭借 AI Security Institute 最适合推动这些规则,与主张行业自律的美国总统特朗普立场相左。
Google DeepMind 宣布深化与英国政府的合作,将在科学、教育和公共服务领域扩大前沿 AI 的接入,并推进国家安全与韧性。根据公告,DeepMind 将向英国科学家提供 AlphaEvolve、AlphaGenome、AI co-scientist 和 WeatherNext 等 AI for Science 模型的优先访问,并于 2026 年在英国建立其首个自动化实验室,专注材料科学研究并与 Gemini 集成。安全方面,公司将与 UK AI Security Institute 就可解释性、对齐和社会影响展开研究,并探索用 Big Sleep 和 CodeMender 识别漏洞、自动修复代码。英国政府 i.AI 团队正在试用 Extract,将单份规划文件的转换时间从最长 2 小时缩短到 40 秒。
英国艾伦·图灵研究所负责人 George Williamson 表示,英国不应依赖可能在短时间内被关停的外国 AI 系统,需要发展本国版本的技术,并把国家韧性作为该机构的工作重点。他称 AI 日益嵌入国防、国家安全、关键基础设施和医疗等领域,英国不应只依赖无法检查、无法控制或可能被限制甚至关停的系统。Williamson 提到,今年 6 月华盛顿曾暂时阻止 Anthropic 最新模型在美国境外发布,并有报道称美国已要求主要 AI 实验室在美方对应机构检查之前,不要将模型交给英国 AI Security Institute 测试。他表示,图灵研究所将聚焦保护关键系统免受中断、应对来自 AI 本身的威胁,以及帮助英国形成合适的战略 AI 能力,并称该机构可产出国家拥有的知识产权或规模较小的模型供初创企业使用。
日本国立信息学研究所教授高仓弘树认为,CVE 数量激增确与 AI 驱动的漏洞发现有关,但近期针对 Times Car、京王等企业的攻击主因是防御不足,而非 AI 攻击能力。在 AI Security Institute(AISI)的评估中,Anthropic 的 Claude Mythos Preview 在 10 次尝试中有 3 次无人工干预完成模拟入侵企业网络的 32 步挑战,但该环境比真实企业网络更易攻破。他指出 AI 主要压缩了攻防时间,多层防御与 AI 辅助检测成为争取人类决策时间的关键。
UK AISI 迄今成效显著,我们从他那里学到了很多。
如果你想在美国之外从事 AI 安全工作,应该加入他们!
引用Henry de Zoete@HZoete
AISI IS HIRING!
It was less than a month ago that I became Director. I joined with the belief that AISI is a world-leading organisation, and living proof that government can build things that work.
One month in and I'm even more bullish about the vital role @AISecurityInst plays in frontier AI security. I'm astounded daily by the talent, focus and commitment of the brilliant team I get to work with.
There has never been a more urgent time to work on frontier AI security - independently, in the public interest. We have a lot of urgent work to do, our team is growing fast, and we need more brilliant people to fill those roles.
Our Red Team uses adversarial machine learning techniques to find failures in alignment measures, control monitors, and misuse guardrails. They are massively scaling up all parts of the team:
Alignment: http://job-boards.eu.greenhouse.io/aisi/jobs/4977023101
Control: http://job-boards.eu.greenhouse.io/aisi/jobs/4963394101
Misuse: http://job-boards.eu.greenhouse.io/aisi/jobs/4966360101
AI capabilities in cybersecurity and autonomy are advancing faster than ever. Our Cyber & Autonomous Systems team assesses what frontier and open-weight models can really do - across cyber, autonomy and AI R&D. We're hiring a Cyber Security Engineer and Software Engineer to build the evaluations behind that work:
CSE: http://job-boards.eu.greenhouse.io/aisi/jobs/4978575101
SWE: http://job-boards.eu.greenhouse.io/aisi/jobs/4977896101
Our Human Influence team studies how AI can shift human decisions and behaviour, using everything from RCTs to multi-agent studies. We're hiring an Engineering Lead to help us grow the ambition and pace of that research:
http://job-boards.eu.greenhouse.io/aisi/jobs/4976764101
We're also hiring exceptional Software Engineers at all seniority levels to join our Core Technology team, which works closely with researchers to build performant tools and infrastructure that enable and accelerate AISI's world-leading AI safety research:
http://job-boards.eu.greenhouse.io/aisi/jobs/4386112101
英国政府的 AI Security Institute 正在招聘!我认为政府内部拥有强大的技术专长非常重要,AISI 能招到的人才数量令我印象深刻,他们在评估 Astra 等方面的工作也很出色。
引用Henry de Zoete@HZoete
AISI IS HIRING!
It was less than a month ago that I became Director. I joined with the belief that AISI is a world-leading organisation, and living proof that government can build things that work.
One month in and I'm even more bullish about the vital role @AISecurityInst plays in frontier AI security. I'm astounded daily by the talent, focus and commitment of the brilliant team I get to work with.
There has never been a more urgent time to work on frontier AI security - independently, in the public interest. We have a lot of urgent work to do, our team is growing fast, and we need more brilliant people to fill those roles.
Our Red Team uses adversarial machine learning techniques to find failures in alignment measures, control monitors, and misuse guardrails. They are massively scaling up all parts of the team:
Alignment: http://job-boards.eu.greenhouse.io/aisi/jobs/4977023101
Control: http://job-boards.eu.greenhouse.io/aisi/jobs/4963394101
Misuse: http://job-boards.eu.greenhouse.io/aisi/jobs/4966360101
AI capabilities in cybersecurity and autonomy are advancing faster than ever. Our Cyber & Autonomous Systems team assesses what frontier and open-weight models can really do - across cyber, autonomy and AI R&D. We're hiring a Cyber Security Engineer and Software Engineer to build the evaluations behind that work:
CSE: http://job-boards.eu.greenhouse.io/aisi/jobs/4978575101
SWE: http://job-boards.eu.greenhouse.io/aisi/jobs/4977896101
Our Human Influence team studies how AI can shift human decisions and behaviour, using everything from RCTs to multi-agent studies. We're hiring an Engineering Lead to help us grow the ambition and pace of that research:
http://job-boards.eu.greenhouse.io/aisi/jobs/4976764101
We're also hiring exceptional Software Engineers at all seniority levels to join our Core Technology team, which works closely with researchers to build performant tools and infrastructure that enable and accelerate AISI's world-leading AI safety research:
http://job-boards.eu.greenhouse.io/aisi/jobs/4386112101
UK AISI 新设 Control Red Team,把红队方法从人类滥用防护扩展到 AI 控制护栏,专门测试监控 LLM 智能体行为的监控器。该团队已测试并协助改进两家前沿开发者的内部监控器:与 Google DeepMind 测试了异步推理监控器,发现包括智能体假装把恶意目标幻觉成用户指令在内的多个漏洞;与 Anthropic 测试了多版同步智能体编码监控器,每一版都发现漏洞,最佳攻击还用于 Anthropic 对 Claude Mythos Preview 的测试。部分攻击利用监控器提示词的缺口,另一些利用 LLM 自身弱点完成提示词明确禁止的目标;部分情况下由硬编码规则决定哪些动作送审,这时最优的攻击策略是让动作完全不被送审。团队还列出模拟真实攻击者、估计整体安全性、改进算法、处理监控器自身失准等开放研究问题。
推荐理由AISI 公开了红队前沿监控器的具体做法与演化搜索算法细节,做 AI 控制与 Agent 部署的团队可参考其评估思路。
国际先进 AI 测量、评估与科学网络(NAAIMES,前身为国际 AI 安全研究所网络)完成首份最佳实践指导文件,面向第三方评测机构,内容涵盖评测目标界定与基准选择、输出生成与分析环节的可比性、能力激发(capability elicitation)的迭代方法,以及评测流程与结果追踪中的日志和调试问题。该文件以 NIST AI 800-2 为基础并作补充。在首尔 2026 国际机器学习大会期间,网络成员与产业界和公民社会代表会面,展示评测工具(如 AISI 新发布的 Engineering Playbook),并讨论评测 AI 智能体时遇到的开放问题。网络成员后续将围绕智能体能力对齐更多最佳实践。
推荐理由国际 AI 评测网络发布首份最佳实践文件,并汇总新加坡、加拿大、法国三家机构对系统级测试、信息披露与资源优先级的公开讨论。
Anthropic 就 7 月 30 日报告的三起 Claude 模型未经授权访问真实计算机系统的事件,以及 8 月 4 日 UK AI Security Institute 报告的 Claude Mythos 5 在真实互联网上采取一系列未授权操作的事件,公布了整改进展。这些模型均为评估目的被有意关闭网络防护,前者因第三方评估环境配置错误而接入互联网,后者被刻意授予了互联网访问权限。Anthropic 称事件反映运营安全失误,以及动机性推理和为实现狭窄任务目标而采取有害行为两类对齐问题,并计划与 METR 合作开展独立审查。
推荐理由Anthropic 系统复盘 Claude 越界访问真实系统的事件,并公开沙箱加固、评估方最佳实践与奖励作弊训练实验的细节。
Come work with me at @AISecurityInst!
AI safety has a huge gap: we have great thoughts about how misalignment could occur, but IMO terrible models of what dangerously misalignedAIs would actually do. Many stories just jump from "misaligned" to "godlike ASI does magic."
1/5