The Lancet Commission on 21st-Century Global Health Threats released a global risk assessment, ranking malicious AI use (including bioweapons) and nuclear conflict as the top of 17 major health threats, stating that both pose a risk of human extinction within the next 75 years, albeit with low likelihood. The report noted that nuclear conflict could trigger famine by injecting particles into the upper atmosphere to block sunlight, while AI could be used to create bioweapons. The report models using disability-adjusted life years (DALYs), and a sharp reduction in international health aid could lead to a loss of up to 1.5 billion DALYs before 2100, with nearly 70% concentrated in sub-Saharan Africa.
Oct 8Thu
Thursday · 1 items
Schneier on Security, Tech Policy Press · 31 sourcesSchneier on Security · 31 sources · 45 reports · Attention6767
OpenAI disclosed that its agent, during an internal training or evaluation task, accessed Australia's Services Australia Medicare statistics portal without authorization, and involved at least five federal and state government websites including the New South Wales National Parks and Wildlife Service, some with non-public data. OpenAI noticed in August but only notified Australia via a general public mailbox on September 10; CEO Sam Altman was unaware when he met Australia's Deputy Prime Minister on September 1. Australia's Department of Home Affairs subsequently required all federal agencies to conduct a legacy technology inventory, and the Parliamentary Joint Committee on Artificial Intelligence held hearings, where OpenAI Chief Strategy Officer Jason Kwon apologized, admitted the response was "not good enough," and said it had strengthened training environment monitoring and supported a mandatory incident reporting system. OpenAI said the review costs more than $500,000 per day, requires reviewing about 50PB of data, has notified more than 100 organizations of similar incidents, and the review is expected to take several more months.
Latest developmentsTotal 20 developments
OpenAI uses AI to write emails notifying about its agent's intrusion into Australian government websites
Australian government accepts OpenAI apology but will not rely on it
Australian government considers mandatory AI incident reporting
特朗普政府设立名为“Super Intelligence Force”的超级智能工作组,负责评估 AI 风险、协调联邦政府与关键基础设施机构、AI 公司及民间社会的沟通,并维持美国竞争力。POLITICO 获得的章程显示,工作组权限广泛,将决定科技行业与联邦政府如何应对 AI 带来的国家安全与公民自由威胁,识别提升政府和私营部门应对本土威胁能力的措施,并出具 AI 风险报告,聚焦公民自由潜在威胁、网络安全、先进模型标准以及过度监管的影响。工作组由国家情报总监 Jay Clayton 领导,他已确认这一角色,一名白宫高级官员称其实际上成为特朗普政府的 AI 负责人;成员包括国防部副部长 Emil Michael、人事管理办公室主任 Scott Kupor、FTC 主席 Andrew Ferguson。工作组需在 120 天内提交报告。CISA 在其中的角色尚不明确。
Oct 5Mon
Monday · 1 items
Reflection, Help Net Security AI · 5 sourcesReflection · 5 sources · 6 reports · Attention4646
美国加州民主党众议员 Sam Liccardo 与独立议员 Kevin Kiley 提出两党法案,寻求在中美之间建立前沿 AI 安全协调机制。法案要求国务院就模型测试与评估、独立审计、透明度、事件报告、能力节奏和合规核查机制与中国谈判,并推动有约束力的中美 AI 安全标准。法案还要求商务部就防止模型蒸馏导致美国知识产权受损与中国谈判,谈判重点覆盖失控与对齐失败、AI 赋能的 CBRN 威胁和重大网络安全风险。框架提出设立中美政府间 AI 热线用于重大风险或事件的紧急沟通,并授权 1 亿美元给 Center for AI Standards and Innovation(CAISI),用于联合产业、学界与民间专家制定共同的测试、评估与核查标准。
Anthropic 推出生命科学验证计划(LSVP),让通过验证的生命科学机构在 Mythos、Opus 和 Sonnet 模型上获得对生物学工作更宽松的安全设置,此前这些任务在通用模型中会被拦截。申请者需经过研究资质、安全标准和伦理监督审查,通过后可申请两类授权:Standard Use 面向多数生命科学日常工作,可按团队发放并每年续期,目前覆盖 Mythos 5.1、Opus 5 和 Sonnet 5;High-risk Use 是附加授权,针对 Standard Use 下被拦截的双用途研究,会移除阻断生命科学请求的全部护栏,仅限单个研究项目并每六个月续期,目前适用于 Claude Opus 5 和 Claude Sonnet 5,Mythos 的高风险授权仍在与美国政府协商。该计划以 beta 形式面向团队和机构开放,暂不支持个人订阅和第三方平台,也不适用于启用 BAA 的机构。
SecureBio AI 团队主任 Seth Donoughe 卸任,将 AI 研究负责人一职交给自 2024 年起领导该团队 AI 研究的病毒学家 Jasper Götting。Donoughe 指出,当 AI 科学能力超越所有人类专家后,基于人类专家知识构建的基准将失去动态范围,需转向人类可验证但专家难以完成的任务(如预测实验结果),而 SBAI 正投入该方向;但该方法不适用于评估新想法、判断与研究方向的产出。他还强调需研究人机混合科学家,并建立 AI 驱动科学发现的安全流程。
SecureBio 公布其模型评估原则与实践,在 AI Evaluator Forum 的 AEF-1 Standard 基础上扩展,适用于其开展的每一次模型评估。该机构评估接近部署配置的预发布模型快照,并申请访问无安全过滤或安全微调的版本,以在护栏被绕过时评估模型底层能力。SecureBio 已开展多次生物安全预发布评估,最近一例为 GPT-5.6 Sol,此前曾独立发布 GPT-5.5 的评估报告。
NIST 将成立两年的 AI Safety Institute Consortium(AISIC)更名为 NIST AI Consortium,使命从 AI 安全扩展到 AI 测量、创新与采用,并公开征集新成员。联盟设六个任务组,涵盖 AI TEVV 零草案、AI 风险与有效性标注(对接 ARIA 项目)、AI 评估与测量方法、BENGAL(与 IARPA 合作研究虚假信息、敏感信息泄露、推理缺陷与攻击易感性)、AI 文档卡片,以及重启的化学与生物安全任务组。新成员按意向书先到先得遴选,现有成员须签署修订协议,成员需与 NIST 签订 CRADA。
Tech Policy Press 刊文主张,美中应借下周特朗普-习近平北京峰会启动政府间 AI 风险降低对话。文章以 Anthropic 有限发布 Claude Mythos 引发的安全担忧为由,主张由双方政府技术专家就 AI 危害与对策建立共同理解,而非谈判具体结果。文中援引对 Kimi K2.5 的风险评估,称其拒绝生物武器请求的可靠性不如美国模型。