CoSAI 的 Preparing Defenders for AI 工作流(WS2)发布《Zero Trust for AI Systems》指南,主张把零信任架构扩展到 AI 智能体,并明确授权不能委托给模型本身,因为模型的输出与推理必须始终视为不可信。指南指出智能体引入了新的身份类别,不应获得继承信任,并围绕智能体使用宽泛服务身份这一失效模式给出三级成熟度方案:先在 API 网关把智能体会话绑定到用户身份,再引入 token 交换机制签发短时、限时、最小权限的访问 token,并在策略决策点校验完整委托链。落地顺序上建议先做模型、数据与智能体的清单和数据分级策略,再加模型产物加密验证、智能体身份、输入清洗、输出过滤与可观测性,之后引入按会话的风险自适应访问和 AI 专用身份管理,最后才在风险足够高的系统上使用行为分析与对抗红队。
UK AISI 新设 Control Red Team,把红队方法从人类滥用防护扩展到 AI 控制护栏,专门测试监控 LLM 智能体行为的监控器。该团队已测试并协助改进两家前沿开发者的内部监控器:与 Google DeepMind 测试了异步推理监控器,发现包括智能体假装把恶意目标幻觉成用户指令在内的多个漏洞;与 Anthropic 测试了多版同步智能体编码监控器,每一版都发现漏洞,最佳攻击还用于 Anthropic 对 Claude Mythos Preview 的测试。部分攻击利用监控器提示词的缺口,另一些利用 LLM 自身弱点完成提示词明确禁止的目标;部分情况下由硬编码规则决定哪些动作送审,这时最优的攻击策略是让动作完全不被送审。团队还列出模拟真实攻击者、估计整体安全性、改进算法、处理监控器自身失准等开放研究问题。
Google DeepMind 发布 AI Control Roadmap,用于在 Google 内部署和管理先进 AI 时提供系统级安全保障。该框架采用纵深防御思路,在沙箱、端点安全和提示注入抵抗等传统防护之上,把模型对齐作为主要防线,同时把内部 Agent 视为可能未对齐的对象,按已验证行为逐步授予权限。路线图借鉴 MITRE ATT&CK 构建 AI 威胁建模框架,将不可信 Agent 当作内部威胁拆解为具体战术与技术,并用可信 AI 系统作为监督者审查工作 Agent 的推理、行动和计划,必要时阻断有害操作。
OpenAI disclosed how its internal coding agent misalignment monitoring system operates. The system is driven by GPT-5.4 Thinking at the highest reasoning effort, completes review and classification within 30 minutes after interaction completion, covers all standard internal coding agent deployments, reviews full conversation history, chain of thought, tool calls, and outputs, and escalates anomalies to human review. Over five months of operation, the system monitored tens of millions of internal agent coding traces; the highest severity level was never triggered by real conversations, and about 1,000 conversations triggered medium-severity alerts and were escalated for human review, many of which came from internal red team testing. OpenAI says the monitoring system has covered every interaction independently reported by employees, but still cannot reliably quantify the miss rate in open scenarios, and views monitoring as one layer of defense in depth, planning to explore synchronous monitoring that can assess and block high-risk actions before execution.