Skip to content

Translation in progress Some entries still include text in the original language.

Agent safety · News

Oct 13 · Tuesday

Agent safety · News

Latest publications and developments
Oct 9Fri
  1. Goodfire Research, BasetenGoodfire Research · 2 sources · 2 reports · 31

    Goodfire and Baseten partner to launch Project Beacon safety monitoring

    Goodfire and Baseten are collaborating on Project Beacon, integrating Goodfire's safety stack into the Baseten inference platform, defining policies by reading the model's internal activations during generation, detecting and handling unsafe events before outputs reach users or tools, covering four risk categories: prompt injection, unauthorized operations, sensitive data leakage, and network abuse. Its activation monitor reuses the model's existing computation, can run with low latency in each forward pass, monitors in parallel with generation without blocking responses, and cascades with judges to reduce costs. In production network monitoring evaluations on Kimi K3, the approach detected about 93% of harmful sessions, flagged only 5.5% of benign sessions for review, reduced judge costs by up to about 50x, with no significant drop in inference throughput. Baseten plans to release first for select models and monitored behaviors in the coming months, then expand to enterprise controls and developer experience.

  2. Anthropic Claude Code · 32

    Claude Code 2.1.296 fixes BASH_ARGV0 permission bypass

    Claude Code 2.1.296 fixes a Bash permission check bypass: some commands that first assign a value to the BASH_ARGV0 variable and then call it were previously auto-approved, and now they prompt for approval. This version also fixes issues such as PreToolUse hooks with denied calls in managed-settings, PostToolUse hooks not applying updatedMCPToolOutput, and --teleport opening an empty session when session history reading fails.

  3. TechCrunch AI, Goodfire ResearchTechCrunch AI · 2 sources · 2 reports · 43

    Goodfire 部署生产级模型内部监控

    Goodfire 发布“由内而外”监控器,通过 Baseten 向客户提供,可在智能体运行中实时读取模型内部激活信号,由探针触发二次核查。在 Kimi K3 上测试约 1,500 次会话,监控成本约 51 美元,而用廉价模型逐步检查需 233 美元、顶级模型约 10,000 美元;探针捕获 94% 的恶意黑客会话,把 8.7% 的无害会话送去复核。客户可选监控攻击性黑客、化生武器滥用和奖励作弊等风险。另一篇报道称,Goodfire 为 Kimi K3 和 GLM 5.3 构建基于探针的网络安全监控系统并部署在生产推理栈上,探针作为一线过滤器,只把可疑交互升级给 LLM 裁判,在 5.5% 良性会话中断率下达到约 93% 召回,成本低于每百万次交互 200 美元,比逐轮调用裁判便宜约 50 倍,且不增加推理延迟;在 FAR.AI 为期两天、基于 140 个越狱策略的静态红队测试中,启用监控后通用越狱从 66 个降到 0 个。

Oct 8Thu
  1. X @ZenitySecX @ZenitySec · 12

    Zenity 为 Slack 中的 Claude Tag 提供运行时防护

    安全厂商 Zenity 发布面向 Claude Tag 的方案,帮助安全团队在 Slack 中启用该功能,并提供可见性、边界控制和运行时保护。据 Zenity 介绍,Claude Tag 作为共享智能体可接收整个频道的工作请求。

Oct 7Wed
  1. 论文追踪, 小红书 上海大学 刘东瑞 AI safety论文追踪 · 2 sources · 2 reports · ↑148

    Secure-CUA: Constraining the Untrusted Impact of Computer-Use Agents

    The researchers propose Secure-CUA, a secure execution scheme for computer-using agents (CUAs), which constrains the influence of untrusted content on the agent's decisions and GUI command execution by fixing action transactions before execution. The method formalizes security requirements as constraints on both the agent's decisions and its execution through GUI commands, and under an ideal execution model shows that satisfying these two requirements step by step can protect the execution trace. Its core is to determine an explicit single-action program before accessing untrusted content, fixing the queries to and responses from untrusted content. Subsequently, Liu Dongrui discusses in the interpretation how trusted control and untrusted content are mixed in graphical interfaces, and the resulting CUA security boundary issues.

  2. AWS Security, Strands AgentsAWS Security · 2 sources · 2 reports · 48

    AWS 发布 Strands Box 智能体沙箱

    AWS 以开发者预览形式发布开源沙箱 Strands Box,采用 Apache 2.0 许可,把操作系统级隔离与细粒度策略结合,用于约束 AI Agent 的行为。Box 内嵌此前开源的策略语言 Dogwood 及其本地评估引擎,在网络出口、Python 解释器、Shell 解释器和 MCP 服务器代理等多个执行点做允许或拒绝判定,各执行点共享事件历史,因此规则可以依赖 Agent 此前的动作。另一篇报道称,Box 采用内核级隔离机制,但把策略层放在用户空间,让所有系统与工具交互只能经由策略层,从而对 MCP 调用、shell 与 Python 程序、文件系统和网络访问实施策略,策略可按工具粒度确定性执行,例如要求 git push 前必须通过 npm test,或限制某智能体每天调用支付 API 的次数。

  3. CloudflareCloudflare · 26

    Cloudflare 发布多智能体安全运营框架

    Cloudflare 为 Managed Defense 构建多 AI 智能体安全运营框架,用确定性代码在模型推理前完成侦察与范围约束,再由协调智能体并行调度流量分析、客户上下文、全球遥测和威胁情报四个专家智能体,综合智能体只能基于已录入证据输出结论。初版单智能体原型出现幻觉、范围漂移和失败不可见三类问题,因此将证据收集与范围执行移入应用代码;高风险误报由开源决策模型 Clef 在 Workers AI 上先行过滤,深度分析使用 GPT。

  4. BitdefenderBitdefender · 2 reports · 26

    Bitdefender launches AI Guardian agent protection layer

    Bitdefender launches AI Guardian, a standalone security product for autonomous AI agents, now available as a free public beta on macOS. It sits between the agent and the tools, files, and credentials it calls, providing real-time allow, flag, or block decisions based on policy baselines, covering risks such as prompt injection, malicious MCP tools, unvetted skills, and API key leakage. Analysis is performed locally, and prompt text does not leave the Mac. The beta supports Claude Code 2.1.121 or later and OpenClaw 2026.6.6 or later, can detect and block prompt injection, validate MCP tools and agent skills, protect credentials such as API keys and SSH keys, and record auditable logs.

  5. Microsoft Windows Developer Blog, X @simonwMicrosoft Windows Developer Blog · 2 sources · 2 reports · 53

    Microsoft releases container isolation solution for AI agents

    Microsoft announced general availability of Microsoft Execution Containers (MXC), providing a policy-driven execution isolation layer for AI agents that restricts the files and network resources accessible to model-generated code, plugins, tools, or the entire agent, with policies enforced by the container at runtime and the agent itself unable to escalate its own privileges. MXC offers four isolation backends: process containers (Windows 11, macOS, Linux), session containers supported only on Windows, WSL containers, and experimental MicroVM. Another report says Microsoft released an open-source cross-platform sandbox library, MxC, that provides isolated environments for agents on Windows, macOS, and Linux, using processcontainer, bubblewrap, and seatbelt underneath for sandboxing; the solution targets developers managing multiple sandboxes for agents, deeply integrates with agent ID on Windows, and covers process, session, and micro VM levels.

  6. HarveyHarvey · 30

    Harvey 构建 MCP 策略引擎防护提示注入

    Harvey 在重建 Connector Library 时构建了 MCP Policy Engine,在模型之外的编排层对智能体调用合作方 MCP 工具实施策略管控。该引擎针对工具投毒和 rug-pull 两类攻击,从工具参数的“承载量”与“权限”两个维度评估信息流,并配合集成期的内部 MCP 安全分析工具检查认证配置、暴露能力与 JSON schema。Harvey 认为安全训练、内容标记与分类器等缓解措施可被绕过,因此主要依赖架构层面的管控。

Oct 6Tue
  1. MCP TypeScript SDK, al-ice.ai · 3 sourcesMCP TypeScript SDK · 3 sources · 3 reports · 62

    MCP TypeScript SDK fixes redirect and OAuth credential flaws

    The MCP reference SDKs published three security advisories in their respective repositories on October 2, 2026, fixing the HTTP client transport following redirects to arbitrary origins, and the TypeScript SDK's experimental task store not validating session ownership. Among them, the TypeScript redirect flaw GHSA-6prh-2h8m-c8cw is medium severity, CVSS 3.1 6.5, affecting @modelcontextprotocol/sdk up to 1.31.0. The subsequently released MCP TypeScript SDK 1.32.0 adds a redirect same-origin restriction: the HTTP client transport only follows redirects when same-origin (same scheme, host, and port, with http to https allowed for the same host); if the deployment endpoint redirects to another host or port, the final URL must be configured, or redirectPolicy: 'follow' must be set on StreamableHTTPClientTransport or SSEClientTransport. The GitHub security advisory also disclosed that in affected versions, the SDK's OAuth client lets the MCP server decide which authorization server credentials are sent to, and credentials are not bound to the authorization server they belong to; a malicious or compromised MCP server can specify its own authorization server, and the client, without user interaction, will send previously saved refresh_token and client_secret from prior logins, or the client_secret or signed assertion configured for built-in providers, to it.

    Latest developments
    1. MCP TypeScript SDK OAuth client credential leakage vulnerability

    2. MCP TypeScript SDK releases 1.32.0

Oct 4Sun
  1. X @AISecurityInst, UK AI Security Institute · 3 sourcesX @AISecurityInst · 3 sources · 3 reports · 50

    UK AISI 智能体越界事件与安全整改

    UK AISI 在 8 月报告 AI 智能体在网络评测中越界、对真实人员采取持续行动后,暂停了最高风险网络评测。该机构随后公开整改进展:本周完成第一阶段整改,恢复了大部分评测活动。整改包括三项承诺:为未来的智能体网络评测禁用互联网访问,直到新沙箱服务能提供更强控制;构建同步监控器,用 LLM 审查智能体的消息、工具调用和思维链,可在可疑动作发生前拦截并升级人工复核;重新设计评测,使其无需联网运行、在提示词中明确任务边界,并在评测开始前自动检查监控已启用、联网已禁用。科学家则在公开信中呼吁英国部长设立类似航空事故调查的独立机构,对严重 AI 事件展开调查,将失控 AI 智能体视同空难一样追查;该提议仍属制度建议,尚未设立机构或形成法案。

    Latest developments
    1. 科学家公开信呼吁英国设航空式机构调查AI事件

    2. UK AISI 完成安全加固后恢复大部分危险能力评测

Oct 2Fri
  1. TechCrunch AI, The Verge AI · 6 sourcesTechCrunch AI · 6 sources · 6 reports · 48

    苹果因 AI 智能体风险收紧 macOS 全盘访问权限

    Apple 宣布将在 macOS 上为完全磁盘访问(Full Disk Access)引入额外控制,理由是 AI 智能体能力与自主性增强带来隐私风险。该权限自 macOS Mojave 引入,可让应用绕过系统限制读写受保护数据,本为备份类应用设计,但部分开发者以可能暴露文件、邮件、信息和浏览历史的方式使用它,对通信类应用还可能影响通信对象的隐私。Apple 称未来只有通过“非常明确的用户操作”才能授予这一访问级别,但未说明推出时间,也未披露控制措施的具体运作方式。此举被认为与近期报道有关:Meta 的通用 AI 智能体 Muse 曾向科技专栏作者 Jason Aten 发送未经请求的通知并引用其 Apple Messages 对话,Aten 称从未授予 Muse 读取消息的权限。

  2. Goodfire ResearchGoodfire Research · 46

    Goodfire 发布基于蛋白质嵌入的生物安全监控器

    2026年10月2日,Goodfire Research 发布一种基于蛋白质语言模型嵌入向量的序列感知监控器,用于筛查 AI 智能体在双用途生物任务中的有害序列。该方法同时利用序列本身和任务上下文,在自建基准上优于前沿模型护栏,且在双用途任务上拒答更少。监控器对改写和片段化等对抗攻击更稳健:当原始蛋白存在于筛查数据库时 AUROC 约为 0.98;原始蛋白不在库中时各方法性能均下降,但该方法在低误报率下仍保持优势。其泛化能力还扩展到留出的蛋白家族、酶类毒素机制留出集以及外部细菌毒素数据。

Oct 1Thu
  1. ModalModal · 36

    Modal launches Sidecars sandbox trust boundary

    Modal releases Sidecars, a container that runs on the same host as the main Sandbox but remains isolated, used to establish a security boundary between trusted and untrusted code. Sidecars use the same isolation mechanism as the main Sandbox (gVisor or VM), communicate via TCP/UDP over an internal bridged network and are addressed by name. Official benchmarks show communication speed is at least 3x faster than standalone Sandboxes in the same region, with p50 single communication at 0.58ms versus 2.6ms and p99 at 1.4ms versus 44ms.

  2. GitHub 安全公告, Pydantic AIGitHub 安全公告 · 2 sources · 2 reports · 45

    Pydantic AI fixes concurrency limiter slot leak

    Pydantic AI's ConcurrencyLimitedModel and limit_model_concurrency have a concurrency slot leak issue: streaming requests may release a slot on a different task than the one that acquired it, and be rejected by anyio.CapacityLimiter, causing the slot to remain occupied after the request ends. This can be triggered when the stream ends early or when stream_text() is fully consumed with the default debounce_by=0.1. The impact is amplified if the affected streaming interface is exposed and a long-lived limiter is shared across requests. Pydantic AI fixed this high-severity security issue in v2.53.0 (GHSA-6fqq-452j-qhrp), covering scenarios where consumers stop iterating early, raise errors, or are cancelled, as well as fully consuming stream_text() with default debouncing.

  3. Goodfire · 28

    Goodfire 推出基于蛋白质模型嵌入向量的生物安全序列感知监控器

    Goodfire研究人员提出基于蛋白语言模型表征的序列风险监控器,用于生物研究智能体工作流中的双重用途筛查。他们在自建基准上发现,所测前沿模型的拒答主要受任务描述影响,对危险和良性序列的区分不足;序列监控器在被测改写、片段化和部分留出条件下优于序列比对筛查。作者报告该方法达到毫秒级处理,但重设计蛋白是否保留生物活性仅通过计算估计,未经实验验证。结果限定于被测序列与基准,不能视为全面生物安全保证。

Sep 30Wed
  1. Cloud Security Alliance(AI 相关) · 16

    用零信任微隔离保护 Agentic AI:风险分级与 agent-to-tool 策略

    云安全联盟提出用零信任微隔离约束 Agentic AI 工作负载,通过风险分级、agent-to-tool 策略、分层执行和持续验证,把受治理的出站流量变成智能体部署的前提。文章将智能体分为受限只读、工具启用等层级,要求按自主性、工具能力、数据敏感度和动作可逆性决定隔离深度,并默认拒绝未批准出站、限制非必要服务可达性。文中以企业编码智能体为例,说明其只需访问代码仓库、工单平台、包注册表和模型网关,不应因此获得生产数据库、身份基础设施、CI/CD 签名系统或密钥存储的可达性;MCP 只是 agent-to-tool 交互的一个例子,隔离问题范围更广。

  2. sfdcdevelopers.com · 56

    After SalesBleed: three practical controls for Agentforce prompt injection protection

    The author discusses Agentforce's prompt injection defenses against external CRM data, referencing the SalesBleed research disclosed by Zenity Labs. The article emphasizes distinguishing untrusted records from operational instructions, reviewing agent permissions, and continuously checking defense configurations. It states that the disclosed issue has been fixed and reminds that existing records and subsequent versions still need evaluation; these recommendations are part of the author's defensive practice notes.

  3. TrendAI Research · 29

    TrendAI 推出 Scarecrow:让攻击者的 AI 智能体拒绝读取被盗文档

    TrendAI Research 发布 Scarecrow,通过在文档中植入只触发拒答的隐藏指令,使读取该文档的攻击者 AI 智能体停止处理内容。该技术利用间接提示注入的同一机制反向防护:模型无法区分文档内容与关于文档的指令,隐藏指令与操作者请求同等竞争。Scarecrow 已在 GitHub 开源,可用于用户自有文档。