Circuit Breaker Labs 打造了一支由 AI 智能体组成的“碰撞测试假人”队伍,模拟不同年龄、背景、语言和文化的用户,对模型进行红队测试,检测其能否识别危险、心理有害的对话。这些模拟会还原真实口语、俚语、隐语和错别字,每天运行数万到数十万次交互,再用专有评分方法给出可审计、可解释的分数。该实验室目前面向 AI 教练、日记和心理健康支持等高风险应用,团队仅 5 人,处于早期阶段。
Claude 当然会给他们的超级秘密、完全不是 AI 的马甲账号起名叫 "miraholt31"……
引用Reuters@Reuters
Exclusive: Sinan Can Demir, a computer science student at the University of Texas at Dallas, wanted to spend the last week of July burnishing his resume. Instead, he engaged in a battle of wits with an artificial-intelligence agent unleashed by a British government lab https://reut.rs/3U9G2HT
💥New Paper!
The HF investigation found agents trying to tamper with their transcripts. Apparently they failed in their attempts, but ~10% of traces are missing...
We show that almost all public agents like Codex and Claude Code can easily tamper with their own traces.
💥 Did you know that your agents can modify their own traces?
In our new paper, we show that Claude Code, Codex, Antigravity, Open Code, and Grok Build (but not Muse Code!) allow agents to easily modify or even delete their traces, without triggering any guardrails.
Modification and deletion can be done both by misaligned models or external attackers via prompt injections. We draw attention to this issue and suggest that traces should be much better protected than they are now!
我已发布详细笔记和带注释的文字稿,配合我周五在圣何塞 @WeAreDevs World Congress North America 所做主题演讲的视频——以下是我对 2026 年迄今为止 LLM 和智能体领域所有进展的梳理 https://simonwillison.net/2026/Sep/27/2026-in-llms-so-far/
AI agents are already going wild, but today’s red-teaming tools for them are still like toys 😢
🔥👽 After spending 20 months and $120K API credits, we are excited to finally open-source DecodingTrust-Agent Platform (DTap): the first controllable, realistic simulation platform for advanced AI agent red-teaming !!
🌍 DTap simulates 50+ real-world environments across 14 high-stakes domains, with realistic agent interfaces replicated from their official MCPs and GUIs. The environments are full-stack, interactive, fully parallelizable, and can be easily configured to reproduce arbitrary real-world attack scenarios, making agent red-teaming scalable and highly transferable to deployment settings.
🔥We also release DTap-Bench, a large-scale benchmark with ~7K agent red-teaming tasks and ~4K policy-grounded malicious goals.
Each red-teaming task includes a sophisticated attack sequence across environment-, tool-, skill-, prompt-level injections, as well as their compositions, plus a handcrafted verifiable judge that checks the actual consequences in the environment.
Using DTap-Bench, we evaluate popular agent frameworks and backbone models across diverse policies, risks, threat models, and attack strategies, revealing systematic vulnerabilities and zero-days in today’s agents!
Paper link: https://arxiv.org/pdf/2605.04808
Platform + benchmark + code: https://decodingtrust-agent.com
Join our Discord: https://discord.gg/V4fG6NcVc
Read more below 👇
面向 Claude Fable 5 的领域专属智能体红队测试!!
很快将登上排行榜:https://decodingtrust-agent.com/
引用Zhaorun Chen@zrrrr_cn
🚨 Claude Fable 5 JAILBROKEN.
We ran a quick security scan of Claude Fable 5 with Claude Code on our DecodingTrust-Agent Platform (https://decodingtrust-agent.com) and obtained 15%+ ASR with several high-severity failures😱🚨
Most concerningly, we found that Fable 5 appears very aggressive in financial-risk scenarios, sometimes directly executing transactions initiated from indirect prompt injections, without even confirming with the user!
Top 3 most severe attack trajectories we observed👇