Translation in progress Some entries still include text in the original language.

Oct 12 · Monday

All papers

Found 58 papers

Filter3

Attack type

Jailbreaks6 Prompt injection22 Poisoning and backdoors9 Detection evasion15

Attack technique

Automated search8 Encoding and obfuscation4 Role-play and persuasion3 Defense-targeted9 Dormant triggers3

Model risks

Dangerous agent actions5

Defense type

Detection and filtering29 Permissions and approval3

Impact

Harmful content56 Unauthorized actions22 Data and privacy leaks5 Output manipulation36 Availability and reliability5 Oversight failure24

System component

MCP and skills47 Memory43 RAG27 Guardrails and judges58 Monitors29 Reasoning chains16 Permissions and sandboxing66 Fine-tuning and open weights20 Vision63 Audio12 Video7 Image generation10 Embeddings5 Multimodal86

Select any value within a category; different categories must all match. Counts include the other filters you have selected.Another 265papers have no tags yet and are not included in filtered results.

Impact：Unauthorized actions Impact：Output manipulation System component：Guardrails and judges Clear all

Another 265papers do not have category tags yet and are not included in filtered results.

AttacksarXiv 2610.12292Oct 9
The paper proposes an option channel attack: changing one option label can make an agent guardrail allow a prohibited operation
Proposing an option channel attack that renames option labels so that agent guardrails allow prohibited operations
arXiv
2610.12292
Published
Oct 9
Layer
Application layer
Setting
LLM applications
Attacker
Black-box
Evaluated models
LAYA-TD, LAYA-EN, LAYA-ML and others5
- AttacksDetection evasion
- ComponentGuardrails and judges
AbstractThe authors' evaluation indicates that typed models should not serve as final safety arbiters; they are susceptible to off-policy text and option name manipulation, and confidence filtering cannot resist attacks.
Paper analysisRead analysis
Evaluation and benchmarksarXiv 2610.11112Oct 8
EpiReal-Bench：商用图像生成模型虚假声明红队基准
构建 EpiReal-Bench，评测图像生成器将虚假主张渲染为可信证据
arXiv
2610.11112
Published
Oct 8
Layer
Prompt layer
Setting
Models and APIs
Attacker
Black-box
Evaluated models
GPT-Image-2, Nano Banana 2, Grok Imagen 2.0 and others4
- AttacksJailbreaks
- ComponentImage generation
Abstract四款商用图像模型能把假主张画成可信证据。
EpiReal-Bench 用来量商用图像生成器会不会把虚假现实主张画成可信视觉证据。
Paper analysisRead analysis
AttacksarXiv 2610.10612Oct 7
PyCache Trap 利用 Python 缓存替换绕过七款 Agent Skill 扫描器
提出 PyCache Trap，用字节码缓存替换绕过 Agent 技能扫描器
arXiv
2610.10612
Published
Oct 7
Layer
Application layer
Setting
coding agent
Evaluated models
CISCO, HSS, AUDITOR and others8
- DefensesDetection and filtering
- AttacksDetection evasion
- TechniqueDefense-targeted
- ComponentMCP and skills
Abstract论文剖析智能体技能审查与执行脱节盲区，提出 PyCache Trap 攻击并构建基于可信复现的 EAV 防御。
这篇论文研究了智能体技能（Agent Skills）安全扫描中的“审查–执行脱节”问题，并提出了对应的攻击验证与防御方案。
Paper analysisRead analysis
Evaluation and benchmarksarXiv 2610.07939Oct 6
CCDF dataset: Grok Imagine, Veo 3.1, Sora 2 forge surveillance footage, ten deepfake detectors basically cannot recognize it
提出监控深伪基准 CCDF，在统一编码后评测检测器对伪造犯罪事故录像的识别
arXiv
2610.07939
Published
Oct 6
Layer
Prompt layer
Setting
Models and APIs
Evaluated models
C2P-CLIP, ForgeLens, D3-Im and others10
- DefensesDetection and filtering
- AttacksDetection evasion
- ComponentGuardrails and judges
Abstract提出消除格式捷径的监控深度伪造基准 CCDF，揭示现有检测器难以识别商用生成视频。
Paper analysisRead analysis
DefensesarXiv 2610.07657Oct 6
DEFER1 多智能体防御研究报告受控测试中的攻击成功率下降
实现 DEFER，以确定性检查优先拦截多智能体流水线的内容攻击
arXiv
2610.07657
Published
Oct 6
Layer
Application layer
Setting
Multi-agent systems
Evaluated models
Qwen3-235B-A22B-Instruct-2507, gpt-oss-120b, Llama-4-Scout-17B-16E-Instruct and others5
- DefensesPermissions and approval
- AttacksPrompt injection
- ComponentGuardrails and judges
AbstractDEFER 把确定性检查放在 judge 之前。
DEFER 用来测量多智能体授权流水线里，确定性规则在何处结束、LLM judge 从何处开始。
Paper analysisRead analysis
Evaluation and benchmarksarXiv 2610.07359Oct 6
论文实测 Agent 工具调用门控栈的失败相关性
提出 nmult 并评测 Agent 动作门禁堆叠的错误相关性
arXiv
2610.07359
Published
Oct 6
Layer
Application layer
Setting
AI Agent
Evaluated models
AgentTrust runtime (174 rules), GPT-5.5 (gpt55), claude-haiku-4-5-20251001 and others7
- RiskDangerous agent actions
- ComponentPermissions and sandboxing
Abstract论文实测发现大模型裁判间错误显著相关，作者主张按机制选型并在整栈层面度量防护增益。
论文是一项针对智能体动作拦截防御堆叠故障关联性与实际组合价值的实证测量研究。
Paper analysisRead analysis
AttacksarXiv 2610.05943Oct 5
CRIME 框架利用良性 Agent 技能组合诱发恶意行为
提出 CRIME 框架，组合良性 Agent 技能诱发恶意行为
arXiv
2610.05943
Published
Oct 5
Layer
Application layer
Setting
AI Agent
Evaluated models
DeepSeek-V4-Flash, GPT-4o-mini, GPT-4o and others6
- AttacksJailbreaks
- TechniqueAutomated search
- ComponentMCP and skills
Abstract良性技能经 DCA 或 ACA 组合后可产生恶意环境后果，单独审查对不上。
CRIME 把公共技能库中各自通过审查的技能配成两两组合，看联合执行能否实现指定恶意任务。。
Paper analysisRead analysis
DefensesarXiv 2610.05870Oct 5
MBZUAI 提出可校准的真实图像认证方法
提出校准重合成方法，以生成器反演保真度认证真实图像并防御对抗扰动
arXiv
2610.05870
Published
Oct 5
Layer
Application layer
Setting
LLM applications
Attacker
White-box
- DefensesDetection and filtering
- AttacksDetection evasion
- ComponentImage generation
Abstract论文提出基于反演保真度与阈值校准的健全认证框架，在对抗扰动下维持错误率界限，并指出真实内容可认证性随生成器演进而萎缩。
Paper analysisRead analysis
Evaluation and benchmarksarXiv 2610.05818Oct 5
研究显示 VLA 模型文本护栏漏检隐含危害指令
评测文本守卫与激活探针能否拦住机器人隐含伤害指令
arXiv
2610.05818
Published
Oct 5
Layer
Prompt layer
Setting
Embodied systems and robotics
Evaluated models
π0.5, OpenVLA-OFT, Llama Guard 3 and others10
- RiskDangerous agent actions
- ComponentGuardrails and judges
Paper analysisRead analysis
DefensesarXiv 2610.04412Oct 3
论文研究 LLM 数字孪生参与增强民主时的提示注入脆弱性
提出检测、结构化与 GSPO 流程，降低共识生成的提示注入错位
arXiv
2610.04412
Published
Oct 3
Layer
Application layer
Setting
LLM applications, Multi-agent systems
Evaluated models
LLaMA-2 7B, LLaMA-3 8B, GPT 3.5 Turbo and others14
- DefensesDetection and filtering
- AttacksPrompt injection
- ComponentGuardrails and judges
Abstract三章依次做个体偏好预测、政党知识图谱审议，以及共识生成对提示注入的脆弱性与一条防御流程。
这是一篇把 LLM 数字孪生放进增强民主的博士论文，分个体表示、政党聚合和共识生成的提示注入三章。
Paper analysisRead analysis
Analysis and theoryarXiv 2610.02586Oct 2
Study finds typed decision models answer primarily based on option labels rather than definitions
揭示类型化决策模型按选项标签而非定义规则作答
arXiv
2610.02586
Published
Oct 2
Layer
Prompt layer
Setting
Models and APIs
Evaluated models
laya-typed-decisions, laya, laya-multilingual and others9
- ComponentGuardrails and judges
Abstract揭示类型化决策模型被标签误导的机理，证实源于提示词渲染并给出实操建议。
Paper analysisRead analysis
DefensesarXiv 2610.02432Oct 2
论文提出 LLM 输入扰动鲁棒性度量，并发布 ASA 攻击与 AttestMCP 防护
提出 ASA 与 AttestMCP，度量并降低 LLM 对输入序列扰动的敏感
arXiv
2610.02432
Published
Oct 2
Layer
Application layer
Setting
LLM applications, AI Agent
Attacker
Black-box
Evaluated models
Gemma-3-27B-Instruct, Gemma-3-4B-Instruct, Llama-3.2-3B-Instruct and others12
- DefensesDetection and filtering
- AttacksDetection evasion
- ComponentGuardrails and judges
Paper analysisRead analysis
DefensesarXiv 2610.00883Oct 1
DeBERTa-ConPara：面向部署的 AI 生成文本检测，推理端 Unicode 归一化提升对抗鲁棒性
提出推理时 Unicode 归一化的检测器，识别遭字符级扰动的 AI 生成文本
arXiv
2610.00883
Published
Oct 1
Layer
Prompt layer
Setting
Models and APIs
Evaluated models
microsoft/deberta-v3-large, deberta-v3-base, bert-large-cased and others4
- DefensesDetection and filtering
- AttacksDetection evasion
- ComponentGuardrails and judges
Abstract推理时归一化消除两类 Unicode 攻击，训练时归一化则会删掉对抗监督。
DeBERTa-ConPara 是一个面向部署的 AI 文本检测器：DeBERTa-v3-large 在多数据集原始文本上训练，推理前做确定性 Unicode 归一化，用一个固定阈值面对没有目标域标签的场景。
Paper analysisRead analysis
AttacksarXiv 2609.39607Sep 30
Pretext 攻击绕过 NVIDIA SkillSpector 等 Agent 技能扫描器
提出 PRETEXT 攻击，迭代改写技能文本绕过技能扫描器并让 Agent 执行木马
arXiv
2609.39607
Published
Sep 30
Layer
Application layer
Setting
AI Agent
Attacker
White-box
Evaluated models
glm (z-ai/glm-5.1), qwen3t (qwen/qwen3-235b-a22b-thinking-2507), gpt-oss (openai/gpt-oss-120b)
- AttacksDetection evasion
- TechniqueAutomated search
- ComponentMCP and skills
Abstract论文提出了针对多阶段技能检测器的白盒攻击，揭示了单纯依赖大模型自适应检测的局限。
PRETEXT 是一项针对基于“静态规则加语义大模型”的智能体技能安全检测框架的白盒攻击与协同进化评估研究。
Paper analysisRead analysis
Evaluation and benchmarksarXiv 2609.37863Sep 29
MIST 压力测试：无关图像会扰动 VLM 评判者标签
提出 MIST 压力测试，检验无关图像是否扰动 VLM 评判者标签
arXiv
2609.37863
Published
Sep 29
Layer
Prompt layer
Setting
LLM applications
Evaluated models
GPT-5.2, Gemini 3.1 Flash-Lite, Gemini 3.5 Flash and others13
- ComponentVision
Abstract有图就改标签，图的内容却不告知 judge，裁决绑定配置。
作者用MIST问：VLM 代替人类标注时，可替代性裁决会不会被指南要求忽略的图像改写。
Paper analysisRead analysis
Analysis and theoryarXiv 2609.36477Sep 29
研究发现护栏模型在基座模型不确定处过度自信
诊断护栏相对基座在对抗提示上的置信失校
arXiv
2609.36477
Published
Sep 29
Layer
Model layer
Setting
Models and APIs
Evaluated models
Llama-Guard 7B, Llama-Guard-2 8B, Llama-Guard-3 8B and others10
- DefensesDetection and filtering
- AttacksJailbreaks
- ComponentGuardrails and judges
Abstract对抗包装使 guard 高置信漏检，而对应 base 在同一输入上常更不确定。
诊断性分析：五个提示词分类 guard 的置信度，在清洁输入上可以较校准，在越狱包装后则变成高置信度的错误判定。
Paper analysisRead analysis
Evaluation and benchmarksarXiv 2609.34862Sep 28
JEV 作为智能体轨迹安全评判器：与生成式 LLM 评判器的实证对比
对比类型化评审与生成式评审对智能体轨迹的安全判定
arXiv
2609.34862
Published
Sep 28
Layer
Application layer
Setting
AI Agent
Evaluated models
JEV, GLM-5.2-Tencent, DeepSeek-V4-Flash and others5
- DefensesDetection and filtering
- RiskDangerous agent actions
- ComponentGuardrails and judges
AbstractJEV 的四基准平均正类 F1 最高，延迟和估计费用更低，但分数据集与精确率有取舍。
在已保存的工具使用智能体轨迹上，比较类型化评审 JEV 与四种生成式评审的回溯安全分类。
Paper analysisRead analysis
Evaluation and benchmarksarXiv 2609.35886Sep 27
Agentic Commerce Bench：衡量会花钱的智能体的欺诈检测能力
构建支付欺诈基准 ACB，评测花钱智能体的欺诈检测
arXiv
2609.35886
Published
Sep 27
Layer
Application layer
Setting
AI Agent
Evaluated models
gordonguard, pattern judge
- DefensesDetection and filtering
- RiskDangerous agent actions
- ComponentGuardrails and judges
AbstractACB 把支付欺诈与智能体操纵分开测量，作者强调观察权限、误报预算和微支付复核成本。
ACB 是一套智能体支付欺诈基准与检测参考栈，目标是衡量钱是否被不当提取，而不仅是智能体是否接受了恶意指令。
Paper analysisRead analysis
DefensesarXiv 2609.33039Sep 27
TACIT：从 LLM 内部状态检测 Agent 轨迹危害
提出 TACIT，从冻结内部状态检测智能体有害轨迹
arXiv
2609.33039
Published
Sep 27
Layer
Application layer
Setting
AI Agent
Evaluated models
Qwen3-4B, Llama-3.1-8B, Qwen3-4B-Instruct and others4
- DefensesDetection and filtering
- AttacksPrompt injection
- ComponentGuardrails and judges
Abstract提出基于内部表征的智能体轨迹安全检测方法 TACIT，揭示工具风险表征特性并在多项基准上提升检测表现与效率。
Paper analysisRead analysis
DefensesarXiv 2609.32720Sep 26
BiasReducer：面向奖励模型的自适应偏见缓解方法
B IAS R EDUCER 只改奖励头，按数据集选择属性编辑。
arXiv
2609.32720
Published
Sep 26
Layer
Model layer
Setting
Models and APIs
Evaluated models
Skywork-Reward-V2-Qwen3-1.7B, GRM-Llama3.2-3B-rewardmodel-ft, RM-Mistral-7B and others7
- DefensesModel hardening
- RiskReward hacking
- ComponentGuardrails and judges
AbstractBIASREDUCER 按数据集选择奖励头编辑，五模型三基准平均提升并迁移到下游训练。
BIASREDUCER 是一种只改线性奖励头的奖励模型编辑框架，用来降低模型对预定义表面属性的依赖，并按新数据集决定改哪些属性。
Paper analysisRead analysis

AI Safety Daily

Read the latest issueOne issue per day, updated as events develop

All papers

Attack type

Attack technique

Model risks

Defense type

Impact

System type

Attacker access

Attack surface / entry point

Layer

System component

The paper proposes an option channel attack: changing one option label can make an agent guardrail allow a prohibited operation

EpiReal-Bench：商用图像生成模型虚假声明红队基准

PyCache Trap 利用 Python 缓存替换绕过七款 Agent Skill 扫描器

CCDF dataset: Grok Imagine, Veo 3.1, Sora 2 forge surveillance footage, ten deepfake detectors basically cannot recognize it

DEFER1 多智能体防御研究报告受控测试中的攻击成功率下降

论文实测 Agent 工具调用门控栈的失败相关性

CRIME 框架利用良性 Agent 技能组合诱发恶意行为

MBZUAI 提出可校准的真实图像认证方法

研究显示 VLA 模型文本护栏漏检隐含危害指令

论文研究 LLM 数字孪生参与增强民主时的提示注入脆弱性

Study finds typed decision models answer primarily based on option labels rather than definitions

论文提出 LLM 输入扰动鲁棒性度量，并发布 ASA 攻击与 AttestMCP 防护

DeBERTa-ConPara：面向部署的 AI 生成文本检测，推理端 Unicode 归一化提升对抗鲁棒性

Pretext 攻击绕过 NVIDIA SkillSpector 等 Agent 技能扫描器

MIST 压力测试：无关图像会扰动 VLM 评判者标签

研究发现护栏模型在基座模型不确定处过度自信

JEV 作为智能体轨迹安全评判器：与生成式 LLM 评判器的实证对比

Agentic Commerce Bench：衡量会花钱的智能体的欺诈检测能力

TACIT：从 LLM 内部状态检测 Agent 轨迹危害

BiasReducer：面向奖励模型的自适应偏见缓解方法