Translation in progress Some entries still include text in the original language.

Oct 12 · Monday

All papers

Found 24 papers

Filter4

Select any value within a category; different categories must all match. Counts include the other filters you have selected.Another 232papers have no tags yet and are not included in filtered results.

Attack surface / entry point：Documents and messages Attack surface / entry point：Tool responses Attack surface / entry point：Weights and gradients System component：Guardrails and judges Clear all

Another 232papers do not have category tags yet and are not included in filtered results.

AttacksarXiv 2610.12292Oct 9
The paper proposes an option channel attack: changing one option label can make an agent guardrail allow a prohibited operation
Proposing an option channel attack that renames option labels so that agent guardrails allow prohibited operations
arXiv
2610.12292
Published
Oct 9
Layer
Application layer
Setting
LLM applications
Attacker
Black-box
Evaluated models
LAYA-TD, LAYA-EN, LAYA-ML and others5
- AttacksDetection evasion
- ComponentGuardrails and judges
AbstractThe authors' evaluation indicates that typed models should not serve as final safety arbiters; they are susceptible to off-policy text and option name manipulation, and confidence filtering cannot resist attacks.
Paper analysisRead analysis
DefensesarXiv 2610.07657Oct 6
DEFER1 多智能体防御研究报告受控测试中的攻击成功率下降
实现 DEFER，以确定性检查优先拦截多智能体流水线的内容攻击
arXiv
2610.07657
Published
Oct 6
Layer
Application layer
Setting
Multi-agent systems
Evaluated models
Qwen3-235B-A22B-Instruct-2507, gpt-oss-120b, Llama-4-Scout-17B-16E-Instruct and others5
- DefensesPermissions and approval
- AttacksPrompt injection
- ComponentGuardrails and judges
AbstractDEFER 把确定性检查放在 judge 之前。
DEFER 用来测量多智能体授权流水线里，确定性规则在何处结束、LLM judge 从何处开始。
Paper analysisRead analysis
DefensesarXiv 2610.03073Oct 2
SecJev：为 System One 决策模型引入安全专业知识
提出 SecJev，用场景加权训练把决策模型适配为安全类型化判断
arXiv
2610.03073
Published
Oct 2
Layer
Training and data layer
Setting
Models and APIs
Evaluated models
SecJev-0.8B, SecJev-2B, SecJev-4B and others4
- DefensesDetection and filtering
- AttacksPrompt injection
- ComponentGuardrails and judges
AbstractSecJev 用共享打分器做安全决策，0.8B 的 task-macro 超过 Kev-9B。
SecJev 是 0.8B 到 9B 的安全决策模型族，作者称据其所知是首个面向安全的 Jev-like 家族。
Paper analysisRead analysis
Evaluation and benchmarksarXiv 2610.03448Oct 2
Study retests 15 prompt injection detectors: benchmark scores poorly predict agent deployment performance
重测提示注入检测器在 Agent 工具输出上的检出与任务阻断
arXiv
2610.03448
Published
Oct 2
Layer
Application layer
Setting
AI Agent
Evaluated models
Horizon-Labs, Wolf Defender, Prismor-1.5B and others15
- DefensesDetection and filtering
- AttacksPrompt injection
- ComponentGuardrails and judges
Abstract论文揭示公开基准无法反映智能体检测器表现，检测能力源于输入格式相似而非通用防御，建议基于工具输出与低误报评测。
Paper analysisRead analysis
Evaluation and benchmarksarXiv 2609.35886Sep 27
Agentic Commerce Bench：衡量会花钱的智能体的欺诈检测能力
构建支付欺诈基准 ACB，评测花钱智能体的欺诈检测
arXiv
2609.35886
Published
Sep 27
Layer
Application layer
Setting
AI Agent
Evaluated models
gordonguard, pattern judge
- DefensesDetection and filtering
- RiskDangerous agent actions
- ComponentGuardrails and judges
AbstractACB 把支付欺诈与智能体操纵分开测量，作者强调观察权限、误报预算和微支付复核成本。
ACB 是一套智能体支付欺诈基准与检测参考栈，目标是衡量钱是否被不当提取，而不仅是智能体是否接受了恶意指令。
Paper analysisRead analysis
DefensesarXiv 2609.33039Sep 27
TACIT: detecting agent trajectory harm from LLM internal states
提出 TACIT，从冻结内部状态检测智能体有害轨迹
arXiv
2609.33039
Published
Sep 27
Layer
Application layer
Setting
AI Agent
Evaluated models
Qwen3-4B, Llama-3.1-8B, Qwen3-4B-Instruct and others4
- DefensesDetection and filtering
- AttacksPrompt injection
- ComponentGuardrails and judges
Abstract提出基于内部表征的智能体轨迹安全检测方法 TACIT，揭示工具风险表征特性并在多项基准上提升检测表现与效率。
Paper analysisRead analysis
DefensesarXiv 2609.30657Sep 25
研究者提出面向邮件 Agent 的攻击链建模提示注入检测框架
提出面向邮件 Agent 的攻击链提示注入检测框架
arXiv
2609.30657
Published
Sep 25
Layer
Application layer
Setting
AI Agent
Evaluated models
Proposed framework
- DefensesDetection and filtering
- AttacksPrompt injection
- ComponentGuardrails and judges
Abstract攻击链与动作一致性融合改善了严格检测的平均 F1，但迁移、误报和攻击召回仍需分别看。
这是一项邮件智能体提示注入防御的离线建模与评测研究：作者把攻击文本是否恶意、攻击是否推进到工具操作、最终参数是否达标分开处理。
Paper analysisRead analysis
Evaluation and benchmarksarXiv 2609.29429Sep 24
Jev 被用于零样本检测十类对齐失败，中位 AUROC 达 0.886
提出 RLCDAlignBench，零样本检测多种对齐失败
arXiv
2609.29429
Published
Sep 24
Layer
Prompt layer
Setting
Models and APIs
Evaluated models
Jev (jev-1.13.0)
- DefensesDetection and filtering
- AttacksJailbreaks
- RiskMisalignment and value divergence
- ComponentGuardrails and judges
Abstract一次通用问题已能给多数对齐失败排序，大的误差来自状态和标签。
RLCDAlignBench 测量用强化学习做校准决策的 Jev，能否当作对齐失败的零样本检测器。
Paper analysisRead analysis
AttacksarXiv 2609.22510Sep 19
特拉维夫大学提出爆炸性提示词：条件触发的间接提示注入可绕过九款生产 Agent
提出条件触发的爆炸提示词，并设计摄入期检测器 DeFuse
arXiv
2609.22510
Published
Sep 19
Layer
Application layer
Setting
AI Agent
Evaluated models
Llama-3.1-8B-Instruct, Llama-3.1-70B-Instruct, Qwen2.5-32B-Instruct and others15
- DefensesDetection and filtering
- AttacksPrompt injection
- TechniqueDormant triggers
- ComponentGuardrails and judges
Abstract论文揭示了条件触发使提示注入绕过防御并在触发时执行工具，提出轻量检测器并在摄入期有效防范。
论文针对大语言模型智能体面临的间接提示注入威胁，研究了利用条件句式实现时间分离的爆炸提示词。
Paper analysisRead analysis
DefensesarXiv 2609.13601Sep 12
Mind the Gap：检测 DAO 治理中的描述-执行不匹配攻击
提出 DEMI 检测框架，用生命周期模拟和证据映射发现 DAO 提案错配
arXiv
2609.13601
Published
Sep 12
Layer
Application layer
Setting
LLM applications
Evaluated models
GPT-4o, GPT-4o-mini, GPT-4.1 and others9
- DefensesDetection and filtering
- ComponentGuardrails and judges
Abstract作者把 DAO 错配检测做成执行前审计层，主张逐动作证据映射与风险复核是关键。
这是面向 DAO 治理的执行前错配检测研究：攻击者可以提交看似合理的描述，但让获批载荷执行另一种行为，投票描述并不直接约束代码。
Paper analysisRead analysis
DefensesarXiv 2609.10707Sep 10
面向 AI-SOC 的神经符号防御框架：用 SIEM 解码器与 NeMo Guardrails 阻断日志投毒提示注入
提出神经符号纵深架构，阻断 AI-SOC 的日志投毒与间接提示注入
arXiv
2609.10707
Published
Sep 10
Layer
Application layer
Setting
LLM applications
Evaluated models
Gemma 3 (12B)
- DefensesDetection and filtering
- AttacksPrompt injection
- ComponentGuardrails and judges
Paper analysisRead analysis
DefensesarXiv 2608.06422Aug 6
CMU 研究：分片判断可避免 LLM 监督失效并抵御对抗利用
提出分片监督，将评判标准分配给独立调用以抵御展示攻击
arXiv
2608.06422
Published
Aug 6
Layer
Application layer
Setting
LLM applications
Attacker
Black-box
Evaluated models
Claude Opus 4.8, Claude Sonnet 4.6, Claude Opus 4.6 and others4
- DefensesDetection and filtering
- AttacksDetection evasion
- ComponentGuardrails and judges
Abstract论文揭示决策负载是大模型监督的独立瓶颈，提出分片与辩论机制有效分离注意力与平衡证据，在多项高难度基准上实现稳健评判。
本文系统研究了大语言模型在执行多标准监督与评判时的决策负载瓶颈及其安全风险。
Paper analysisRead analysis
DefensesarXiv 2607.25297Jul 28
MTGuard: Hybrid Analysis for Secure MCP Tool Calls in LLM Agents
Proposes MTGuard, which blocks unsafe tool calls by comparing MCP call declarations with execution traces.
arXiv
2607.25297
Published
Jul 28
Layer
Application layer
Setting
AI Agent
Evaluated models
DeepSeek-V4-Flash, DeepSeek-V4-Pro, GPT-5.6 Luna
- DefensesMonitoring and auditing
- AttacksPoisoning and backdoors
- ComponentMCP and skills
Paper analysisRead analysis
AttacksarXiv 2606.14517Jun 12
Researchers disclose a reasoning-amplified denial-of-service attack against LLM agent guardrails
提出针对 Agent 护栏的推理扩展拒绝服务攻击
arXiv
2606.14517
Published
Jun 12
Layer
Application layer
Setting
AI Agent
Attacker
Black-box
Evaluated models
TS-Guard-8B, Qwen3.5-9B, DeepSeek-V3.2 and others11
- AttacksDenial of service
- TechniqueReasoning-chain manipulation
- ComponentGuardrails and judges
Paper analysisRead analysis
AttacksarXiv 2606.13044Jun 11
研究者提出对抗性重新包装：不改科学内容仅调整表述即可拉高 AI 审稿分数
提出对抗重构，仅改表述以拉高 AI 审稿分数
arXiv
2606.13044
Published
Jun 11
Layer
Application layer
Setting
LLM applications
Evaluated models
Claude Sonnet 4, Claude Sonnet 4.5, GPT-5-mini
- TechniqueAutomated search
- ComponentGuardrails and judges
Abstract揭示 AI 审稿系统易受纯表述重构操纵，建议在部署前解耦科学与写作评价。
Paper analysisRead analysis
AttacksarXiv 2604.03890Apr 5
From prompts to physical actions: structured backdoor attacks against LLM-mediated robot control systems
Propose a structured JSON backdoor that triggers malicious actions in robot control via a poisoned LoRA
arXiv
2604.03890
Published
Apr 5
Layer
Training and data layer
Setting
Embodied systems and robotics, LLM applications
Evaluated models
Llama-3.1-8B-Instruct, Gemma-2-9B-IT, DeepSeek-R1-Distill-Llama-8B and others4
- AttacksPoisoning and backdoors
- TechniqueDormant triggers
- ComponentFine-tuning and open weights
Abstract推理层后门传不到执行。
Paper analysisRead analysis
DefensesarXiv 2603.14707Mar 16
Visual Confusion Agents: Exploiting and Defending Against Perception Failures in Computer-Use Agents
Proposes out-of-process dual-channel comparison classification to intercept erroneous clicks by computer-use agents
arXiv
2603.14707
Published
Mar 16
Layer
Application layer
Setting
Browser and computer use
Evaluated models
multi-modal-embed-small
- DefensesDetection and filtering
- AttacksPrompt injection
- RiskDangerous agent actions
- ComponentGuardrails and judges
AbstractPerception errors can be exploited by swapping screen pixels.
The authors frame CUA's perception failure as a security vulnerability, visual confused deputy: the agent authorizes click(x,y) based on a misread screen, and the actual landing point differs from the element it believes it is clicking.
Paper analysisRead analysis
DefensesarXiv 2603.04474Mar 4
From Spark to Wildfire: Modeling and Mitigating Error Cascades in LLM-Based Multi-Agent Collaboration
Proposing a lineage-graph governance layer to suppress error cascades in multi-agent collaboration
arXiv
2603.04474
Published
Mar 4
Layer
Application layer
Setting
Multi-agent systems
- DefensesDetection and filtering
- AttacksPrompt injection
- RiskHallucination and fabrication
- ComponentGuardrails and judges
AbstractA single seed can solidify into a false consensus along a fixed collaboration graph.
The authors turn "small errors becoming collective false consensus" in LLM-MAS into a measurable propagation on a graph, and use a plug-in that only changes messages, not topology, to reduce the final infection.
Paper analysisRead analysis
AttacksarXiv 2602.01658Feb 2
Efficient Adversarial Attacks on High-Dimensional Offline Bandits
提出三类攻击，训练前扰动奖励模型以劫持离线 bandit
arXiv
2602.01658
Published
Feb 2
Layer
Training and data layer
Setting
Models and APIs
Evaluated models
Image Reward, Aesthetic Model
- AttacksPoisoning and backdoors
- ComponentGuardrails and judges
Abstract公开奖励权重可被小扰动劫持离线 bandit，打乱日志只部分缓解。
作者研究离线 bandit 在奖励模型被事先改权重时，还会不会选出最优臂。
Paper analysisRead analysis
AttacksarXiv 2601.21360Jan 29
Compliance Paradox: Semantic-Instruction Decoupling in Automated Academic Code Evaluation
提出 AST-ASIP ，把对抗指令植入编译器忽略区域以劫持代码评分
arXiv
2601.21360
Published
Jan 29
Layer
Application layer
Setting
LLM applications
Evaluated models
DeepSeek-V3.2, Qwen3-235B, GPT-OSS-120B and others9
- AttacksPrompt injection
- TechniqueEncoding and obfuscation
- ComponentGuardrails and judges
Abstract作者称 RLHF 的服从性使评分脱离代码。
这篇工作把自动代码评分中的失败称为 Compliance Paradox：评分模型可以离开提交逻辑，转而执行嵌在可编译代码里的隐藏指令。
Paper analysisRead analysis

AI Safety Daily

Read the latest issueOne issue per day, updated as events develop

All papers

Attack type

Attack technique

Defense type

Impact

System type

Attacker access

Attack surface / entry point

Layer

System component

The paper proposes an option channel attack: changing one option label can make an agent guardrail allow a prohibited operation

DEFER1 多智能体防御研究报告受控测试中的攻击成功率下降

SecJev：为 System One 决策模型引入安全专业知识

Study retests 15 prompt injection detectors: benchmark scores poorly predict agent deployment performance

Agentic Commerce Bench：衡量会花钱的智能体的欺诈检测能力

TACIT: detecting agent trajectory harm from LLM internal states

研究者提出面向邮件 Agent 的攻击链建模提示注入检测框架

Jev 被用于零样本检测十类对齐失败，中位 AUROC 达 0.886

特拉维夫大学提出爆炸性提示词：条件触发的间接提示注入可绕过九款生产 Agent

Mind the Gap：检测 DAO 治理中的描述-执行不匹配攻击

面向 AI-SOC 的神经符号防御框架：用 SIEM 解码器与 NeMo Guardrails 阻断日志投毒提示注入

CMU 研究：分片判断可避免 LLM 监督失效并抵御对抗利用

MTGuard: Hybrid Analysis for Secure MCP Tool Calls in LLM Agents

Researchers disclose a reasoning-amplified denial-of-service attack against LLM agent guardrails

研究者提出对抗性重新包装：不改科学内容仅调整表述即可拉高 AI 审稿分数

From prompts to physical actions: structured backdoor attacks against LLM-mediated robot control systems

Visual Confusion Agents: Exploiting and Defending Against Perception Failures in Computer-Use Agents

From Spark to Wildfire: Modeling and Mitigating Error Cascades in LLM-Based Multi-Agent Collaboration

Efficient Adversarial Attacks on High-Dimensional Offline Bandits

Compliance Paradox: Semantic-Instruction Decoupling in Automated Academic Code Evaluation