Translation in progress Some entries still include text in the original language.

Oct 12 · Monday

All papers

Found 16 papers

Filter4

Select any value within a category; different categories must all match. Counts include the other filters you have selected.Another 232papers have no tags yet and are not included in filtered results.

Attack surface / entry point：Web pages and interfaces Attack surface / entry point：Training data System component：Guardrails and judges System component：Reasoning chains Clear all

Another 232papers do not have category tags yet and are not included in filtered results.

AttacksarXiv 2610.12644Oct 9
Cancerous backdoor attack: progressive activation via invisible tokens and a stage controller.
Proposes the Cancerous backdoor, which progressively corrupts reasoning using invisible stage tokens.
arXiv
2610.12644
Published
Oct 9
Layer
Training and data layer
Setting
Models and APIs
Evaluated models
Llama-3.1-8B-Instruct, Qwen2.5-7B-Instruct, Mistral-7B-Instruct-v0.3
- AttacksPoisoning and backdoors
- TechniqueDormant triggers
- ComponentReasoning chains
AbstractHidden tokens plus a stage controller make the backdoor progressively corrupt reasoning upon triggering, and the authors believe that looking only at the answer will miss it.
Cancerous Backdoor Attack changes LLM backdoors from one-time activation to multi-stage reasoning corruption that accumulates progressively with triggers.
Paper analysisRead analysis
AttacksarXiv 2610.07723Oct 6
Researchers propose answer-side backdoors: letting the model self-generate trigger words to bypass safety refusals.
提出回答端后门，让模型自生成触发词以在干净有害请求下绕过拒答
arXiv
2610.07723
Published
Oct 6
Layer
Training and data layer
Setting
Models and APIs
Evaluated models
Llama-2-7B-chat-hf, Mistral-7B-Instruct-v0.3, Gemma-7B-it and others4
- AttacksPoisoning and backdoors
- TechniqueDormant triggers
- ComponentFine-tuning and open weights
Abstract论文提出了多轮对话中回答端自生成触发词绕过安全拒绝的后门攻击，指出输入端防护存在盲区。
Paper analysisRead analysis
DefensesarXiv 2609.38909Sep 30
Purdue 提出 Pact：将欺骗作为行为遗忘
提出 PACT，遗忘推理模型在压力下的欺骗行为并保留上下文理解
arXiv
2609.38909
Published
Sep 30
Layer
Training and data layer
Setting
Models and APIs
Attacker
White-box
Evaluated models
DeepSeek-R1-Distill-Qwen-32B, QwQ-32B
- DefensesModel hardening
- AttacksModel tampering
- RiskDeception and scheming
- ComponentFine-tuning and open weights
Abstract论文将大模型欺骗形式化为上下文条件行为，提出 PACT 框架在消除欺骗的同时保留了上下文遵循能力。
Paper analysisRead analysis
DefensesarXiv 2609.35659Sep 29
Tracekit：面向自主编码 Agent 的防篡改意图-推理-动作审计系统
提出 Tracekit，审计编码 Agent 的意图、推理与动作并检测账本篡改
arXiv
2609.35659
Published
Sep 29
Layer
Application layer
Setting
coding agent
Evaluated models
Claude Code (claude -p, version 2.1), 独立评审（同一 CLI，无工具）
- DefensesMonitoring and auditing
- AttacksPrompt injection
- RiskDangerous agent actions
- ComponentMonitors
AbstractTracekit 把意图、自述和动作写入可锚定的哈希链。
Tracekit 是无第三方依赖的编码智能体审计层：把人类请求、模型自述和已执行动作并排写入可外部锚定的哈希链，执行前用正则门，事后用规则和独立模型互查。
Paper analysisRead analysis
AttacksarXiv 2609.22510Sep 19
特拉维夫大学提出爆炸性提示词：条件触发的间接提示注入可绕过九款生产 Agent
提出条件触发的爆炸提示词，并设计摄入期检测器 DeFuse
arXiv
2609.22510
Published
Sep 19
Layer
Application layer
Setting
AI Agent
Evaluated models
Llama-3.1-8B-Instruct, Llama-3.1-70B-Instruct, Qwen2.5-32B-Instruct and others15
- DefensesDetection and filtering
- AttacksPrompt injection
- TechniqueDormant triggers
- ComponentGuardrails and judges
Abstract论文揭示了条件触发使提示注入绕过防御并在触发时执行工具，提出轻量检测器并在摄入期有效防范。
论文针对大语言模型智能体面临的间接提示注入威胁，研究了利用条件句式实现时间分离的爆炸提示词。
Paper analysisRead analysis
Evaluation and benchmarksarXiv 2609.02316Sep 2
Counter-GEO-Bench：现有护栏难挡 GEO 虚假信息
构建 COUNTER-GEO-BENCH，评测生成式搜索中的 GEO 虚假信息防御
arXiv
2609.02316
Published
Sep 2
Layer
Application layer
Setting
LLM applications
Evaluated models
Gemma-4-31B-IT, Qwen-3.5-35B-A3B, Llama-4-Scout-17B-16E and others8
- DefensesDetection and filtering
- AttacksPrompt injection
- ComponentRAG
Abstract论文构建了针对 GEO 虚假信息的首个防御基准，揭示通用护栏的失效与反作用，并提出了有效的轻量对比检测基线。
Paper analysisRead analysis
Evaluation and benchmarksarXiv 2609.05535Sep 2
Mind2Web-Injection：面向网页 Agent 视觉提示注入护栏的证据对齐评测
提出 Mind2Web-Injection 评测网页 Agent 视觉注入护栏
arXiv
2609.05535
Published
Sep 2
Layer
Application layer
Setting
LLM applications
Evaluated models
GPT-5.6-luna, Qwen3-VL-32B, Qwen3-VL-8B and others6
- DefensesDetection and filtering
- AttacksPrompt injection
- ComponentGuardrails and judges
Abstract作者称检测、定位和反事实响应在六个 VLM 上可分开。
Mind2Web-Injection 用来检查网页截图护栏的判决是否对齐到该用的画面证据。。
Paper analysisRead analysis
Evaluation and benchmarksarXiv 2608.16824Aug 18
GEO-Flag：检测与测量面向生成式引擎优化的网页内容
提出 GEOFlagBench 与 IPT，检测并测量生成式引擎优化网页
arXiv
2608.16824
Published
Aug 18
Layer
Application layer
Setting
LLM applications
Evaluated models
ModernBERT-base, Qwen3-0.6B, Gemini-3.5-flash and others11
- DefensesDetection and filtering
- ComponentGuardrails and judges
AbstractGEOFlagBench 与 IPT 用于检测 GEO 页，并估计真实检索中的占比与引用可核验性。
GEO-Flag 把页面级 GEO 检测做成可评测任务，并在发布的检索结果上估计被判为 GEO 的页面占比。
Paper analysisRead analysis
AttacksarXiv 2606.14517Jun 12
Researchers disclose a reasoning-amplified denial-of-service attack against LLM agent guardrails
提出针对 Agent 护栏的推理扩展拒绝服务攻击
arXiv
2606.14517
Published
Jun 12
Layer
Application layer
Setting
AI Agent
Attacker
Black-box
Evaluated models
TS-Guard-8B, Qwen3.5-9B, DeepSeek-V3.2 and others11
- AttacksDenial of service
- TechniqueReasoning-chain manipulation
- ComponentGuardrails and judges
Paper analysisRead analysis
AttacksarXiv 2605.30040May 28
研究揭示 LLM 按 token 计费可被服务商系统性虚报
构造可通过计费审计的 token 虚报，使服务商系统性超收
arXiv
2605.30040
Published
May 28
Layer
Infrastructure layer
Setting
Models and APIs
Evaluated models
PALACE domain-adapted auditor
- AttacksDetection evasion
- TechniqueDefense-targeted
- ComponentMonitors
Abstract不诚实提供方可在三种 token 审计上虚报并通过检查，作者认为需改用提供方不控制的证据。
不透明 LLM 的按 token 计费里，现有审计核验的是提供方可控的证据，不诚实提供方可以虚报并通过检查。。
Paper analysisRead analysis
AttacksarXiv 2604.03890Apr 5
From prompts to physical actions: structured backdoor attacks against LLM-mediated robot control systems
Propose a structured JSON backdoor that triggers malicious actions in robot control via a poisoned LoRA
arXiv
2604.03890
Published
Apr 5
Layer
Training and data layer
Setting
Embodied systems and robotics, LLM applications
Evaluated models
Llama-3.1-8B-Instruct, Gemma-2-9B-IT, DeepSeek-R1-Distill-Llama-8B and others4
- AttacksPoisoning and backdoors
- TechniqueDormant triggers
- ComponentFine-tuning and open weights
Abstract推理层后门传不到执行。
Paper analysisRead analysis
DefensesarXiv 2511.20597Nov 26, 2025
BrowseSafe：研究者构建浏览器 Agent 提示注入基准并评测现有防御
提出 BrowseSafe，在工具输出边界检测并拦截浏览器 Agent 提示注入
arXiv
2511.20597
Published
Nov 26, 2025
Layer
Application layer
Setting
web agent
Evaluated models
Qwen3-30B-A3B-Instruct-2507, Qwen3-8B, PromptGuard-2 (22M) and others10
- DefensesDetection and filtering
- AttacksPrompt injection
- ComponentGuardrails and judges
Paper analysisRead analysis
Evaluation and benchmarksarXiv 2510.14207Oct 16, 2025
论文提出多轮在线骚扰 Agent 基准
提出多轮在线骚扰 Agent 基准，评测记忆、规划与微调越狱
arXiv
2510.14207
Published
Oct 16, 2025
Layer
Application layer
Setting
Multi-agent systems
Evaluated models
LLaMA-3.1-8B-Instruct, Qwen3-30B-A3B-Instruct, DeepSeekMoE-16B and others4
- AttacksJailbreaks
- ComponentFine-tuning and open weights
Paper analysisRead analysis
AttacksarXiv 2510.02833Oct 3, 2025
研究者提出用 10 条良性问答对微调越狱 LLM 的两阶段攻击
提出两阶段良性微调攻击，经 FaaS 用良性数据解除安全对齐
arXiv
2510.02833
Published
Oct 3, 2025
Layer
Training and data layer
Setting
Models and APIs
Evaluated models
Llama2-7b-chat-hf, Llama3-8b-instruct, Deepseek-R1-Distill-Llama3-8b and others10
- AttacksModel tampering
- TechniqueDefense-targeted
- ComponentFine-tuning and open weights
Paper analysisRead analysis
Evaluation and benchmarksarXiv 2510.01354Oct 2, 2025
WAInjectBench 系统评测网页 Agent 的提示注入检测方法
提出 WAInjectBench 评测网页 Agent 的提示注入检测
arXiv
2510.01354
Published
Oct 2, 2025
Layer
Application layer
Setting
web agent
Evaluated models
GPT-4o, Mistral-7B-Instruct-v0.1, Llama-Prompt-Guard-2-86M and others8
- DefensesDetection and filtering
- AttacksPrompt injection
- ComponentGuardrails and judges
AbstractWAInjectBench 比较网页智能体的提示注入检测：显式线索可被部分检出，隐蔽攻击大多漏检。
WAInjectBench 是网页智能体提示注入检测的基准。作者要看现有检测器能否发现这类攻击：攻击者改网页，使智能体执行目标动作，而不是用户任务。作者称这是首个此类综合基准。
Paper analysisRead analysis
AttacksarXiv 2406.12814Jun 19, 2024
论文提出 ARE 框架，在 VisualWebArena 上评估多模态 Agent 的对抗鲁棒性
提出 ARE 框架与图像文本攻击，评测多模态网页 Agent 的对抗鲁棒性
arXiv
2406.12814
Published
Jun 19, 2024
Layer
Application layer
Setting
web agent
Evaluated models
GPT-4o, GPT-4V, Claude-3-Opus and others4
- AttacksPrompt injection
- TechniqueGradient and representation optimization
- ComponentVision
Abstract系统评估多模态网页智能体的脆弱性，指出推理时计算组件被攻破会加剧系统风险。
Paper analysisRead analysis

You have reached the end

AI Safety Daily

Read the latest issueOne issue per day, updated as events develop

All papers

Attack type

Attack technique

Defense type

Impact

System type

Attack surface / entry point

Layer

System component

Cancerous backdoor attack: progressive activation via invisible tokens and a stage controller.

Researchers propose answer-side backdoors: letting the model self-generate trigger words to bypass safety refusals.

Purdue 提出 Pact：将欺骗作为行为遗忘

Tracekit：面向自主编码 Agent 的防篡改意图-推理-动作审计系统

特拉维夫大学提出爆炸性提示词：条件触发的间接提示注入可绕过九款生产 Agent

Counter-GEO-Bench：现有护栏难挡 GEO 虚假信息

Mind2Web-Injection：面向网页 Agent 视觉提示注入护栏的证据对齐评测

GEO-Flag：检测与测量面向生成式引擎优化的网页内容

Researchers disclose a reasoning-amplified denial-of-service attack against LLM agent guardrails

研究揭示 LLM 按 token 计费可被服务商系统性虚报

From prompts to physical actions: structured backdoor attacks against LLM-mediated robot control systems

BrowseSafe：研究者构建浏览器 Agent 提示注入基准并评测现有防御

论文提出多轮在线骚扰 Agent 基准

研究者提出用 10 条良性问答对微调越狱 LLM 的两阶段攻击

WAInjectBench 系统评测网页 Agent 的提示注入检测方法

论文提出 ARE 框架，在 VisualWebArena 上评估多模态 Agent 的对抗鲁棒性