Translation in progress Some entries still include text in the original language.

Oct 12 · Monday

All papers

Found 92 papers

Filter5

System type

Models and APIs75 AI Agent15

Attacker access

Black-box44 Gray-box3 White-box5

Attack surface / entry point

User input89 Tool responses6 Images, audio and video37

Layer

Prompt layer66 Model layer11 Application layer15

System component

Memory3 Guardrails and judges55 Monitors11 Reasoning chains15 Permissions and sandboxing5 Fine-tuning and open weights4 Vision35 Audio12 Image generation15 Multimodal61

Select any value within a category; different categories must all match. Counts include the other filters you have selected.Another 240papers have no tags yet and are not included in filtered results.

Attack type：Jailbreaks System component：Memory System component：Guardrails and judges System component：Monitors System component：Vision Clear all

Another 240papers do not have category tags yet and are not included in filtered results.

Evaluation and benchmarksarXiv 2610.11112Oct 8
EpiReal-Bench：商用图像生成模型虚假声明红队基准
构建 EpiReal-Bench，评测图像生成器将虚假主张渲染为可信证据
arXiv
2610.11112
Published
Oct 8
Layer
Prompt layer
Setting
Models and APIs
Attacker
Black-box
Evaluated models
GPT-Image-2, Nano Banana 2, Grok Imagen 2.0 and others4
- AttacksJailbreaks
- ComponentImage generation
Abstract四款商用图像模型能把假主张画成可信证据。
EpiReal-Bench 用来量商用图像生成器会不会把虚假现实主张画成可信视觉证据。
Paper analysisRead analysis
DefensesarXiv 2610.09703Oct 7
研究揭示 Token-Pruning 对 VLM 安全性的影响并提出 SAP 缓解机制
提出 Safety-Aware Pruning，缓解视觉 token 剪枝放大的多模态越狱
arXiv
2610.09703
Published
Oct 7
Layer
Model layer
Setting
Models and APIs
Attacker
Black-box
Evaluated models
LLaVA-1.5-7B
- DefensesInference-time intervention
- AttacksJailbreaks
- ComponentVision
Abstract剪枝去掉背景 token 后注意力塌向恶意前景。
这篇工作评估视觉 token 剪枝如何改变 VLM 在多模态越狱下的行为，并给出推理时防御 SAP。
Paper analysisRead analysis
DefensesarXiv 2610.09167Oct 7
Persist-3 用顺序检验识别未见越狱来源，提升 MTK 与 Prompt Guard 2 防御效果
提出 Persist-3 序贯检验，从隐藏表示识别未见越狱源
arXiv
2610.09167
Published
Oct 7
Layer
Model layer
Setting
Models and APIs
Evaluated models
Llama2-7B-chat, Qwen2.5-7B-Instruct, Gemma-2-9B-it and others4
- DefensesMonitoring and auditing
- AttacksJailbreaks
- ComponentMonitors
AbstractPersist-3 用有限窗口检验攻击流是否超出已知越狱源，检出率低于全历史 SR-DKW，但决策只依赖最近观测。
Persist-3 是一个有限记忆的序贯检测器，用来判断攻击表示流是否已经不能由任何一个已知越狱源解释。。
Paper analysisRead analysis
Evaluation and benchmarksarXiv 2610.07089Oct 5
研究者提出统一滥用监控基准，覆盖分解攻击与提示注入约 6200 条轨迹
提出危害窗口与区间指标，统一评测分解攻击和提示注入下的智能体滥用监视
arXiv
2610.07089
Published
Oct 5
Layer
Application layer
Setting
AI Agent
Evaluated models
Gemma 4 26B-A4B, Qwen3.6 27B, Qwen3.5 27B and others12
- DefensesMonitoring and auditing
- AttacksJailbreaks
- ComponentMonitors
Abstract统一智能体滥用监视框架，动作视角兼顾两类威胁但危害定位仍存瓶颈。
Paper analysisRead analysis
AttacksarXiv 2610.05943Oct 5
CRIME 框架利用良性 Agent 技能组合诱发恶意行为
提出 CRIME 框架，组合良性 Agent 技能诱发恶意行为
arXiv
2610.05943
Published
Oct 5
Layer
Application layer
Setting
AI Agent
Evaluated models
DeepSeek-V4-Flash, GPT-4o-mini, GPT-4o and others6
- AttacksJailbreaks
- TechniqueAutomated search
- ComponentMCP and skills
Abstract良性技能经 DCA 或 ACA 组合后可产生恶意环境后果，单独审查对不上。
CRIME 把公共技能库中各自通过审查的技能配成两两组合，看联合执行能否实现指定恶意任务。。
Paper analysisRead analysis
Analysis and theoryarXiv 2610.07009Oct 4
研究受控拆解图像到文本越狱中图像属性的作用
固定有害指令，单轴归因图像到文本越狱的图像属性
arXiv
2610.07009
Published
Oct 4
Layer
Prompt layer
Setting
Models and APIs
Evaluated models
GPT-4o, Claude Sonnet 4.5, Qwen3-VL-8B-Instruct and others6
- AttacksJailbreaks
- ComponentVision
Abstract固定文本的图像越狱拆解，只在 Qwen 上给出一条有界相关性归因。
Paper analysisRead analysis
DefensesarXiv 2610.03434Oct 2
Persona Guardrail：面向智能体系统的生产级防御框架
提出 Persona Guardrail，强制客服智能体守住功能边界
arXiv
2610.03434
Published
Oct 2
Layer
Application layer
Setting
AI Agent
Attacker
Black-box
Evaluated models
Qwen3-30B-A3B, Qwen3-VL-30B, Qwen3-8B and others5
- DefensesDetection and filtering
- AttacksJailbreaks
- ComponentGuardrails and judges
AbstractPersona Guardrail 用功能边界做同步输入输出校验，并在 PAGE 与生产延迟预算下给出评测。
Persona Guardrail 是客服向智能体的同步运行时防御：先写明功能边界，再决定放行或拦截，而不是只检测某一攻击类。
Paper analysisRead analysis
DefensesarXiv 2610.02853Oct 2
研究提出基于收缩约束状态空间模型的有界可达性与越狱检测方法
提出收缩约束 SSM 安全头，认证嵌入扰动并检测越狱
arXiv
2610.02853
Published
Oct 2
Layer
Prompt layer
Setting
Models and APIs
Attacker
White-box, Black-box
Evaluated models
ToySSMClassifier, S4 safety head
- DefensesDetection and filtering
- AttacksJailbreaks
- ComponentGuardrails and judges
Abstract收缩给出玩具 LTI 的有界认证。
给 SSM 安全头一个收缩条件，并在越狱检测上使用收缩正则的 S4 头。。
Paper analysisRead analysis
DefensesarXiv 2610.02413Oct 2
研究：分类后缀可提升激活探针的分布外恶意输入检测
用分类后缀提升激活探针对分布外恶意输入的检测
arXiv
2610.02413
Published
Oct 2
Layer
Model layer
Setting
Models and APIs
Evaluated models
Llama-3.1-8B, Qwen3.5-9B, Gemma-4-12B
- DefensesMonitoring and auditing
- AttacksJailbreaks
- ComponentMonitors
Abstract作者称排序增益来自分类格式，低 FPR 精度来自点名准则。
用户轮后的分类后缀，被作者用作激活探针的分布外检测干预：排序看格式，低 FPR 精度看是否点名恶意/良性。
Paper analysisRead analysis
DefensesarXiv 2610.01058Oct 1
MOMAT：面向量化 LLM 低功耗越狱防御的多图集混合框架
提出 MOMAT 多图集检索与轻量判定，防御量化模型越狱
arXiv
2610.01058
Published
Oct 1
Layer
Prompt layer
Setting
Models and APIs
Evaluated models
Llama2-13B, Llama2-7B, Mistral-7B
- DefensesDetection and filtering
- AttacksJailbreaks
- ComponentGuardrails and judges
Paper analysisRead analysis
DefensesarXiv 2610.00400Sep 30
DART: Detecting emerging safety risks in multi-turn LLM agents with representation shift
提出 DART，用去噪表征转移检测多轮智能体攻击并注入提醒
arXiv
2610.00400
Published
Sep 30
Layer
Application layer
Setting
AI Agent
Attacker
Black-box
Evaluated models
Qwen3-8B, Qwen3-14B, Qwen3-32B and others6
- DefensesMonitoring and auditing
- AttacksJailbreaks
- ComponentMonitors
Paper analysisRead analysis
AttacksarXiv 2609.38253Sep 29
CollageAttack: Exploiting cross-modal alignment defects for spatial text composition jailbreaks against T2I models.
提出 CollageAttack，用空间文本碎片重组越狱文生图模型
arXiv
2609.38253
Published
Sep 29
Layer
Prompt layer
Setting
Models and APIs
Attacker
Black-box
Evaluated models
GPT-Image-2, Doubao-Seedream-5.0-Lite, Wan2.7-Image-Pro and others5
- AttacksJailbreaks
- TechniqueEncoding and obfuscation
- ComponentImage generation
Abstract系统揭示 T2I 跨模态空间合成安全漏洞，证实离散无害元素可在图像层面重构有害意图并放大传播强度。
Paper analysisRead analysis
Evaluation and benchmarksarXiv 2609.36849Sep 29
研究评估 GradSafe 在多轮对话中的越狱检测脆弱性
评测多轮对话中 GradSafe 越狱检测的可分性与脆弱性
arXiv
2609.36849
Published
Sep 29
Layer
Prompt layer
Setting
Models and APIs
Evaluated models
Llama 3.1 8B Instruct, Qwen2.5-7B-Instruct
- DefensesDetection and filtering
- AttacksJailbreaks
- TechniqueMulti-turn escalation
- ComponentGuardrails and judges
AbstractCWS 显示不安全梯度在真实多轮流量中变弱，且随攻击族和模型而变。
作者评估梯度越狱检测从单条提示词迁到多轮对话后还剩多少可分性，方法是用 Context Window Scanner 把 GradSafe 原封不动地套到用户轮窗口上。
Paper analysisRead analysis
DefensesarXiv 2609.36603Sep 29
SED：面向 LLM 智能体的免训练持续安全策略学习框架
提出免训练框架 SED，将有害智能体轨迹蒸馏为可检索安全策略
arXiv
2609.36603
Published
Sep 29
Layer
Application layer
Setting
AI Agent
Evaluated models
DeepSeek-V4-Flash, GLM-5.2, Kimi K3
- DefensesInstruction–data separation
- AttacksPrompt injection
- ComponentMemory
Paper analysisRead analysis
DefensesarXiv 2609.36562Sep 29
ThinkingGuard：通过逐步风险归因解码多模态大语言模型中的隐式危害
构建 TriggerBench 并训练 ThinkingGuard，检测图文隐式风险
arXiv
2609.36562
Published
Sep 29
Layer
Prompt layer
Setting
Models and APIs
Evaluated models
ThinkingGuard（Qwen3-VL-8B-Instruct）
- DefensesDetection and filtering
- AttacksJailbreaks
- ComponentGuardrails and judges
AbstractTriggerBench 隔离触发条件，ThinkingGuard 用分步归因在隐式风险检测上取得作者报告的较高平均分。
这篇工作研究多模态隐式风险检测：图文各自良性，风险只在关键元素与触发元素交互时出现。
Paper analysisRead analysis
Analysis and theoryarXiv 2609.36477Sep 29
研究发现护栏模型在基座模型不确定处过度自信
诊断护栏相对基座在对抗提示上的置信失校
arXiv
2609.36477
Published
Sep 29
Layer
Model layer
Setting
Models and APIs
Evaluated models
Llama-Guard 7B, Llama-Guard-2 8B, Llama-Guard-3 8B and others10
- DefensesDetection and filtering
- AttacksJailbreaks
- ComponentGuardrails and judges
Abstract对抗包装使 guard 高置信漏检，而对应 base 在同一输入上常更不确定。
诊断性分析：五个提示词分类 guard 的置信度，在清洁输入上可以较校准，在越狱包装后则变成高置信度的错误判定。
Paper analysisRead analysis
AttacksarXiv 2609.34920Sep 28
RISE：用迭代策略演化对现代文生图模型做红队测试
提出 RISE，迭代演化可复用策略对文生图模型做红队
arXiv
2609.34920
Published
Sep 28
Layer
Prompt layer
Setting
Models and APIs
Attacker
Black-box
Evaluated models
DALL·E 3, Nano Banana 2, GPT-Image-2 and others7
- AttacksJailbreaks
- TechniqueAutomated search
- ComponentImage generation
Paper analysisRead analysis
DefensesarXiv 2609.30841Sep 25
为何扩散语言模型的越狱会成功：能量景观分析
用能量势垒信号检测扩散语言模型的越狱
arXiv
2609.30841
Published
Sep 25
Layer
Model layer
Setting
Models and APIs
Evaluated models
LLaDA-8B-Instruct, LLaDA-1.5, Dream-v0-Instruct-7B and others4
- DefensesMonitoring and auditing
- AttacksJailbreaks
- ComponentMonitors
Abstract能量势垒把 dLLM 越狱分成掩盖初始状态或中途越垒，三个 logit 信号据此互补检测。
Bach 等提出用去噪能量景观解释 masked diffusion 语言模型上的越狱，并由此得到三个免训练检测信号。。
Paper analysisRead analysis
Evaluation and benchmarksarXiv 2609.29429Sep 24
Jev 被用于零样本检测十类对齐失败，中位 AUROC 达 0.886
提出 RLCDAlignBench，零样本检测多种对齐失败
arXiv
2609.29429
Published
Sep 24
Layer
Prompt layer
Setting
Models and APIs
Evaluated models
Jev (jev-1.13.0)
- DefensesDetection and filtering
- AttacksJailbreaks
- RiskMisalignment and value divergence
- ComponentGuardrails and judges
Abstract一次通用问题已能给多数对齐失败排序，大的误差来自状态和标签。
RLCDAlignBench 测量用强化学习做校准决策的 Jev，能否当作对齐失败的零样本检测器。
Paper analysisRead analysis
Evaluation and benchmarksarXiv 2609.21793Sep 18
CASCADE 研究：跨流水线阶段的越狱防御组合评估
用 CASCADE 在统一威胁模型下筛选跨阶段越狱防御组合
arXiv
2609.21793
Published
Sep 18
Layer
Prompt layer
Setting
Models and APIs
Evaluated models
Vicuna, Llama2, Llama3 and others9
- AttacksJailbreaks
- ComponentGuardrails and judges
Paper analysisRead analysis

AI Safety Daily

Read the latest issueOne issue per day, updated as events develop

All papers

Attack type

Attack technique

Defense type

Impact

System type

Attacker access

Attack surface / entry point

Layer

System component

EpiReal-Bench：商用图像生成模型虚假声明红队基准

研究揭示 Token-Pruning 对 VLM 安全性的影响并提出 SAP 缓解机制

Persist-3 用顺序检验识别未见越狱来源，提升 MTK 与 Prompt Guard 2 防御效果

研究者提出统一滥用监控基准，覆盖分解攻击与提示注入约 6200 条轨迹

CRIME 框架利用良性 Agent 技能组合诱发恶意行为

研究受控拆解图像到文本越狱中图像属性的作用

Persona Guardrail：面向智能体系统的生产级防御框架

研究提出基于收缩约束状态空间模型的有界可达性与越狱检测方法

研究：分类后缀可提升激活探针的分布外恶意输入检测

MOMAT：面向量化 LLM 低功耗越狱防御的多图集混合框架

DART: Detecting emerging safety risks in multi-turn LLM agents with representation shift

CollageAttack: Exploiting cross-modal alignment defects for spatial text composition jailbreaks against T2I models.

研究评估 GradSafe 在多轮对话中的越狱检测脆弱性

SED：面向 LLM 智能体的免训练持续安全策略学习框架

ThinkingGuard：通过逐步风险归因解码多模态大语言模型中的隐式危害

研究发现护栏模型在基座模型不确定处过度自信

RISE：用迭代策略演化对现代文生图模型做红队测试

为何扩散语言模型的越狱会成功：能量景观分析

Jev 被用于零样本检测十类对齐失败，中位 AUROC 达 0.886

CASCADE 研究：跨流水线阶段的越狱防御组合评估