Translation in progress Some entries still include text in the original language.

Oct 12 · Monday

All papers

Found 139 papers

Filter2

System component

Guardrails and judges25 Reasoning chains13 Fine-tuning and open weights4 Vision24 Audio5 Image generation15 Multimodal43

Select any value within a category; different categories must all match. Counts include the other filters you have selected.Another 265papers have no tags yet and are not included in filtered results.

System type：Models and APIs Attack surface / entry point：User input Clear all

Another 265papers do not have category tags yet and are not included in filtered results.

AttacksarXiv 2610.12644Oct 9
Cancerous backdoor attack: progressive activation via invisible tokens and a stage controller.
Proposes the Cancerous backdoor, which progressively corrupts reasoning using invisible stage tokens.
arXiv
2610.12644
Published
Oct 9
Layer
Training and data layer
Setting
Models and APIs
Evaluated models
Llama-3.1-8B-Instruct, Qwen2.5-7B-Instruct, Mistral-7B-Instruct-v0.3
- AttacksPoisoning and backdoors
- TechniqueDormant triggers
- ComponentReasoning chains
AbstractHidden tokens plus a stage controller make the backdoor progressively corrupt reasoning upon triggering, and the authors believe that looking only at the answer will miss it.
Cancerous Backdoor Attack changes LLM backdoors from one-time activation to multi-stage reasoning corruption that accumulates progressively with triggers.
Paper analysisRead analysis
AttacksarXiv 2610.11112Oct 8
EpiReal-Bench：商用图像生成模型虚假声明红队基准
构建 EpiReal-Bench，评测图像生成器将虚假主张渲染为可信证据
arXiv
2610.11112
Published
Oct 8
Layer
Prompt layer
Setting
Models and APIs
Attacker
Black-box
Evaluated models
GPT-Image-2, Nano Banana 2, Grok Imagen 2.0 and others4
- AttacksJailbreaks
- ComponentImage generation
Abstract四款商用图像模型能把假主张画成可信证据。
EpiReal-Bench 用来量商用图像生成器会不会把虚假现实主张画成可信视觉证据。
Paper analysisRead analysis
AttacksarXiv 2610.09027Oct 7
P-VMI：对抗图像经 KV cache 持续影响多轮对话
提出持久视觉记忆注入，使对抗图像经缓存在触发时操控输出
arXiv
2610.09027
Published
Oct 7
Layer
Infrastructure layer
Setting
Models and APIs
Evaluated models
Qwen3-8B, Qwen3-VL-8B-Instruct, Gemma-4-12B
- AttacksPrompt injection
- TechniqueDormant triggers
- ComponentVision
Abstract论文提出 P-VMI，证实对抗输入被掩蔽后仍可经 KV 缓存持久控制模型，强调了保留状态的安全风险。
Paper analysisRead analysis
AttacksarXiv 2610.07723Oct 6
Researchers propose answer-side backdoors: letting the model self-generate trigger words to bypass safety refusals.
提出回答端后门，让模型自生成触发词以在干净有害请求下绕过拒答
arXiv
2610.07723
Published
Oct 6
Layer
Training and data layer
Setting
Models and APIs
Evaluated models
Llama-2-7B-chat-hf, Mistral-7B-Instruct-v0.3, Gemma-7B-it and others4
- AttacksPoisoning and backdoors
- TechniqueDormant triggers
- ComponentFine-tuning and open weights
Abstract论文提出了多轮对话中回答端自生成触发词绕过安全拒绝的后门攻击，指出输入端防护存在盲区。
Paper analysisRead analysis
AttacksarXiv 2610.06670Oct 6
研究者提出 Reward Stealing Attack，从对齐模型行为反推安全奖励实施攻击
提出 ReSA，从对齐模型行为恢复安全奖励并在解码时反转
arXiv
2610.06670
Published
Oct 6
Layer
Model layer
Setting
Models and APIs
Attacker
White-box, Gray-box
Evaluated models
Llama3.1-8B-Instruct, Gemma-7B-Instruct, Qwen2.5-7B-Instruct and others9
- AttacksJailbreaks
- TechniqueDefense-targeted
AbstractReSA 用最大熵 IRL 从对齐模型行为恢复代理安全奖励，并在解码时反转以诱导有害输出。
ReSA 是一种对抗攻击：从对齐模型的可观察行为恢复代理安全奖励，再在解码时把该奖励反过来用。。
Paper analysisRead analysis
AttacksarXiv 2610.06093Oct 5
研究评测训练数据提取风险的跨语言迁移
用翻译前缀做跨语言训练数据提取，恢复英语训练文本中的 PII
arXiv
2610.06093
Published
Oct 5
Layer
Prompt layer
Setting
Models and APIs
Evaluated models
GPT-Neo 1.3B, GPT-Neo 2.7B, GPT-J 6B and others7
- AttacksExtraction and theft
Abstract跨语言 TDE 能恢复英语记忆的 PII，转移随多语训练和措辞保真变化。
这篇工作检验训练数据抽取是否跨语言：英语段落里记住的 PII，能否被其他语言的前缀逐字抽出。
Paper analysisRead analysis
AttacksarXiv 2610.05346Oct 5
研究者提出序列马赛克攻击并给出防御理论
形式化 watchman 防御以阻止序贯马赛克攻击
arXiv
2610.05346
Published
Oct 5
Layer
Prompt layer
Setting
Models and APIs
Evaluated models
Qwen3.6-35B-A3B, Gemma-4-26B-A4B-it, DeepSeek-V2-Lite (16B-A2.4B)
- DefensesModel hardening
- AttacksJailbreaks
Abstractwatchman 理论给出零失败且有用的条件。
这篇工作同时给出序贯马赛克防御的博弈理论，以及冻结 LLM 上的分角色自博弈训练。
Paper analysisRead analysis
AttacksarXiv 2610.05282Oct 4
Red-TTT：用测试时训练实现大语言模型自动化越狱
提出 Red-TTT，在攻击过程中更新攻击者实现自动化越狱
arXiv
2610.05282
Published
Oct 4
Layer
Prompt layer
Setting
Models and APIs
Attacker
Black-box
Evaluated models
Llama-3-8B-Instruct, gpt-oss-20b, gpt-4.1-mini and others4
- AttacksJailbreaks
- TechniqueAutomated search
Paper analysisRead analysis
AttacksarXiv 2610.03430Oct 2
JIL：通过对抗后缀操纵 LLM 请求调度优先级
提出 JIL 对抗后缀，压低长度预测以抬高请求调度优先级
arXiv
2610.03430
Published
Oct 2
Layer
Infrastructure layer
Setting
Models and APIs
Evaluated models
Llama-3-8B-Instruct, Qwen2.5-7B-Instruct, Mistral-7B-Instruct-v0.3 and others4
- TechniqueGradient and representation optimization
AbstractJIL 用对抗后缀压低长度预测，在预测式 LLM 调度中提高自身优先级。
JIL 是对预测式 LLM 调度的攻击：用对抗后缀压低长度预测，使请求在 SRPT 式队列里更早得到服务。
Paper analysisRead analysis
AttacksarXiv 2610.02302Oct 2
研究者提出意图隐藏越狱的信息论框架，分析组合式攻击
提出组合式意图隐藏越狱，按先验后验匹配选择辅助任务
arXiv
2610.02302
Published
Oct 2
Layer
Prompt layer
Setting
Models and APIs
Evaluated models
Qwen3-1.7B, Qwen3-4B, Qwen3-8B and others7
- AttacksJailbreaks
- TechniqueEncoding and obfuscation
Abstract用信息论选含目标的任务组合，再看查询能否既隐藏意图又让目标被执行。
这篇工作把组合式意图隐藏写成任务选择：有害目标必须留在 bundle 里，同时让估计的有害意图概率贴近全集先验，或降到阈值以下。
Paper analysisRead analysis
AttacksarXiv 2609.39902Sep 30
CodeMimicry: Exploiting code-domain safety generalization lag for structured code completion jailbreaks.
提出 CodeMimicry，以结构化代码补全诱导模型生成有害内容
arXiv
2609.39902
Published
Sep 30
Layer
Prompt layer
Setting
Models and APIs
Attacker
Black-box
Evaluated models
GPT-4o, GPT-4.1, GPT-5-chat and others10
- AttacksJailbreaks
- TechniqueEncoding and obfuscation
Paper analysisRead analysis
AttacksarXiv 2609.38899Sep 30
SceneJail: Jailbreaking multimodal large models using video scene context
提出 SceneJail，利用视频情境上下文越狱视频多模态模型
arXiv
2609.38899
Published
Sep 30
Layer
Prompt layer
Setting
Models and APIs
Attacker
Black-box
Evaluated models
Qwen2.5-VL-7B-Instruct, Qwen2.5-VL-32B-Instruct, Qwen3-VL-8B-Instruct and others8
- AttacksJailbreaks
- TechniqueCross-modal payloads
- ComponentVideo
Paper analysisRead analysis
AttacksarXiv 2609.38253Sep 29
CollageAttack: Exploiting cross-modal alignment defects for spatial text composition jailbreaks against T2I models.
提出 CollageAttack，用空间文本碎片重组越狱文生图模型
arXiv
2609.38253
Published
Sep 29
Layer
Prompt layer
Setting
Models and APIs
Attacker
Black-box
Evaluated models
GPT-Image-2, Doubao-Seedream-5.0-Lite, Wan2.7-Image-Pro and others5
- AttacksJailbreaks
- TechniqueEncoding and obfuscation
- ComponentImage generation
Abstract系统揭示 T2I 跨模态空间合成安全漏洞，证实离散无害元素可在图像层面重构有害意图并放大传播强度。
Paper analysisRead analysis
AttacksarXiv 2609.36956Sep 29
研究者提出 Controlled Decoding Attacks，通过纯文本接口越狱黑盒 LLM
提出 BLINDBIAS，通过纯文本续写与前缀控制越狱黑盒模型
arXiv
2609.36956
Published
Sep 29
Layer
Prompt layer
Setting
Models and APIs
Attacker
Black-box
Evaluated models
GLM-5, Gemini-3.5-Flash, Qwen3-32B and others4
- AttacksJailbreaks
- TechniqueConversation structure manipulation
Abstract作者认为，隐藏数值概率并不足以消除支持重复采样和前缀续写的解码攻击面。
BLINDBIAS 是一种针对 sample-only 续写接口的 black-box 解码时越狱框架，研究没有目标数值概率时，采样能否支持逐步控制。
Paper analysisRead analysis
AttacksarXiv 2609.36941Sep 29
研究者提出针对黑盒 LLM 的实用密钥提取框架
提出用扰动蒸馏本地代理并离线过滤来提取黑盒模型凭据
arXiv
2609.36941
Published
Sep 29
Layer
Model layer
Setting
Models and APIs
Attacker
Black-box
Evaluated models
DeepSeek-Coder-7B, LLaMA-3-8B, Qwen2.5-Coder-7B and others6
- AttacksExtraction and theft
Abstract论文提出扰动蒸馏与离线过滤两阶段框架，实现了 output-only 访问下延迟降低的黑盒大模型 API key 提取。
Paper analysisRead analysis
AttacksarXiv 2609.36474Sep 29
FinRT：将自适应红队策略蒸馏为消费金融领域的可复用对抗生成器
提出 FinRT，将消费金融自适应红队轨迹蒸馏为可复用对抗提示生成器
arXiv
2609.36474
Published
Sep 29
Layer
Prompt layer
Setting
Models and APIs
Attacker
Black-box
Evaluated models
Llama-3.3-70B, Llama-3.1-8B, Mixtral-8x7B and others6
- AttacksJailbreaks
- TechniqueAutomated search
AbstractFinRT 用前向失效发现加校准代理 DPO，把消费金融红队收成单次前向生成器。
FinRT 是面向消费金融的红队框架：先发现政策失效，再训练一个按政策、领域与目标行为出提示的生成器。
Paper analysisRead analysis
AttacksarXiv 2609.36331Sep 29
无需参考模型：用邻居样本校准单轮成员推理攻击
提出用邻居分数替代参考模型的单轮成员推断
arXiv
2609.36331
Published
Sep 29
Layer
Model layer
Setting
Models and APIs
Attacker
Black-box
Evaluated models
目标分类模型（架构与训练流程遵循 Aerni et al.）
- AttacksExtraction and theft
Abstract邻居查询在单模型上恢复逐样本校准，合成邻居加早期检查点最接近 LiRA。
本文研究 one-round 成员推断：只有一个目标模型、不能再训练参考模型时，如何仍做逐样本校准，并用于单次运行的 DP 审计。
Paper analysisRead analysis
AttacksarXiv 2609.35699Sep 29
论文：蒸馏防御在蒸馏后强化学习下易被攻破
提出用 API 推理摘要重建轨迹再蒸馏加强化学习，攻破蒸馏防御
arXiv
2609.35699
Published
Sep 29
Layer
Training and data layer
Setting
Models and APIs
Attacker
Black-box
Evaluated models
Qwen2.5-0.5B, Qwen2.5-1.5B, Qwen2.5-3B and others12
- AttacksExtraction and theft
- ComponentReasoning chains
Abstract更现实的蒸馏攻击是蒸馏后再 RL。
这篇工作修订蒸馏攻击的威胁模型：攻击者经合法 API 收集闭源推理数据，先蒸馏自有模型，再做强化学习。作者认为只在蒸馏后评估防御，会漏掉后续 RL，从而高估防御。
Paper analysisRead analysis
AttacksarXiv 2609.34920Sep 28
RISE：用迭代策略演化对现代文生图模型做红队测试
提出 RISE，迭代演化可复用策略对文生图模型做红队
arXiv
2609.34920
Published
Sep 28
Layer
Prompt layer
Setting
Models and APIs
Attacker
Black-box
Evaluated models
DALL·E 3, Nano Banana 2, GPT-Image-2 and others7
- AttacksJailbreaks
- TechniqueAutomated search
- ComponentImage generation
Paper analysisRead analysis
AttacksarXiv 2609.33985Sep 28
研究：众包微调数据投毒可放大专有指令抽取
提出主题锚点投毒，放大众包微调后的专有指令提取
arXiv
2609.33985
Published
Sep 28
Layer
Training and data layer
Setting
Models and APIs
Attacker
Black-box
Evaluated models
Qwen2.5-7B-Instruct, Qwen2.5-14B-Instruct, Llama-3.1-8B-Instruct and others4
- AttacksPoisoning and backdoors
- TechniqueDormant triggers
- ComponentFine-tuning and open weights
Abstract论文提出基于主题锚点的数据中毒攻击，证明低比例的中毒样本可在黑盒条件下隐蔽放大微调指令泄漏。
Paper analysisRead analysis

AI Safety Daily

Read the latest issueOne issue per day, updated as events develop

All papers

Attack type

Attack technique

Defense type

Impact

System type

Attacker access

Attack surface / entry point

Layer

System component

Cancerous backdoor attack: progressive activation via invisible tokens and a stage controller.

EpiReal-Bench：商用图像生成模型虚假声明红队基准

P-VMI：对抗图像经 KV cache 持续影响多轮对话

Researchers propose answer-side backdoors: letting the model self-generate trigger words to bypass safety refusals.

研究者提出 Reward Stealing Attack，从对齐模型行为反推安全奖励实施攻击

研究评测训练数据提取风险的跨语言迁移

研究者提出序列马赛克攻击并给出防御理论

Red-TTT：用测试时训练实现大语言模型自动化越狱

JIL：通过对抗后缀操纵 LLM 请求调度优先级

研究者提出意图隐藏越狱的信息论框架，分析组合式攻击

CodeMimicry: Exploiting code-domain safety generalization lag for structured code completion jailbreaks.

SceneJail: Jailbreaking multimodal large models using video scene context

CollageAttack: Exploiting cross-modal alignment defects for spatial text composition jailbreaks against T2I models.

研究者提出 Controlled Decoding Attacks，通过纯文本接口越狱黑盒 LLM

研究者提出针对黑盒 LLM 的实用密钥提取框架

FinRT：将自适应红队策略蒸馏为消费金融领域的可复用对抗生成器

无需参考模型：用邻居样本校准单轮成员推理攻击

论文：蒸馏防御在蒸馏后强化学习下易被攻破

RISE：用迭代策略演化对现代文生图模型做红队测试

研究：众包微调数据投毒可放大专有指令抽取