Select any value within a category; different categories must all match. Counts include the other filters you have selected.Another 233papers have no tags yet and are not included in filtered results.

Attack type：Jailbreaks Attack type：Extraction and theft System component：Permissions and sandboxing System component：Audio System component：Embeddings Clear all

Another 233papers do not have category tags yet and are not included in filtered results.

AttacksarXiv 2610.09920Oct 7
研究：多向量视觉文档索引可被反演还原页面内容
提出流匹配逆向框架，从多向量视觉文档索引重构页面
arXiv
2610.09920
Published
Oct 7
Layer
Application layer
Setting
LLM applications
Evaluated models
Tomoro-ColQwen3-8B, ColQwen3.5-4.5B, Tomoro-colqwen3-embed-4b and others13
- AttacksExtraction and theft
- ComponentEmbeddings
Paper analysisRead analysis
Analysis and theoryarXiv 2610.07005Oct 4
研究审计 Qwen2-Audio 音频越狱 AdvWave-P 的频率与解码层归因
遮挡审计 AdvWave-P 音频越狱的频带必要性与解码层关联
arXiv
2610.07005
Published
Oct 4
Layer
Prompt layer
Setting
Models and APIs
Evaluated models
Qwen2-Audio-7B-Instruct, LLaMA-Omni
- AttacksJailbreaks
- ComponentAudio
AbstractAdvWave-P 的八频带排序与能量相关。
这是对加性音频越狱 AdvWave-P 的频率与解码器深度审计，受害模型主要是 Qwen2-Audio-7B-Instruct，不提出新攻击。
Paper analysisRead analysis
DefensesarXiv 2609.34518Sep 28
SEAD 论文提出工具型 Agent 的状态视角攻击与防御方法 DART 与 SAGE
提出工具型 Agent 的自适应攻击 DART 与预执行防御 SAGE
arXiv
2609.34518
Published
Sep 28
Layer
Application layer
Setting
AI Agent
Evaluated models
GPT-5.6 Luna, GPT-5.6 Terra, Gemini 3.8 Flash and others5
- DefensesPermissions and approval
- AttacksJailbreaks
- ComponentPermissions and sandboxing
Abstract论文提出状态控制框架 SEAD、攻击 DART 与防御 SAGE，实验显示执行反馈与环境探测在攻防交互中具有关键作用。
Paper analysisRead analysis
Evaluation and benchmarksarXiv 2609.34092Sep 28
机器遗忘在 ASR 中是否有效：arXiv 论文评估现有算法与评测工具
arXiv
2609.34092
Published
Sep 28
Layer
Training and data layer
Setting
Models and APIs
Evaluated models
E-Branchformer
- DefensesModel hardening
- AttacksExtraction and theft
- ComponentAudio
Abstract梯度上升类方法更接近简单评测下的重训参照，复杂方法会被知情攻击认出，多说话人更差。
Paper analysisRead analysis
DefensesarXiv 2609.29287Sep 24
AEGIS：用内部信号在中间层拦截大型音频语言模型越狱
提出 AEGIS，用中层内部信号选择性激活后层适配器拦截音频越狱
arXiv
2609.29287
Published
Sep 24
Layer
Model layer
Setting
Models and APIs
Attacker
White-box
Evaluated models
Qwen2-Audio-7B, VITA-1.5, Ultravox v0.5 and others6
- DefensesInference-time intervention
- AttacksJailbreaks
- ComponentAudio
Paper analysisRead analysis
DefensesarXiv 2609.28693Sep 24
skilder：以角色化技能包为工具型 LLM 智能体做访问控制
提出 Skilder，按角色渐进下发企业工具并确定性拦截越权调用
arXiv
2609.28693
Published
Sep 24
Layer
Application layer
Setting
AI Agent
Evaluated models
Claude Haiku 4.5, Qwen 3.5 122B, Gemma 4 31B and others6
- DefensesPermissions and approval
- AttacksJailbreaks
- ComponentMCP and skills
AbstractSkilder 用单 MCP 服务器按角色交付工具并硬拦截。
Skilder 把企业工具型智能体的能力按角色打包，经一个 MCP 服务器做渐进发现，使工具范围由服务器而不是提示词来执行。
Paper analysisRead analysis
DefensesarXiv 2609.27378Sep 23
PALS：面向全双工语音对话模型对抗鲁棒性的心理声学对齐潜空间平滑
提出 PALS，用潜空间平滑防御全双工语音对话的对抗扰动
arXiv
2609.27378
Published
Sep 23
Layer
Model layer
Setting
Models and APIs
Attacker
White-box
Evaluated models
Moshi-style agent
- DefensesModel hardening
- AttacksJailbreaks
- ComponentAudio
Paper analysisRead analysis
Evaluation and benchmarksarXiv 2609.22076Sep 19
APort Vault released: evaluating AI agent payment authorization with 4,371 human-written attacks
构建 APort Vault 基准，对比支付智能体模型拒答与确定性授权
arXiv
2609.22076
Published
Sep 19
Layer
Application layer
Setting
AI Agent
Evaluated models
Claude Fable 5.1, Claude Sonnet 5, Claude Haiku 4.5 and others14
- DefensesPermissions and approval
- AttacksJailbreaks
- ComponentPermissions and sandboxing
Paper analysisRead analysis
DefensesarXiv 2609.14003Sep 12
FlowSeal：用信息流控制阻断 LLM Agent 的隐私泄露
提出 FLOWSEAL，用工具级信息流控制阻断智能体隐私泄露
arXiv
2609.14003
Published
Sep 12
Layer
Application layer
Setting
AI Agent
Evaluated models
DeepSeek-V3.2, GPT-4.1-mini, Gemini-Flash-2.5
- DefensesInstruction–data separation
- AttacksJailbreaks
- ComponentPermissions and sandboxing
Abstract作者分析大模型智能体隐私检查与攻击面重合问题，提出外部拦截防御 FLOWSEAL 并验证有效性。
Paper analysisRead analysis
AttacksarXiv 2609.06749Sep 7
PMA 攻击：针对隐私保护 LLM 中嵌入到嵌入混淆的语义流形对齐方法
提出 PMA，用流形对齐从 E2EO 混淆向量恢复明文
arXiv
2609.06749
Published
Sep 7
Layer
Model layer
Setting
Models and APIs
Evaluated models
BERT-base, RoBERTa-base, T5-base and others6
- AttacksExtraction and theft
- ComponentEmbeddings
Paper analysisRead analysis
DefensesarXiv 2609.05903Sep 5
EvoSafeHarness：为 Agent 自动演化模型与领域专属安全护栏
为冻结模型与目标域自动搜索可部署的 Agent 安全 harness
arXiv
2609.05903
Published
Sep 5
Layer
Application layer
Setting
AI Agent
Evaluated models
Sonnet 4.6, GLM-5, Kimi-K2.5 and others11
- DefensesInstruction–data separation
- AttacksPrompt injection
- ComponentPermissions and sandboxing
Paper analysisRead analysis
DefensesarXiv 2609.04767Sep 4
SHAQ: Privacy-preserving retrieval in vector databases using shadow queries
Proposes SHAQ, which replaces document embeddings with shadow query embeddings to defend against embedding inversion.
arXiv
2609.04767
Published
Sep 4
Layer
Application layer
Setting
LLM applications
Attacker
Black-box
Evaluated models
GTR-T5 encoder, DPR
- AttacksExtraction and theft
- ComponentEmbeddings
AbstractSHAQ replaces document embeddings with shadow query embeddings.
SHAQ is a defense against EIA for cloud vector databases: instead of directly modifying document embeddings, it replaces them with shadow query embeddings.
Paper analysisRead analysis
AttacksarXiv 2609.01723Sep 2
研究者提出针对微调 TTS 模型的黑盒成员推断攻击
提出针对微调 TTS 的黑盒成员推断攻击
arXiv
2609.01723
Published
Sep 2
Layer
Model layer
Setting
Models and APIs
Attacker
Black-box
Evaluated models
CosyVoice2, F5-TTS, XTTS-v2
- AttacksExtraction and theft
- ComponentAudio
Abstract黑盒 TTS 成员推断以 recitation 加分层语音表示为分数，说话人级强于记录级，DP-SGD 可压到随机附近。
作者给出一个针对微调文本转语音模型的黑盒成员推断框架，分别审计说话人是否进入微调集，以及某一条文本–音频记录是否进入微调集。微调 TTS 以合成文本和参考语音双条件生成，既有生成模型 MIA 的单条件查询和图像/文本上的逐点比较不能直接套用。作者用 scorable extent 与 memorization elicitation 比较五种查询，认为只有 recitation（用目标转写和干净目标语音要求模型复现）同时满足两条，并把它作为主查询。
Paper analysisRead analysis
DefensesarXiv 2609.01677Sep 1
Skill-as-API：面向智能体软件工程的机密多智能体协作协议
提出 Skill-as-API，隔离多智能体协作中的 skill 正文与系统提示词
arXiv
2609.01677
Published
Sep 1
Layer
Application layer
Setting
Multi-agent systems, coding agent
- DefensesInstruction–data separation
- AttacksExtraction and theft
- ComponentMCP and skills
Abstract协议把 skill 正文留在所有者侧，四层限制发现内容、权限期限和注入拼接。
Skill-as-API 是面向智能体软件工程的协调协议，用来在跨组织调用时把 skill 的代码和系统提示词留在所有者进程。作者针对三条威胁：经通道或命名空间抽取专有方法，用注入让被调用模型输出系统提示词，以及协作结束后权限继续保留。
Paper analysisRead analysis
AttacksarXiv 2608.15578Aug 16
ARENA：面向大型音频语言模型的自动化红队框架
提出 ARENA 音频红队框架，自动构造文本安全且音频接地的越狱输入
arXiv
2608.15578
Published
Aug 16
Layer
Prompt layer
Setting
Models and APIs
Attacker
Black-box
Evaluated models
Audio Flamingo 3, Qwen2-Audio-7B-Instruct, MiMo-Audio-7B-Instruct and others4
- AttacksJailbreaks
- TechniqueAutomated search
- ComponentAudio
Paper analysisRead analysis
AttacksarXiv 2607.01276Jul 1
研究者提出嵌入推断攻击 EIA，可在黑盒 IR 系统中识别所用嵌入模型
提出嵌入推断攻击 EIA，在黑盒 IR 中识别所用嵌入模型
arXiv
2607.01276
Published
Jul 1
Layer
Application layer
Setting
LLM applications
Attacker
Black-box
Evaluated models
e5-small, e5-small-v2, e5-base and others19
- AttacksExtraction and theft
- ComponentEmbeddings
AbstractEIA 用无序文档集的差异从候选中识别嵌入模型，并给出 reranker 与阈值下的计数。
EIA 是在黑盒 IR 上识别所用嵌入模型的攻击。。
Paper analysisRead analysis
Evaluation and benchmarksarXiv 2606.06037Jun 4
SpeechJBB：语码转换语音下大型音频语言模型的越狱基准
提出 SpeechJBB，评测语码转换语音下音频语言模型的越狱稳健性
arXiv
2606.06037
Published
Jun 4
Layer
Prompt layer
Setting
Models and APIs
Evaluated models
Qwen2.5-Omni-7B, Qwen3-Omni-30B-A3B-Instruct, Voxtral-Small-24B and others9
- AttacksJailbreaks
- TechniqueEncoding and obfuscation
- ComponentAudio
AbstractSpeechJBB 用五语语码转换语音和伪词探针评测九个 LALM，作者将越狱归为对齐失败而非听不懂。
SpeechJBB 用合成语音测量 LALM 在五语、语码转换和局部伪词混淆下的安全对齐，并另设口语理解对照。
Paper analysisRead analysis
AttacksarXiv 2605.20519May 20
CodecAttack：在音频编解码器潜空间构造对抗扰动，绕过有损压缩防御
提出 CodecAttack，在编解码器隐空间构造可穿透有损压缩的对抗音频
arXiv
2605.20519
Published
May 20
Layer
Prompt layer
Setting
Models and APIs
Attacker
White-box
Evaluated models
Qwen2-Audio-7B-Instruct, Audio Flamingo 3, Qwen2.5-Omni
- AttacksJailbreaks
- TechniqueGradient and representation optimization
- ComponentAudio
Paper analysisRead analysis
AttacksarXiv 2605.18168May 18
研究者提出声学干扰攻击 AIA，可通用越狱 10 个大型音频语言模型
提出声学干扰攻击 AIA，用良性音频潜语义越狱音频语言模型
arXiv
2605.18168
Published
May 18
Layer
Prompt layer
Setting
Models and APIs
Attacker
Black-box
Evaluated models
Qwen3-Omni, Qwen2.5-Omni, Qwen2-Audio and others10
- AttacksJailbreaks
- ComponentAudio
Paper analysisRead analysis
AttacksarXiv 2603.13847Mar 14
SWhisper 用近超声隐蔽声学通道对语音驱动 LLM 实现黑盒越狱
提出 SWhisper，用近超声波向语音驱动 LLM 隐蔽注入越狱提示
arXiv
2603.13847
Published
Mar 14
Layer
Application layer
Setting
LLM applications
Attacker
Black-box
Evaluated models
Meta-Llama-3.1-8B-Instruct, Gemma-3-4B, Qwen2.5-7B-Instruct and others9
- AttacksJailbreaks
- TechniqueCross-modal payloads
- ComponentAudio
Paper analysisRead analysis

AI Safety Daily

Read the latest issueOne issue per day, updated as events develop