Skip to content

Translation in progress Some entries still include text in the original language.

Oct 13 · Tuesday

All papers

Found 23 papers
Filter6

Attack type

Jailbreaks6Prompt injection11Poisoning and backdoors5Detection evasion5

Attack technique

Gradient and representation optimization3Reasoning-chain manipulation3Cross-modal payloads4

Defense type

Detection and filtering4Monitoring and auditing6

Impact

Harmful content5Unauthorized actions5Output manipulation7

System type

Models and APIs10LLM applications3AI Agent10coding agent3

Attacker access

Black-box5White-box3

Attack surface / entry point

User input51Web pages and interfaces7Documents and messages12Tool responses17Tool and skill descriptions7Code repositories and dependencies3Retrieval corpus9Memory writes3Images, audio and video8Training data4Weights and gradients4Distribution channels5Inference services and hardware3

Layer

Prompt layer3Model layer3Application layer13Training and data layer3

System component

MCP and skills9Memory15RAG5Guardrails and judges32Monitors9Reasoning chains13Permissions and sandboxing11Fine-tuning and open weights39Vision89Audio21Video8Image generation16Embeddings3Multimodal127
Select any value within a category; different categories must all match. Counts include the other filters you have selected.Another 213papers have no tags yet and are not included in filtered results.
Attack surface / entry point：Documents and messagesAttack surface / entry point：Images, audio and videoAttack surface / entry point：Training dataSystem component：MonitorsSystem component：Reasoning chainsSystem component：EmbeddingsClear all

Another 213papers do not have category tags yet and are not included in filtered results.

  1. AttacksarXiv 2610.12644

    Cancerous backdoor attack: progressive activation via invisible tokens and a stage controller.

    Proposes the Cancerous backdoor, which progressively corrupts reasoning using invisible stage tokens.

    arXiv
    2610.12644
    Published
    Layer
    Training and data layer
    Setting
    Models and APIs
    Evaluated models
    Llama-3.1-8B-Instruct, Qwen2.5-7B-Instruct, Mistral-7B-Instruct-v0.3
    AbstractHidden tokens plus a stage controller make the backdoor progressively corrupt reasoning upon triggering, and the authors believe that looking only at the answer will miss it.

    Cancerous Backdoor Attack changes LLM backdoors from one-time activation to multi-stage reasoning corruption that accumulates progressively with triggers.

    Paper analysisRead analysis
  2. DefensesarXiv 2610.09793

    用时序逻辑监控工具 Agent

    提出 MFOTL 离线时序验证以监控工具 Agent 的危险动作链

    arXiv
    2610.09793
    Published
    Layer
    Application layer
    Setting
    AI Agent
    Attacker
    Black-box
    Abstract形式化验证可有效识别多步攻击，但其有效性高度依赖轨迹的可信历史记录。

    这篇论文对工具调用大语言模型智能体的安全监控进行了形式化时序验证研究。

    Paper analysisRead analysis
  3. DefensesarXiv 2610.05163

    Researchers construct segmented prompt injection on Claude Code and Codex and release a boundary action audit benchmark

    提出 PAA 方法，在边界动作前审计长流程智能体的分阶段提示注入

    arXiv
    2610.05163
    Published
    Layer
    Application layer
    Setting
    coding agent
    Evaluated models
    Claude Code (v2.1.172, Claude Sonnet 5), Codex CLI (v0.144.0, GPT-5.5)
    Abstract论文针对长流程注入提出边界动作审计与 PAA，在保持低误阻的同时实现高拦截率。

    本研究针对长流程智能体在执行任务时面临的分阶段间接提示注入威胁，提出了在动作生效前进行拦截的边界动作审计框架与路径对齐归因方法。

    Paper analysisRead analysis
  4. DefensesarXiv 2609.38909

    Purdue 提出 Pact：将欺骗作为行为遗忘

    提出 PACT，遗忘推理模型在压力下的欺骗行为并保留上下文理解

    arXiv
    2609.38909
    Published
    Layer
    Training and data layer
    Setting
    Models and APIs
    Attacker
    White-box
    Evaluated models
    DeepSeek-R1-Distill-Qwen-32B, QwQ-32B

    Abstract论文将大模型欺骗形式化为上下文条件行为，提出 PACT 框架在消除欺骗的同时保留了上下文遵循能力。

    Paper analysisRead analysis
  5. DefensesarXiv 2609.36562

    ThinkingGuard：通过逐步风险归因解码多模态大语言模型中的隐式危害

    构建 TriggerBench 并训练 ThinkingGuard，检测图文隐式风险

    arXiv
    2609.36562
    Published
    Layer
    Prompt layer
    Setting
    Models and APIs
    Evaluated models
    ThinkingGuard（Qwen3-VL-8B-Instruct）
    AbstractTriggerBench 隔离触发条件，ThinkingGuard 用分步归因在隐式风险检测上取得作者报告的较高平均分。

    这篇工作研究多模态隐式风险检测：图文各自良性，风险只在关键元素与触发元素交互时出现。

    Paper analysisRead analysis
  6. Evaluation and benchmarksarXiv 2609.28915

    研究提出 ACE 语料库，评估内核级证据对 Agent 安全检测的有效性

    构建 ACE 语料，评测内核级证据对 Agent 攻击检测的有效性

    arXiv
    2609.28915
    Published
    Layer
    Application layer
    Setting
    coding agent
    Evaluated models
    AdaBoost, XGBoost, TabPFN and others10
    AbstractACE 用配对的内核与应用层证据表明，syscall 痕迹可判别智能体攻击，两层组合通常更好。

    ACE 是一份把智能体会话的内核 syscall 痕迹与应用层清单、转录配对起来的检测语料和基准。。

    Paper analysisRead analysis
  7. AttacksarXiv 2609.27542

    Researchers use control token injection to delete gpt-oss-20b's reasoning chain and bypass chain-of-thought monitoring

    提出控制标记注入，清空工具 Agent 的思维链以绕过推理监控

    arXiv
    2609.27542
    Published
    Layer
    Application layer
    Setting
    AI Agent
    Evaluated models
    openai/gpt-oss-20b, google/gemma-4-26B-A4B-it, deepseek-ai/DeepSeek-R1-Distill-Qwen-7B and others4

    Abstract论文阐明控制标记注入可抑制 CoT 使监控失效并促成拒绝绕过，且工具执行受解析器宽容度影响。

    Paper analysisRead analysis
  8. DefensesarXiv 2608.12876

    SPARED：用对抗编辑数据训练基于推理的 AI 生成图像检测器

    提出 SPARED，用对抗编辑数据训练基于推理的 AI 生成图像检测器

    arXiv
    2608.12876
    Published
    Layer
    Training and data layer
    Setting
    Models and APIs
    Attacker
    Black-box, White-box
    Evaluated models
    Qwen3.5-9B
    AbstractSPARED 用防捷径的编辑对抗环训练 9B 推理检测器。

    SPARED 是一套用于可解释 AI 生成图像检测的攻防强化学习流程，报告模型是 Qwen3.5-9B 上的第三轮防御器。

    Paper analysisRead analysis
  10. DefensesarXiv 2607.14737

    GeoDetect：面向视觉语言预训练模型的几何对抗检测方法

    提出 GeoDetect 以几何分数检测视觉语言预训练模型的对抗样本

    arXiv
    2607.14737
    Published
    Layer
    Model layer
    Setting
    Models and APIs
    Attacker
    White-box
    Evaluated models
    CLIPViT (ViT-B/16), CLIPCNN (ResNet-50), ALBEF and others4
    AbstractGeoDetect 用几何分数检测 VLP 对抗样本。

    GeoDetect 是面向视觉-语言预训练模型的对抗样本检测方法，用嵌入几何而不是任务 logits 区分干净样本与对抗样本。

    Paper analysisRead analysis
  12. AttacksarXiv 2606.12716

    PaperGuard：针对多模态 AI 同行评审的攻击与防御基准

    提出 PaperGuard，用图文注入与梯度扰动抬高自动审稿分数并检测恶意指令

    arXiv
    2606.12716
    Published
    Layer
    Application layer
    Setting
    LLM applications
    Evaluated models
    Claude-sonnet-4.5, GPT-4o, GPT-4.1 and others11
    AbstractPaperGuard 测到提示注入与白盒视觉抬分，并比较分块检索防御。

    PaperGuard同时评测并检测多模态 AI 审稿中的评分操纵。

    Paper analysisRead analysis
  13. AttacksarXiv 2605.30040

    研究揭示 LLM 按 token 计费可被服务商系统性虚报

    构造可通过计费审计的 token 虚报，使服务商系统性超收

    arXiv
    2605.30040
    Published
    Layer
    Infrastructure layer
    Setting
    Models and APIs
    Evaluated models
    PALACE domain-adapted auditor
    Abstract不诚实提供方可在三种 token 审计上虚报并通过检查，作者认为需改用提供方不控制的证据。

    不透明 LLM 的按 token 计费里，现有审计核验的是提供方可控的证据，不诚实提供方可以虚报并通过检查。。

    Paper analysisRead analysis
  14. DefensesarXiv 2605.17380

    Uber proposes ADR enterprise-level agent detection system, deployed on 7,200 hosts for ten months

    提出企业级检测系统 ADR，识别编程 Agent 的提示注入与恶意 MCP

    arXiv
    2605.17380
    Published
    Layer
    Application layer
    Setting
    coding agent

    Abstract提出了端点因果遥测、两级在线分流与离线红队闭环的 ADR 系统，并在生产环境中验证了有效性。

    Paper analysisRead analysis
  15. AttacksarXiv 2603.09246

    研究者提出 Reasoning-Oriented Programming，通过链式语义片段越狱大型视觉语言模型

    提出 VROP，把有害意图拆成良性视觉 gadget 越狱视觉语言模型

    arXiv
    2603.09246
    Published
    Layer
    Prompt layer
    Setting
    Models and APIs
    Attacker
    Black-box
    Evaluated models
    GPT-4o, Claude 3.7 Sonnet, GLM-4V-Plus and others7
    AbstractVROP 以良性视觉 gadget 与控制流提示词做组合越狱。

    VROP是 black-box 多模态越狱框架：不在输入中放显式恶意触发，而把有害目标拆成语义正交的良性视觉 gadget，再用控制流提示词把合成留到后期推理。

    Paper analysisRead analysis
  16. DefensesarXiv 2602.20708

    ICON：基于推理时纠正的 Agent 间接提示注入防御

    Proposes inference-time defense ICON, which detects attention over-focusing and redirects to correct indirect prompt injection in agents.

    arXiv
    2602.20708
    Published
    Layer
    Application layer
    Setting
    AI Agent
    Evaluated models
    Qwen-3-8B, Llama-3.1-8B, Mistral-8B and others6
    AbstractICON detects over-focusing and redirects at inference time to balance IPI safety and task utility.

    ICON detects attention over-focusing from IPI at inference time and redirects, avoiding refusal that would terminate the agent trajectory.

    Paper analysisRead analysis
  17. AttacksarXiv 2602.18514

    招聘中的特洛伊木马：标准模型与推理模型间接提示注入的红队测试案例研究

    用简历中的不可见指令做间接提示注入，操纵招聘推荐

    arXiv
    2602.18514
    Published
    Layer
    Application layer
    Setting
    LLM applications
    Evaluated models
    Qwen 3 30B A3B Instruct 2507, Qwen 3 30B A3B Thinking 2507

    Abstract作者称推理在简单注入下会帮攻击圆谎，在不合逻辑的复杂注入下又会把欺骗漏出来。

    Paper analysisRead analysis
  18. Evaluation and benchmarksarXiv 2602.14161

    When benchmarks lie: evaluating malicious prompt classifiers under real-world distribution shift

    提出 LODO 评测恶意提示探针的跨数据集泛化

    arXiv
    2602.14161
    Published
    Layer
    Model layer
    Setting
    Models and APIs
    Evaluated models
    Llama-3.1-8B-Instruct, Gemma-3-27B, Qwen-3.5-4B and others5
    Abstract作者称 LODO 露出 CV 高估，数据集身份与安全方向部分重叠，标准域泛化干预关不上。

    LODO 用来检查恶意提示词分类器换成没见过的数据集后，还剩多少检测能力。。

    Paper analysisRead analysis
  19. DefensesarXiv 2602.11096

    Safety Recovery of Reasoning Models Only Needs a Few Early Steps of Guidance

    Proposes SafeThink, an inference-time defense that monitors the chain of thought and injects a redirect prefix when it falls below a threshold.

    arXiv
    2602.11096
    Published
    Layer
    Model layer
    Setting
    Models and APIs
    Evaluated models
    R1-Onevision-7B, OpenVLThinker-7B, VLAA-Thinker-7B and others6
    Paper analysisRead analysis
  20. AttacksarXiv 2602.00364

    "Someone hid it": Query-agnostic black-box attacks against LLM retrieval

    提出查询无关黑盒攻击 DQ-A，用可迁移文末后缀降低文档检索排名

    arXiv
    2602.00364
    Published
    Layer
    Application layer
    Setting
    LLM applications
    Evaluated models
    Qwen-1.5-7B-Instruct, SFR-Embedding-Mistral, E5-Mistral-7B-Instruct and others10
    AbstractDQ-A 以无查询黑盒后缀压低 LLM 检索排名，并在 BRIGHT 上比较三种基线。

    DQ-A 是一种 query-agnostic、complete black-box 的文档隐藏攻击，对象是用最后隐层相似度匹配的 LLM 检索。

    Paper analysisRead analysis
Load more