Skip to content

Translation in progress Some entries still include text in the original language.

Oct 12 · Monday

All papers

Found 60 papers
Filter3

Attack type

Prompt injection18Poisoning and backdoors29Model tampering3Extraction and theft7Detection evasion4

Attack technique

Gradient and representation optimization10Automated search9Cross-modal payloads4Side channels4Defense-targeted9Dormant triggers19

Defense type

Detection and filtering3

Impact

Harmful content8Unauthorized actions10Data and privacy leaks10Output manipulation25Availability and reliability6

System type

Models and APIs30LLM applications6AI Agent25web agent9Embodied systems and robotics3

Attacker access

Black-box5Gray-box3

Attack surface / entry point

User input206Web pages and interfaces21Documents and messages30Tool responses35Tool and skill descriptions34Code repositories and dependencies18Retrieval corpus22Memory writes16Inter-agent messages10Images, audio and video78Training data27Weights and gradients47Distribution channels40Inference services and hardware13

Layer

Model layer6Application layer23Training and data layer23Infrastructure layer8

System component

Memory3Guardrails and judges5Reasoning chains4Fine-tuning and open weights15Vision5Multimodal8
Select any value within a category; different categories must all match. Counts include the other filters you have selected.Another 254papers have no tags yet and are not included in filtered results.
Attack surface / entry point：Web pages and interfacesAttack surface / entry point：Training dataAttack surface / entry point：Inference services and hardwareClear all

Another 254papers do not have category tags yet and are not included in filtered results.

  1. AttacksarXiv 2610.12644

    Cancerous backdoor attack: progressive activation via invisible tokens and a stage controller.

    Proposes the Cancerous backdoor, which progressively corrupts reasoning using invisible stage tokens.

    arXiv
    2610.12644
    Published
    Layer
    Training and data layer
    Setting
    Models and APIs
    Evaluated models
    Llama-3.1-8B-Instruct, Qwen2.5-7B-Instruct, Mistral-7B-Instruct-v0.3
    AbstractHidden tokens plus a stage controller make the backdoor progressively corrupt reasoning upon triggering, and the authors believe that looking only at the answer will miss it.

    Cancerous Backdoor Attack changes LLM backdoors from one-time activation to multi-stage reasoning corruption that accumulates progressively with triggers.

    Paper analysisRead analysis
  2. AttacksarXiv 2610.11932

    A Fudan team measures the citation fragility of AI search: ordinary posts can enter citations and answer text

    Measures citation fragility in AI search, demonstrating that ordinary public posts can enter citations and answers.

    arXiv
    2610.11932
    Published
    Layer
    Application layer
    Setting
    LLM applications
    Evaluated models
    Grok, Doubao, ChatGPT and others10

    AbstractThe authors claim the paper quantifies vulnerabilities in AI search caused by citation preferences and low posting thresholds; tests show public web posts can penetrate AI citations, and they call for auditing the citation supply chain.

    Paper analysisRead analysis
  3. AttacksarXiv 2610.12524

    An exploration of attacks on AI agent systems

    重测并扩展对 AI Agent 的提示注入，并用 Agent Card 欺骗操纵 A2A 选择

    arXiv
    2610.12524
    Published
    Layer
    Application layer
    Setting
    AI Agent
    Evaluated models
    Ollama Mistral 0.9.5, llama3-8b-8192, gemini-2.0-flash-lite and others5
    Abstract单智能体过滤与分类可被格式或措辞绕过，A2A 选卡还会偏向名称和描述。

    作者在自建的 LangChain、AutoGen 和 A2A 系统上重测提示注入，并改 Agent Card 看选择是否偏移。摘要称试验主要凸显单智能体弱点，同时把攻击扩到多智能体。

    Paper analysisRead analysis
  5. AttacksarXiv 2610.09981

    研究：LLM 路由决策日志可推断用户医疗、自伤等敏感话题

    提出利用路由元数据推断敏感话题的侧信道攻击

    arXiv
    2610.09981
    Published
    Layer
    Infrastructure layer
    Setting
    LLM applications
    Evaluated models
    RouteLLM (BERT router), notdiamond-0001, Vela (vLLM Semantic Router)

    Abstract论文揭示并测量了 LLM 路由决策在无文本日志中的敏感话题泄露，验证了按类别公平性防御的有效性与边界。

    Paper analysisRead analysis
  6. AttacksarXiv 2610.09240

    WebMirage hijacks visual web agents' browser operations with adversarial images

    提出 WebMirage，用局部对抗图像劫持视觉网页 Agent 的浏览器操作

    arXiv
    2610.09240
    Published
    Layer
    Application layer
    Setting
    web agent
    Evaluated models
    LLaVA-v1.5-13B, LLaVA-v1.6-34B, MiniCPM-o-8B and others6
    Abstract论文提出了针对网络智能体的端到端视觉红队框架 WebMirage，揭示了从视觉定位到浏览器执行的安全脆弱性。

    WebMirage 是一个面向视觉定位网络智能体的流水线感知端到端红队攻击框架。

    Paper analysisRead analysis
  7. AttacksarXiv 2610.08951

    ASPIRE：面向 LLM Agent 的开放式提示注入红队引擎

    提出 ASPIRE 行为图搜索，开放式发现 Agent 提示注入漏洞

    arXiv
    2610.08951
    Published
    Layer
    Application layer
    Setting
    AI Agent
    Evaluated models
    Gemini-3.7-Flash, DeepSeek-V4-Flash, Gemini-2.5-flash
    AbstractASPIRE 把提示注入红队从载荷优化改成行为图引导的开放式漏洞发现。

    ASPIRE 是给搭建方用的智能体提示注入红队引擎，目标是开放式画出可利用的行为路径，而不是在固定任务上打磨一条载荷。

    Paper analysisRead analysis
  8. AttacksarXiv 2610.07723

    Researchers propose answer-side backdoors: letting the model self-generate trigger words to bypass safety refusals.

    提出回答端后门，让模型自生成触发词以在干净有害请求下绕过拒答

    arXiv
    2610.07723
    Published
    Layer
    Training and data layer
    Setting
    Models and APIs
    Evaluated models
    Llama-2-7B-chat-hf, Mistral-7B-Instruct-v0.3, Gemma-7B-it and others4

    Abstract论文提出了多轮对话中回答端自生成触发词绕过安全拒绝的后门攻击，指出输入端防护存在盲区。

    Paper analysisRead analysis
  9. AttacksarXiv 2610.07654

    研究揭示策略蒸馏的安全后门风险

    测量带后门教师经 OPD 向干净学生迁移潜伏后门

    arXiv
    2610.07654
    Published
    Layer
    Training and data layer
    Setting
    Models and APIs
    Evaluated models
    Qwen2.5-1.5B-Instruct, Qwen2.5-3B-Instruct, Qwen2.5-7B-Instruct
    Abstract作者称 OPD 可在压低无触发词有害率时迁入后门。

    带后门的外部教师经 OPD，可在压低无触发词有害率的同时，把触发条件下的有害行为传给初始干净学生。。

    Paper analysisRead analysis
  10. AttacksarXiv 2610.06848

    TranScope：利用 CPU 性能计数器对 Transformer 做成员推断攻击

    提出 TRANSCOPE，用 CPU 硬件计数器推断训练集成员

    arXiv
    2610.06848
    Published
    Layer
    Infrastructure layer
    Setting
    Models and APIs
    Attacker
    Black-box
    Evaluated models
    BERT (4.4M-340M), ViT (22M-307M), Pythia (70M-2.8B) and others5

    AbstractTRANSCOPE 揭示了分词器在嵌入表引发的硬件访问局部性差异，证实微架构计数器可突破恒定时间与置信度遮蔽防御。

    Paper analysisRead analysis
  11. AttacksarXiv 2610.05894

    研究者提出闭环偏见注入攻击

    提出闭环激活引导，在扩散语言模型去噪过程中注入定向偏见

    arXiv
    2610.05894
    Published
    Layer
    Model layer
    Setting
    Models and APIs
    Attacker
    Gray-box
    Evaluated models
    LLaDA-8B-Instruct, Dream-v0-Instruct-7B, LLaDA-MoE-7B-A1B-Instruct

    Abstract通过去噪中间概率反馈动态调节残差流转向，证明扩散语言模型在推理服务栈中存在闭环偏见注入脆弱性。

    Paper analysisRead analysis
  12. AttacksarXiv 2610.05793

    研究者训练出按多智能体拓扑触发的代码后门模型

    提出拓扑条件后门，使代码模型在推断多智能体部署时植入漏洞

    arXiv
    2610.05793
    Published
    Layer
    Training and data layer
    Setting
    coding agent
    Evaluated models
    Qwen2.5-7B-Instruct, Qwen2.5-Coder-7B-Instruct

    Abstract论文证实了拓扑条件后门的可训练性与泛化性，展示了常规评测盲区，并提出差异化拓扑审计作为检测防御方向。

    Paper analysisRead analysis
  14. AttacksarXiv 2610.01367

    研究：高质量数据筛选挡不住投毒

    提出 Bi-QSTO，优化能穿过质量筛选并在微调中削弱安全对齐的投毒样本

    arXiv
    2610.01367
    Published
    Layer
    Training and data layer
    Setting
    Models and APIs
    Evaluated models
    Llama2-7B-Chat, Llama3-8B-Instruct, Qwen2-7B-Instruct and others5
    Abstract质量筛选挡不住部分高分样本的安全削弱。

    Bi-QSTO 研究一个实际训练缺口：基于质量的数据选择会先于微调决定样本去留，而以往微调投毒多假设毒样本直接进入训练集。已对齐 LLM 的拒答会被少量有害或看似无害的微调样本削弱。作者用三个问题串起实验：选择能否排除毒样本。

    Paper analysisRead analysis
  15. AttacksarXiv 2610.01365

    ReGap：无需真实隐私数据即可通过微调重新提取模型隐私关联

    提出 ReGap，用自生成伪监督微调恢复模型中的隐私关联

    arXiv
    2610.01365
    Published
    Layer
    Training and data layer
    Setting
    Models and APIs
    Attacker
    White-box, Gray-box
    Evaluated models
    GPT-2 Small, GPT-2 Medium, GPT-2 Large and others6

    Abstract提出无真实私有监督的恢复攻击 ReGap，实验显示微调可通过自生成数据唤醒模型潜在隐私关联，突显后适应隐私风险。

    Paper analysisRead analysis
  16. AttacksarXiv 2609.39788

    研究：多智能体系统潜在通信链路可被训练用于提升有害顺从

    提出只训练或篡改多智能体潜在通信链路来提高有害遵从

    arXiv
    2609.39788
    Published
    Layer
    Application layer
    Setting
    Multi-agent systems
    Evaluated models
    Llama-3.2-3B-Instruct, Qwen2.5-3B-Instruct, Gemma-3-1B-IT and others7
    Abstract潜在链路本身即可改变冻结智能体系统的安全，奖励既能攻击也能修复。

    学习得到的潜在通信链路会改变由冻结、对齐智能体组成的系统是否遵从有害请求。。作者研究三种拓扑：双智能体、三级顺序协作，以及双专家加汇总。

    Paper analysisRead analysis
  18. AttacksarXiv 2609.33985

    研究：众包微调数据投毒可放大专有指令抽取

    提出主题锚点投毒，放大众包微调后的专有指令提取

    arXiv
    2609.33985
    Published
    Layer
    Training and data layer
    Setting
    Models and APIs
    Attacker
    Black-box
    Evaluated models
    Qwen2.5-7B-Instruct, Qwen2.5-14B-Instruct, Llama-3.1-8B-Instruct and others4

    Abstract论文提出基于主题锚点的数据中毒攻击，证明低比例的中毒样本可在黑盒条件下隐蔽放大微调指令泄漏。

    Paper analysisRead analysis
  19. AttacksarXiv 2609.33628

    用课程强化学习对前沿模型做提示注入红队测试

    用跨目标课程强化学习训练提示注入红队，攻破前沿智能体

    arXiv
    2609.33628
    Published
    Layer
    Application layer
    Setting
    AI Agent
    Evaluated models
    GPT-4o-mini, GPT-5-nano, GPT-5.6-Luna and others12
    Abstract提出跨目标模型课程强化学习，攻破前沿模型冷启动瓶颈并实现跨模型迁移。

    该研究针对强化学习自动化提示注入红队在攻击前沿大模型时的冷启动问题，提出了一种基于目标模型鲁棒性递进的课程强化学习方法。

    Paper analysisRead analysis
  20. AttacksarXiv 2609.33569

    信息黑洞：3D 点云重建中的后门机制研究

    提出 Information Blackhole 与 AGM，控制点云自编码器后门重建到指定形状

    arXiv
    2609.33569
    Published
    Layer
    Training and data layer
    Setting
    Models and APIs
    Evaluated models
    FoldingNet, PointDiffusion
    AbstractAGM 用自适应高斯匹配压制 PCAE 投毒潜码中的源几何，九组设置的 CD-ASD 均下降。

    这篇工作研究点云自编码器的训练时后门：干净输入仍自重建，带触发的输入应重建成攻击者指定形状。分类后门只需把预测推向离散标签。PCAE 把源几何与目标信息编进同一个连续潜空间，残留源几何会把重建拉回源形状。

    Paper analysisRead analysis
Load more