Skip to content

Translation in progress Some entries still include text in the original language.

Oct 13 · Tuesday

All papers

Found 38 papers
Filter5

Attack type

Prompt injection9Poisoning and backdoors10Extraction and theft15

Attack technique

Defense-targeted4Dormant triggers5

Defense type

Model hardening7

Impact

Harmful content40Unauthorized actions47Data and privacy leaks27Output manipulation57Availability and reliability11Oversight failure8

System type

Models and APIs26AI Agent12web agent5

Attacker access

Black-box3White-box7

Attack surface / entry point

User input71Web pages and interfaces10Documents and messages10Tool responses11Tool and skill descriptions14Code repositories and dependencies5Retrieval corpus4Memory writes10Inter-agent messages6Images, audio and video14Training data14Weights and gradients17Distribution channels12Inference services and hardware10

Layer

Model layer5Application layer11Training and data layer19

System component

Reasoning chains3Fine-tuning and open weights8Multimodal6
Select any value within a category; different categories must all match. Counts include the other filters you have selected.Another 215papers have no tags yet and are not included in filtered results.
Impact：Data and privacy leaksImpact：Availability and reliabilityAttack surface / entry point：Web pages and interfacesAttack surface / entry point：Training dataAttack surface / entry point：Weights and gradientsClear all

Another 215papers do not have category tags yet and are not included in filtered results.

  1. AttacksarXiv 2610.12524

    An exploration of attacks on AI agent systems

    Retest and extend prompt injection against AI agents, and use Agent Card deception to manipulate A2A selection

    arXiv
    2610.12524
    Published
    Layer
    Application layer
    Setting
    AI Agent
    Evaluated models
    Ollama Mistral 0.9.5, llama3-8b-8192, gemini-2.0-flash-lite and others5
    AbstractSingle-agent filtering and classification can be bypassed by format or wording, and A2A card selection is also biased toward names and descriptions.

    The authors retested prompt injection on their own LangChain, AutoGen, and A2A systems, and modified the Agent Card to see if choices shifted. The abstract states the experiments mainly highlight single-agent weaknesses while extending attacks to multi-agent.

    Paper analysisRead analysis
  2. AttacksarXiv 2610.09819

    FAB 攻击：给 HuBERT/WavLM 植入后门，背景警笛声即可让下游语音任务失灵

    提出 FAB 攻击，向声学基础模型植入可由环境声触发的后门

    arXiv
    2610.09819
    Published
    Layer
    Training and data layer
    Setting
    Models and APIs
    Evaluated models
    HuBERT-base, WavLM-base

    Abstract论文提出了针对声学基模的 FAB 后门攻击，在弱威胁模型下实现了任务无关、物理可触发的后门植入。

    Paper analysisRead analysis
  5. AttacksarXiv 2610.07510

    PersistBD: attackers can strengthen backdoors before release so that they maintain high attack success rates even after developer SFT and RL.

    提出 PERSISTBD，发布前强化供应链后门使其经良性 SFT 与 RL 后仍外发凭据

    arXiv
    2610.07510
    Published
    Layer
    Application layer
    Setting
    coding agent
    Evaluated models
    Qwen2.5-Coder-3B-Instruct, Qwen2.5-Coder-7B-Instruct, Qwen3-Coder-30B-A3B-Instruct
    Abstract总结了论文关于后门在 SFT/RL 中存活的发现、两因子机制、PERSISTBD 方法与核心数字。

    本文研究第三方 LLM 智能体在开发者进行良性监督微调（SFT）和强化学习（RL）后训练时的后门持久性风险。

    Paper analysisRead analysis
  6. DefensesarXiv 2610.06636

    差分隐私混合公共数据集提升隐私学习效果

    提出 DP-MixMin，私有学习公开数据混合并做差分隐私微调

    arXiv
    2610.06636
    Published
    Layer
    Training and data layer
    Setting
    Models and APIs
    Evaluated models
    ResNet-18, GPT-2 Medium
    AbstractDP-MixMin 私下学公开混合再 DP 微调，胸片 macro AUC 与邮件困惑度优于均匀混合。

    DP-MixMin 是一条端到端的差分隐私训练流程：先私下决定多个公开数据集的预训练混合，再预训练并在敏感数据上 DP 微调。作者称这是首个这样的流程。

    Paper analysisRead analysis
  7. AttacksarXiv 2610.05601

    Tracer 框架可从遗忘后的扩散模型中识别被擦除概念

    提出 TRACER，从权重差识别扩散模型被擦除概念

    arXiv
    2610.05601
    Published
    Layer
    Model layer
    Setting
    Models and APIs
    Attacker
    White-box
    Evaluated models
    Stable Diffusion v1.5, Stable Diffusion 3, FLUX.1 and others5
    AbstractTRACER 用权重足迹识别未披露擦除概念并估计个数，作者称数秒内完成且准确率高于对照。

    TRACER 针对的是遗忘扩散模型上的目标识别：提供者从公开基座 W 用已发表方法擦掉未知集合 S⋆，只发布 W′。攻击者还有假定包含这些概念的词表，但不知擦了谁、擦了几个、用了什么算法。

    Paper analysisRead analysis
  8. AttacksarXiv 2610.05453

    研究者提出 Privacy-Leaking Watermarks，可在本地多模态模型中泄露对话隐私

    提出 PLW，用微调把先前对话敏感触发编码进后续生成图像

    arXiv
    2610.05453
    Published
    Layer
    Training and data layer
    Setting
    Models and APIs
    Evaluated models
    OmniGen2, BAGEL-7B

    AbstractPLW 把先前对话线索写成后续图像中的隐水印，检出随触发、模型和间隔变化。

    Paper analysisRead analysis
  9. DefensesarXiv 2610.04426

    UnAct：无需梯度的定向激活干预实现类别遗忘

    提出 UnAct，仅以前向激活干预从分类器中遗忘指定类别

    arXiv
    2610.04426
    Published
    Layer
    Model layer
    Setting
    Models and APIs
    Evaluated models
    ResNet-18, ViT-B/16
    AbstractUnAct 用前向激活做类别遗忘。

    UnAct 是一种事后类别遗忘：删除请求可以只有少量该类图像，且没有标签和保留数据。它不用梯度，只对遗忘图像做前向。编辑对象是分类器所读的后期单元。

    Paper analysisRead analysis
  10. AttacksarXiv 2610.04128

    研究显示拆分学习中的梯度可提升客户端文本重建率

    衡量分裂学习中梯度相对激活对客户文本重建的额外贡献

    arXiv
    2610.04128
    Published
    Layer
    Training and data layer
    Setting
    Models and APIs
    Evaluated models
    GPT-2 (openai-community/gpt2, 124M), Qwen3-0.6B
    Abstract激活已恢复大部分 token。

    这篇工作测量 split learning 分裂点上的文本泄漏：激活单独能恢复多少，训练回传的梯度再增加多少，以及按 token 还是按整篇计分会不会改写结论。

    Paper analysisRead analysis
  11. AttacksarXiv 2610.01365

    ReGap：无需真实隐私数据即可通过微调重新提取模型隐私关联

    提出 ReGap，用自生成伪监督微调恢复模型中的隐私关联

    arXiv
    2610.01365
    Published
    Layer
    Training and data layer
    Setting
    Models and APIs
    Attacker
    White-box, Gray-box
    Evaluated models
    GPT-2 Small, GPT-2 Medium, GPT-2 Large and others6

    Abstract提出无真实私有监督的恢复攻击 ReGap，实验显示微调可通过自生成数据唤醒模型潜在隐私关联，突显后适应隐私风险。

    Paper analysisRead analysis
  13. DefensesarXiv 2609.37858

    存储不等于策略：面向 LLM 遗忘的状态条件支持控制

    提出 Intervention Score 与 DIR-R，按干预效果选择并修订遗忘参数支持

    arXiv
    2609.37858
    Published
    Layer
    Training and data layer
    Setting
    Models and APIs
    Evaluated models
    Llama 3 8B, Gemma 2 2B, Qwen2.5-7B and others4
    Abstract作者把局部化遗忘拆成初始干预选择和检查点修订，并在三个设置里分别给证据和边界。

    Storage Is Not Strategy 研究局部化 LLM 遗忘里两个常被混在一起的决定：一开始该更新哪些参数，以及优化改变模型之后这个选择要不要改。

    Paper analysisRead analysis
  15. AttacksarXiv 2609.33985

    研究：众包微调数据投毒可放大专有指令抽取

    提出主题锚点投毒，放大众包微调后的专有指令提取

    arXiv
    2609.33985
    Published
    Layer
    Training and data layer
    Setting
    Models and APIs
    Attacker
    Black-box
    Evaluated models
    Qwen2.5-7B-Instruct, Qwen2.5-14B-Instruct, Llama-3.1-8B-Instruct and others4

    Abstract论文提出基于主题锚点的数据中毒攻击，证明低比例的中毒样本可在黑盒条件下隐蔽放大微调指令泄漏。

    Paper analysisRead analysis
  16. Analysis and theoryarXiv 2609.32288

    预训练数据投毒的可解理论：依赖极限顺序的缩放指数

    证明预训练投毒的清洁超额风险指数随极限顺序变化

    arXiv
    2609.32288
    Published
    Layer
    Training and data layer
    Setting
    Models and APIs
    Evaluated models
    OLMo-style models
    Abstract投毒清洁超额风险的幂指数取决于先固定宽高比还是先取充足数据极限。

    可解的预训练数据投毒理论：清洁超额风险的缩放指数随极限顺序改变。。

    Paper analysisRead analysis
  17. AttacksarXiv 2609.27124

    Fed-ADR：用恶意编排服务器协同客户端梯度操纵攻击联邦学习

    提出 Fed-ADR，用编排服务器协同操纵梯度攻破联邦学习并检测恢复

    arXiv
    2609.27124
    Published
    Layer
    Training and data layer
    Setting
    Models and APIs
    Attacker
    Gray-box
    Evaluated models
    CNN（MNIST：2 Conv + 3 FC）, CNN（Fashion-MNIST：6 Conv + 1 FC）, ResNet18（CIFAR-10）
    AbstractFed-ADR 用 gray-box 编排攻击绕过多种聚合防御，再用 Hessian 一致性检测和窗口内恢复把准确率拉回。

    Fed-ADR 是一套面向联邦学习的攻击、检测与恢复框架：编排服务器改写合谋客户端的梯度，使参数服务器的现有防御把投毒更新当成良性，同时在发现后用历史梯度把全局模型原地修回来。

    Paper analysisRead analysis
  20. AttacksarXiv 2609.22711

    UBA-ORL：针对离线强化学习的遗忘激活后门攻击

    提出 UBA-ORL，以 BD/CM 样本在遗忘后激活离线 RL 后门

    arXiv
    2609.22711
    Published
    Layer
    Training and data layer
    Setting
    Embodied systems and robotics
    Evaluated models
    TD3+BC, BCQ, IQL
    AbstractUBA-ORL 用竞争的 BD/CM 样本，使离线 RL 后门在删除伪装轨迹后激活。

    UBA-ORL 研究离线强化学习中由合规轨迹删除激活的后门：合法贡献者同时上传共享触发的后门轨迹和伪装轨迹，训练后触发响应被压低，删除伪装批次后再升高。受害系统是使用静态数据集和轨迹级遗忘接口的离线 RL 服务。对手不能看到训练算法、参数、梯度或具体遗忘算法，但能提交可区分批次，并在部署时向观测注入预定触发。

    Paper analysisRead analysis
Load more