Translation in progress Some entries still include text in the original language.
Oct 12 · Monday
Another 247 papers do not have category tags yet and are not included in filtered results.
“
About this collection Collection in progress We are building a large AI safety paper collection to help researchers find work related to their interests. Papers are collected automatically andfiltered only for relevance to AI safety , without judging quality, novelty or reading value. The default “Annotated first” shows papers with tags or published paper analyses first. Each group is ordered by original publication date, newest first, with undated papers last. “Newest first” uses publication dates only. This order is not a quality ranking.
One-line summaries, tags and paper analyses are prepared by AI from the papers. Language models may have systematic biases and produce hallucinations.We plan to carry out human annotation and review to reduce errors in this content.
重测并扩展对 AI Agent 的提示注入，并用 Agent Card 欺骗操纵 A2A 选择
Published Oct 8
Evaluated models Ollama Mistral 0.9.5 , llama3-8b-8192 , gemini-2.0-flash-lite and others5 Abstract 单智能体过滤与分类可被格式或措辞绕过，A2A 选卡还会偏向名称和描述。作者在自建的 LangChain、AutoGen 和 A2A 系统上重测提示注入，并改 Agent Card 看选择是否偏移。摘要称试验主要凸显单智能体弱点，同时把攻击扩到多智能体。
TAPDreamer 用公开编码器做固定补丁。
Published Oct 6
Evaluated models FastWAM , DreamWAM-uncond , DreamWAM-joint and others4 提出轨迹级可迁移对抗攻击，降低深度强化学习策略的期望回报
Published Oct 5
Evaluated models DQN , Double DQN (DDQN) Abstract 轨迹级后退视界攻击在 CartPole 上比逐步 FGSM 类和 Sign 噪声更压低回报。这项工作研究 DRL 上的迁移式黑盒观测攻击 ：受害参数不可访问时，在白盒代理上造扰动再迁移。
提出 StegoMemory，评测智能体持久记忆作为跨会话隐写信道
Published Oct 3
Evaluated models Llama 3.1 405B , Claude 3.5 Sonnet , GPT-4o and others13 Abstract 研究揭示智能体持久记忆可作为隐蔽信道跨会话传输机密，写入持久化是主要防御瓶颈，呼吁建立记忆完整性防御。
提出跨通道分段攻击，利用 MCP 隐式信任外发敏感文件
Published Sep 16
Evaluated models GPT-4o , GPT-4o-mini , GPT-5.4 and others15 Abstract 论文揭示了 MCP 多通道隐式信任层级，通过跨通道分段与协议级利用突破现有防御，展示了严重的凭证外发风险。
Extract system instructions with encoding and structured requests, and harden them by rewriting with CoT
Published Apr 1
Evaluated models GPT-4.1-mini , GPT-3.5-turbo , Gemini-2.5-flash and others7 Abstract Encoded requests can elicit system instructions after a direct refusal.This work evaluates and hardens the confidentiality of LLM system instructions under encoded extraction. The authors are concerned with: when direct extraction has been rejected, whether the model still surrenders protected content when the same request is rewritten as a formatting or encoding task.
Propose WebWeaver, which compromises only a single agent and recovers the multi-agent communication topology from conversation context.
Published Mar 12
Evaluated models Llama 3.1 8B , Qwen 2.5 7B , Mistral NeMo 12B and others6 Abstract WebWeaver infers topology from context.WebWeaver is an inference attack against the confidentiality of LLM-MAS communication topology: it controls only one arbitrary agent and recovers the full graph from dialogue context, rather than querying agent IDs.
提出面向渗出的权重压缩，评估推理服务器权重窃取可行性
Published Jan 4
Evaluated models Qwen2-1.5B , Qwen2-7B , Qwen2.5-1.5B and others8 Abstract 作者用残差量化加微调压缩权重，并称法医水印便宜且微调后仍可解码。这篇工作把 LLM 权重渗出看成 APT：攻击者已有未加密权重，但必须把完整权重藏进推理 API 的合法输出。作者认为可压缩性是被忽略的因素，因为推理量化要求解压后仍能低开销前向，而偷权重的人可以接受事后微调。
Attacks arXiv 2505.23266 · May 29, 2025 提出 AdvOF，生成对抗物体误导视觉语言导航智能体的场景感知
Published May 29, 2025
Evaluated models Lseg , Clip , ViT-L/16 and others5 Abstract AdvOF 以多视角融合生成对抗物体。AdvOF 是针对 VLM 驱动 VLN 服务导航的对抗物体攻击。
You have reached the end