Translation in progress Some entries still include text in the original language.
Oct 12 · Monday
Another 265 papers do not have category tags yet and are not included in filtered results.
“
About this collection Collection in progress We are building a large AI safety paper collection to help researchers find work related to their interests. Papers are collected automatically andfiltered only for relevance to AI safety , without judging quality, novelty or reading value. The default “Annotated first” shows papers with tags or published paper analyses first. Each group is ordered by original publication date, newest first, with undated papers last. “Newest first” uses publication dates only. This order is not a quality ranking.
One-line summaries, tags and paper analyses are prepared by AI from the papers. Language models may have systematic biases and produce hallucinations.We plan to carry out human annotation and review to reduce errors in this content.
Trace attention to identify harmfulness to the MLP refusal write pathway and propose Sentinel
Published Oct 9
Evaluated models Llama-3.1-8B-Instruct , Qwen2.5-7B-Instruct , Qwen3-8B-Instruct and others18 Abstract Attention updates are more readable for harmfulness, MLP updates write refusal, and Sentinel routes alerts using two-path signals.The authors reframe jailbreaking and excessive refusal of aligned models as a pathway problem on the residual stream from recognition to writing, and accordingly provide conditional reminders before decoding.
提出 Safety Operator，用谱优化调节安全指令表达强度
Published Sep 29
Evaluated models Gemma 3 1B , Gemma 3 4B 用后验重加权解释多模态上下文越狱，并提出风险门控的良性示例注入防御
Published Sep 8
Evaluated models Qwen3-VL-2B , Qwen3-VL-4B , Qwen3-VL-8B and others8 Abstract 后验重加权解释上下文越狱的缩放，并支持风险门控的良性示例防御。Zhang 等把多模态 ICL 越狱写成对齐模型在安全与有害隐模式之间的推理时后验漂移，并据此做风险门控防御。
提出 AlcaTRAz，用锚定字符规则树改写提示以防御越狱
Published Sep 3
Evaluated models LLaMA 2 , LLaMA 3.1 , LLaMA 3.2 and others14 分析少样本示范对 RoP 与 ToP 提示防御抗越狱效果的影响
Published Feb 4
Evaluated models Pangu-Embedded-1B , Pangu-Embedded-7B , Pangu-Embedded-7B-Think and others9 Abstract few-shot 可抬高 RoP、压低 ToP，但表内有反号，Think 模型更低。作者评估 few-shot 示范如何改变 RoP 与 ToP 对越狱的防御效果。
提出 TAME 调节 Agent 测试时记忆演化，并用 Trust-Memevo 评测可信度
Published Feb 3
Evaluated models Qwen3-32B , GPT-5.2 Abstract TAME 在共享记忆上同时调节效用与信任。TAME 针对测试时记忆演化中的 Agent Memory Misevolution：良性任务只按成败保留经验时，任务表现可以上升，安全、隐私、公平等可信度下降或随领域波动。
提出 DASE，以对抗自演化知识库在推理时加固角色扮演抗越狱
Published Jan 29
Evaluated models Kimi-K2-Instruct , GPT-5.2 Abstract DASE 用双循环知识库，在两家闭源模型上同时提高保真分与拒答率。DASE 是给专有闭源角色扮演智能体用的训练自由防御，目标是人设保真与安全拒答一起成立，而不是改模型权重。问题在于人设沉浸，尤其负面人设，会压过原有护栏。SFT、数据筛选和正则又跟不上新人设与新攻击，也不能用于没有梯度的 black-box API。
分析按 Schwartz 价值对齐后的意外危害，并提出上下文对齐方法
Published Jun 6, 2025
Evaluated models Llama-2 7B , Llama-3.1 8B , Gemma-2 9B and others6 Abstract 价值对齐会随 Schwartz 价值改变有害倾向。作者研究按 Schwartz 价值对齐的 LLM 为何在安全评测里更易有害，并用提示做了一个不直说要安全的缓解试验。
Defenses arXiv 2312.06632 · Dec 12, 2023 提出 SciGuard 外部护栏与 SciMT 基准，控制化学科学 AI 误用
Published Dec 12, 2023
Evaluated models SciGuard (GPT-4) , SciGuard (Gemini) , SciGuard (Mixtral) and others5 Abstract SciGuard 以外部智能体控制化学 AI 误用。这篇工作把化学科学 AI 的误用控制做成外部智能体，而不是改模型权重。作者担心合成规划、毒性预测和科学语言模型被用来获取有害物质信息、规避监管或传播危险知识，同时合法工业用途又不能靠一律拒绝来处理。SciGuard 由 LLM 担任中介，把原则和指南放进短期记忆，把分子库、危害库、法规和交互历史放进可检索的长期记忆，再按 Chain of Thought 调用 Chemprop、LocalRetro 等工具，或用 RAG 补法规片段，最后回答、追问或拒绝。
You have reached the end