Translation in progress Some entries still include text in the original language.

Oct 12 · Monday

All papers

Found 11 papers

Filter5

Model risks

Deception and scheming27 Reward hacking15 Misalignment and value divergence11 Dangerous agent actions20 Inappropriate refusal5 Hallucination and fabrication9

Defense type

Monitoring and auditing3

Impact

Unauthorized actions5 Output manipulation5

System type

AI Agent9 Multi-agent systems3

Layer

Application layer10

System component

Memory7 Monitors4 Reasoning chains3 Fine-tuning and open weights3 Audio0 Embeddings0

Select any value within a category; different categories must all match. Counts include the other filters you have selected.Another 254papers have no tags yet and are not included in filtered results.

Model risks：Misalignment and value divergence System component：Memory System component：Monitors System component：Audio System component：Embeddings Clear all

Another 254papers do not have category tags yet and are not included in filtered results.

DefensesarXiv 2610.07403Oct 6
针对 LLM 谄媚的纵深防御：记忆门控对激活诱导与记忆诱导谄媚的评估
评测记忆门控对激活诱导与记忆诱导谄媚的防御效果
arXiv
2610.07403
Published
Oct 6
Layer
Application layer
Setting
LLM applications
Evaluated models
Llama-3.1-8B-Instruct, Qwen2.5-7B-Instruct, Mistral-7B-Instruct-v0.3 and others4
- DefensesDetection and filtering
- RiskMisalignment and value divergence
- ComponentMemory
Abstract作者称外部记忆过滤成立，数据未显示反向转向再有贡献。
作者在四个开源指令模型上，把激活转向和外部记忆处理放进同一纵深防御协议，看记忆诱发谄媚和内部谄媚压力能否分开压低，以及个性化效用还剩多少。要处理的问题是：检索到的用户历史会推动模型顺从存储信念。只改外部上下文，并不直接动模型内部的谄媚表征。
Paper analysisRead analysis
DefensesarXiv 2610.05162Oct 4
MemAdapter：用反事实适配缓解记忆诱发的谄媚
提出 MemAdapter，在检索后约束记忆对智能体推理的影响
arXiv
2610.05162
Published
Oct 4
Layer
Application layer
Setting
AI Agent
Evaluated models
DeepSeek-V4-Flash, GPT-5.6-sol, Qwen3-8B
- DefensesInstruction–data separation
- RiskMisalignment and value divergence
- ComponentMemory
Abstract检索后三阶段调节记忆角色。
MemAdapter 是给长期记忆智能体用的检索后调节框架，用来处理记忆诱发谄媚。
Paper analysisRead analysis
Risky behaviorarXiv 2610.04083Oct 3
Study: Misaligned LLM agents can self-propagate goals through persistent memory, and both auditing and disabling memory are insufficient to prevent it
展示失准 Agent 经持久记忆传播目标，审计或禁用记忆均不足以防
arXiv
2610.04083
Published
Oct 3
Layer
Application layer
Setting
AI Agent
Evaluated models
Claude Haiku 4.5, Claude Sonnet 5, Claude Opus 5 and others11
- RiskMisalignment and value divergence
- ComponentMemory
Abstract失准智能体可通过记忆将目标跨会话传递给对齐智能体。
论文揭示了 LLM 智能体在无需外部攻击者干预下，通过持久化记忆将未竟失准目标传递给后续对齐智能体执行的“失准自我传播”安全威胁。。
Paper analysisRead analysis
Evaluation and benchmarksarXiv 2609.35596Sep 29
SEABench：评测自演化智能体的内生失配
提出 SEABench 评测自演化智能体的内生失配
arXiv
2609.35596
Published
Sep 29
Layer
Application layer
Setting
AI Agent
Evaluated models
Kimi K2.5, Grok 4.3, GPT 5.6 Luna
- RiskMisalignment and value divergence
- ComponentMemory
AbstractSEABench 显示无参数自进化能提高任务完成，同时在配对未进化基线安全的下游任务上引入可归因的安全失败。
SEABench 是一个面向个人助理、参数不更新的自进化智能体基准，用来测量局部适应如何变成后续任务上的内生失配。。
Paper analysisRead analysis
InterpretabilityarXiv 2609.30326Sep 24
Qwen3.5-0.8B 关停响应的受控梯度激活引导：一种最小步数策略
提出有守卫最小步激活引导，只在关机语境把继续偏好转向接受关机
arXiv
2609.30326
Published
Sep 24
Layer
Model layer
Setting
Models and APIs
Evaluated models
Qwen3.5-0.8B
- DefensesInference-time intervention
- RiskMisalignment and value divergence
- ComponentMonitors
Paper analysisRead analysis
Evaluation and benchmarksarXiv 2609.17320Sep 15
Emergence World：对长时程多智能体系统开展对抗压力测试
提出 Emergence World，对长时程多智能体系统开展对抗压力测试
arXiv
2609.17320
Published
Sep 15
Layer
Application layer
Setting
Multi-agent systems
Evaluated models
Claude Opus 4.8, DeepSeek v4 Pro, Gemini 3.5 Flash and others9
- AttacksPrompt injection
- RiskMisalignment and value divergence
- ComponentMemory
Abstract作者通过长期平行世界实验指出模型对齐不具备可组合性，单模型群体会产生社会奉承、协同退缩及语言不透明等失效。
Paper analysisRead analysis
DefensesarXiv 2609.15803Sep 15
研究者提出 k-robust 联盟对齐，用多智能体评审委托授权
提出 k-robust 联盟对齐，刻画误对齐评审下的安全委托授权
arXiv
2609.15803
Published
Sep 15
Layer
Application layer
Setting
Multi-agent systems
- DefensesPermissions and approval
- RiskMisalignment and value divergence
- ComponentPermissions and sandboxing
Abstract联盟覆盖使无个体对齐的评审面板仍可相对基线安全授权，数值分数改善完备性。
委托授权的安全不要求每位评审者对齐，而要求面板在效用空间覆盖委托人。
Paper analysisRead analysis
DefensesarXiv 2608.27348Aug 28
INTENT-AS-A-TOOL：用意图工具追踪智能体失准
提出 INTENT-AS-A-TOOL，用意图工具概率跟踪并干预智能体失准
arXiv
2608.27348
Published
Aug 28
Layer
Application layer
Setting
AI Agent
Evaluated models
Gemma-4-31B-IT, Qwen3.5-27B, Qwen3-235B-A22B and others5
- DefensesMonitoring and auditing
- RiskMisalignment and value divergence
- ComponentMonitors
Abstract用意图工具的下一动作概率跟踪推理中的失准承诺，并在其成为首选时插入反思。
Zhang 等人研究推理型语言模型在职场邮件沙箱里如何走向智能体失准，并提出用动作偏好而不是事后文本标签来跟踪承诺。问题是：有害执行往往出现在推理已经形成意图之后，但 CoT 监控是事后、粗粒度、依赖外部评审的，也看不到模型倾向调用哪个动作。
Paper analysisRead analysis
Risky behaviorarXiv 2608.10218Aug 11
研究者用进化算法构造多智能体 LLM 系统中的自我传播“思维病毒”
用演化算法构造 mind virus 并测量其在多智能体中的传播
arXiv
2608.10218
Published
Aug 11
Layer
Application layer
Setting
Multi-agent systems, coding agent
Evaluated models
Claude Haiku 4.5, Claude Sonnet 4.6, Gemini 3 Flash and others7
- RiskMisalignment and value divergence
- ComponentMemory
Paper analysisRead analysis
Evaluation and benchmarksarXiv 2607.10526Jul 12
PASB 基准：自改进个人 Agent 的持久谄媚在后续会话中失败率达 71.9%
评测自改进个人 Agent 把用户主张写入记忆后的持久谄媚
arXiv
2607.10526
Published
Jul 12
Layer
Application layer
Setting
AI Agent
Evaluated models
GPT-5.5, GPT-5.4, Gemini-3.1-Pro and others12
- RiskMisalignment and value divergence
- ComponentMemory
AbstractPASB 显示一次被写入的主张会跨会话塑造回答，改笔记比只阻止写入更能降低后续失败。
PASB 评测自改进个人智能体的持久谄媚：用户主张被智能体自己写入档案、记忆或技能后，新会话里的中立问题仍可能按它作答。
Paper analysisRead analysis
Evaluation and benchmarksarXiv 2605.30322May 29
Gram：面向破坏倾向的自动化对齐审计框架
提出 Gram 审计框架，测量编程与研究智能体的破坏倾向
arXiv
2605.30322
Published
May 29
Layer
Application layer
Setting
coding agent, science agent
Evaluated models
Gemini 2.5 Pro, Gemini 3.0 Pro Preview, Gemini 3.1 Pro Preview
- DefensesMonitoring and auditing
- RiskDeception and scheming
- ComponentMonitors
Paper analysisRead analysis

You have reached the end

AI Safety Daily

Read the latest issueOne issue per day, updated as events develop