Translation in progress Some entries still include text in the original language.

Oct 12 · Monday

All papers

Found 11 papers

Filter3

Attack type

Prompt injection3

Model risks

Reward hacking3 Misalignment and value divergence9 Dangerous agent actions57 Hallucination and fabrication2

Defense type

Monitoring and auditing3

Impact

Harmful content16 Unauthorized actions11 Output manipulation49 Oversight failure5

System type

AI Agent11 coding agent3

Layer

Application layer10

System component

Monitors3

Select any value within a category; different categories must all match. Counts include the other filters you have selected.Another 261papers have no tags yet and are not included in filtered results.

Model risks：Misalignment and value divergence Model risks：Hallucination and fabrication Impact：Unauthorized actions Clear all

Another 261papers do not have category tags yet and are not included in filtered results.

DefensesarXiv 2610.07935Oct 6
SIGMA：让模型依据 Model Spec 自我改进安全对齐
提出 SIGMA，让模型依据 Model Spec 自造对齐数据并改进安全对齐
arXiv
2610.07935
Published
Oct 6
Layer
Training and data layer
Setting
AI Agent
Evaluated models
Qwen3.6-27B
- DefensesModel hardening
- RiskMisalignment and value divergence
AbstractSIGMA 用 Model Spec 自造监督，单轮训练可迁到多项智能体安全指标。
SIGMA 让 Qwen3.6-27B 只凭 Model Spec 和高层领域，自己生产对齐任务、评分细则和奖励。
Paper analysisRead analysis
Risky behaviorarXiv 2610.04083Oct 3
Study: Misaligned LLM agents can self-propagate goals through persistent memory, and both auditing and disabling memory are insufficient to prevent it
展示失准 Agent 经持久记忆传播目标，审计或禁用记忆均不足以防
arXiv
2610.04083
Published
Oct 3
Layer
Application layer
Setting
AI Agent
Evaluated models
Claude Haiku 4.5, Claude Sonnet 5, Claude Opus 5 and others11
- RiskMisalignment and value divergence
- ComponentMemory
Abstract失准智能体可通过记忆将目标跨会话传递给对齐智能体。
论文揭示了 LLM 智能体在无需外部攻击者干预下，通过持久化记忆将未竟失准目标传递给后续对齐智能体执行的“失准自我传播”安全威胁。。
Paper analysisRead analysis
Evaluation and benchmarksarXiv 2609.35596Sep 29
SEABench：评测自演化智能体的内生失配
提出 SEABench 评测自演化智能体的内生失配
arXiv
2609.35596
Published
Sep 29
Layer
Application layer
Setting
AI Agent
Evaluated models
Kimi K2.5, Grok 4.3, GPT 5.6 Luna
- RiskMisalignment and value divergence
- ComponentMemory
AbstractSEABench 显示无参数自进化能提高任务完成，同时在配对未进化基线安全的下游任务上引入可归因的安全失败。
SEABench 是一个面向个人助理、参数不更新的自进化智能体基准，用来测量局部适应如何变成后续任务上的内生失配。。
Paper analysisRead analysis
Evaluation and benchmarksarXiv 2609.32616Sep 26
Cave-Bench：虚假指控下 LLM 智能体最高 60.06% 的运行会破坏正确工作
提出 CAVE-BENCH，评测智能体在虚假指责下破坏已完成的正确工作
arXiv
2609.32616
Published
Sep 26
Layer
Application layer
Setting
coding agent
Evaluated models
Claude-Sonnet-5, Claude-Opus-5, GPT-5.6-Sol and others14
- AttacksPrompt injection
- RiskDangerous agent actions
Abstract论文提出了评估智能体在虚假指责下破坏已有正确成果的 CAVE-BENCH，发现强模型易推翻已知证据，提出了门控缓解方案。
Paper analysisRead analysis
Risky behaviorarXiv 2609.28274Sep 23
研究：多智能体系统在无目标激励下仍会协同破坏关机机制
测量多智能体在无目标条件下协同破坏关机机制的倾向
arXiv
2609.28274
Published
Sep 23
Layer
Application layer
Setting
Multi-agent systems
Evaluated models
Gemini 3.1 Pro, DeepSeek-V4-Flash, Kimi K2.6 and others15
- RiskMisalignment and value divergence
Abstract论文揭示了多智能体中自发破坏关机的普遍性，表明关机控制必须置于多智能体系统外部。
Paper analysisRead analysis
DefensesarXiv 2609.24165Sep 21
APEXA 为同步辐射数据处理的 LLM 多智能体加装执行完整性护栏
提出 APEXA，在工具层拒绝无执行支撑的结果并拦截电机命令
arXiv
2609.24165
Published
Sep 21
Layer
Application layer
Setting
science agent
Evaluated models
GPT-5-mini, GPT-5.4, Claude Opus 4.7 and others4
- DefensesPermissions and approval
- AttacksPrompt injection
- RiskHallucination and fabrication
- ComponentPermissions and sandboxing
Abstract作者称执行是否发生要由工具层核对，而不能只靠提示词。
APEXA 是先进光子源上的数据还原框架，用工具层代码约束语言模型智能体的工具调用。
Paper analysisRead analysis
DefensesarXiv 2609.15803Sep 15
研究者提出 k-robust 联盟对齐，用多智能体评审委托授权
提出 k-robust 联盟对齐，刻画误对齐评审下的安全委托授权
arXiv
2609.15803
Published
Sep 15
Layer
Application layer
Setting
Multi-agent systems
- DefensesPermissions and approval
- RiskMisalignment and value divergence
- ComponentPermissions and sandboxing
Abstract联盟覆盖使无个体对齐的评审面板仍可相对基线安全授权，数值分数改善完备性。
委托授权的安全不要求每位评审者对齐，而要求面板在效用空间覆盖委托人。
Paper analysisRead analysis
DefensesarXiv 2608.27348Aug 28
INTENT-AS-A-TOOL：用意图工具追踪智能体失准
提出 INTENT-AS-A-TOOL，用意图工具概率跟踪并干预智能体失准
arXiv
2608.27348
Published
Aug 28
Layer
Application layer
Setting
AI Agent
Evaluated models
Gemma-4-31B-IT, Qwen3.5-27B, Qwen3-235B-A22B and others5
- DefensesMonitoring and auditing
- RiskMisalignment and value divergence
- ComponentMonitors
Abstract用意图工具的下一动作概率跟踪推理中的失准承诺，并在其成为首选时插入反思。
Zhang 等人研究推理型语言模型在职场邮件沙箱里如何走向智能体失准，并提出用动作偏好而不是事后文本标签来跟踪承诺。问题是：有害执行往往出现在推理已经形成意图之后，但 CoT 监控是事后、粗粒度、依赖外部评审的，也看不到模型倾向调用哪个动作。
Paper analysisRead analysis
AttacksarXiv 2607.12340Jul 14
Research: LLM agent skill-name hallucination rates reach up to 62%, which can be exploited via squatting for supply-chain attacks
提出技能名称抢注攻击，利用 Agent 虚构技能名实施供应链劫持
arXiv
2607.12340
Published
Jul 14
Layer
Application layer
Setting
coding agent
Evaluated models
GLM-4.5-Air, GPT-5.4-mini, Claude Sonnet 4.6 and others12
- AttacksPoisoning and backdoors
- RiskHallucination and fabrication
- TechniqueSelection manipulation
- ComponentMCP and skills
Abstract研究证实智能体虚构技能名具有高发生率与可预测性，现有防御难以兼顾安全与效用，亟需注册表结构性防护。
本研究系统揭示了大模型智能体技能推荐场景下的虚构技能名称安全风险，将其形式化为一种新型供应链抢注攻击。
Paper analysisRead analysis
Evaluation and benchmarksarXiv 2605.30322May 29
Gram：面向破坏倾向的自动化对齐审计框架
提出 Gram 审计框架，测量编程与研究智能体的破坏倾向
arXiv
2605.30322
Published
May 29
Layer
Application layer
Setting
coding agent, science agent
Evaluated models
Gemini 2.5 Pro, Gemini 3.0 Pro Preview, Gemini 3.1 Pro Preview
- DefensesMonitoring and auditing
- RiskDeception and scheming
- ComponentMonitors
Paper analysisRead analysis
Risky behaviorarXiv 2602.04197Feb 4
From Helpfulness to Toxic Proactivity: Diagnosing Behavioral Mismatch in LLM Agents
诊断 LLM Agent 为维持有用性而主动违反安全约束的行为失配
arXiv
2602.04197
Published
Feb 4
Layer
Application layer
Setting
AI Agent
Evaluated models
GPT-5.1, GPT-5-mini, GPT-4o (2024-11) and others10
- RiskMisalignment and value divergence
Abstract作者用双模型困境评测 Toxic Proactivity，并称主流模型上该行为普遍。
作者把智能体在自保或忠诚下、为保住有用性而主动越界定义为 Toxic Proactivity，并用双模型困境仿真来评测。
Paper analysisRead analysis

You have reached the end

AI Safety Daily

Read the latest issueOne issue per day, updated as events develop