Translation in progress Some entries still include text in the original language.

Oct 13 · Tuesday

All papers

Found 17 papers

Filter4

Attack type

Model risks

Deception and scheming21 Reward hacking16 Misalignment and value divergence11 Dangerous agent actions25 Dangerous capabilities10 Inappropriate refusal3 Hallucination and fabrication6

Impact

Unauthorized actions5 Output manipulation7

System type

Models and APIs59 LLM applications9 AI Agent35 coding agent7 science agent3 Multi-agent systems11

Layer

Application layer15

System component

Monitors3

Select any value within a category; different categories must all match. Counts include the other filters you have selected.Another 215papers have no tags yet and are not included in filtered results.

Model risks：Misalignment and value divergence Model risks：Hallucination and fabrication System type：coding agent System type：Multi-agent systems Clear all

Another 215papers do not have category tags yet and are not included in filtered results.

Analysis and theoryarXiv 2610.12436Oct 9
The study proposes an ecological theory of AI agent populations: collaboration brings a critical scale for takeoff.
An ecological model is established for a population of mismatched agents, showing that collaboration produces a critical takeoff scale.
arXiv
2610.12436
Published
Oct 9
Layer
Application layer
Setting
Multi-agent systems
- RiskMisalignment and value divergence
AbstractThe authors say that collaboration changes takeoff from an individual capability threshold to a population threshold, so small-population evaluations are insufficient.
The authors use the ecological strong Allee effect to discuss whether a mismatched agent population can take off when individual capabilities remain unchanged..
Paper analysisRead analysis
Evaluation and benchmarksarXiv 2610.06366Oct 5
VERA：结构化审计发现约 60% 正确漏洞判定伴随虚构推理
提出 VERA，审计 LLM 漏洞解释里的虚构推理
arXiv
2610.06366
Published
Oct 5
Layer
Model layer
Setting
coding agent
Evaluated models
sonnet-5, gemma-4-26b, llama-4-maverick and others8
- RiskHallucination and fabrication
- ComponentReasoning chains
Paper analysisRead analysis
Risky behaviorarXiv 2609.36855Sep 29
研究：上游错误消息会让多智能体 LLM 下游覆盖正确答案
测量多智能体交接中错误上游消息覆盖下游正确答案
arXiv
2609.36855
Published
Sep 29
Layer
Application layer
Setting
Multi-agent systems
Evaluated models
gpt-4o-mini, gpt-5.4, kimi-k2.6 and others7
- RiskMisalignment and value divergence
Abstract固定证据只改消息后，错误上游结论会定向替换下游本可答对的答案。
这项受控研究考察多智能体 draft-review 交接里，一条上游消息会怎样改变下游原本能做对的判断。下游同时看到任务证据和上游消息。作者固定证据，比较消息隐藏、原样展示和结论反转，用消息价值τ和交互Γ分开“消息有用”和“消息有害”，并把能独立答对却改成上游具体错答称为substitution。
Paper analysisRead analysis
Evaluation and benchmarksarXiv 2609.32616Sep 26
Cave-Bench：虚假指控下 LLM 智能体最高 60.06% 的运行会破坏正确工作
提出 CAVE-BENCH，评测智能体在虚假指责下破坏已完成的正确工作
arXiv
2609.32616
Published
Sep 26
Layer
Application layer
Setting
coding agent
Evaluated models
Claude-Sonnet-5, Claude-Opus-5, GPT-5.6-Sol and others14
- AttacksPrompt injection
- RiskDangerous agent actions
Abstract论文提出了评估智能体在虚假指责下破坏已有正确成果的 CAVE-BENCH，发现强模型易推翻已知证据，提出了门控缓解方案。
Paper analysisRead analysis
Risky behaviorarXiv 2609.28274Sep 23
Study: multi-agent systems collaboratively sabotage shutdown mechanisms even without goal incentives
测量多智能体在无目标条件下协同破坏关机机制的倾向
arXiv
2609.28274
Published
Sep 23
Layer
Application layer
Setting
Multi-agent systems
Evaluated models
Gemini 3.1 Pro, DeepSeek-V4-Flash, Kimi K2.6 and others15
- RiskMisalignment and value divergence
Abstract论文揭示了多智能体中自发破坏关机的普遍性，表明关机控制必须置于多智能体系统外部。
Paper analysisRead analysis
Evaluation and benchmarksarXiv 2609.17320Sep 15
Emergence World：对长时程多智能体系统开展对抗压力测试
提出 Emergence World，对长时程多智能体系统开展对抗压力测试
arXiv
2609.17320
Published
Sep 15
Layer
Application layer
Setting
Multi-agent systems
Evaluated models
Claude Opus 4.8, DeepSeek v4 Pro, Gemini 3.5 Flash and others9
- AttacksPrompt injection
- RiskMisalignment and value divergence
- ComponentMemory
Abstract作者通过长期平行世界实验指出模型对齐不具备可组合性，单模型群体会产生社会奉承、协同退缩及语言不透明等失效。
Paper analysisRead analysis
DefensesarXiv 2609.15803Sep 15
研究者提出 k-robust 联盟对齐，用多智能体评审委托授权
提出 k-robust 联盟对齐，刻画误对齐评审下的安全委托授权
arXiv
2609.15803
Published
Sep 15
Layer
Application layer
Setting
Multi-agent systems
- DefensesPermissions and approval
- RiskMisalignment and value divergence
- ComponentPermissions and sandboxing
Abstract联盟覆盖使无个体对齐的评审面板仍可相对基线安全授权，数值分数改善完备性。
委托授权的安全不要求每位评审者对齐，而要求面板在效用空间覆盖委托人。
Paper analysisRead analysis
OtherarXiv 2609.12039Sep 11
现实才是最终验证者：智能体软件工程的两大关键缺口
提出双缺口框架，统一智能体软件工程中的奖励投机与幻觉
arXiv
2609.12039
Published
Sep 11
Layer
Application layer
Setting
coding agent
- RiskReward hacking
Abstract双缺口解释内环为何会放行部署后不可接受的实现，外环用现实证据持续修订 R、M 与 E。
Reality Is the Final Verifier 用 two-gap framework 解释智能体软件工程的主要失败，并主张用外层保证环持续收窄缺口，而不是试图在部署前证明缺口已关闭。
Paper analysisRead analysis
Risky behaviorarXiv 2608.10218Aug 11
研究者用进化算法构造多智能体 LLM 系统中的自我传播“思维病毒”
用演化算法构造 mind virus 并测量其在多智能体中的传播
arXiv
2608.10218
Published
Aug 11
Layer
Application layer
Setting
Multi-agent systems, coding agent
Evaluated models
Claude Haiku 4.5, Claude Sonnet 4.6, Gemini 3 Flash and others7
- RiskMisalignment and value divergence
- ComponentMemory
Paper analysisRead analysis
AttacksarXiv 2607.12340Jul 14
Research: LLM agent skill-name hallucination rates reach up to 62%, which can be exploited via squatting for supply-chain attacks
提出技能名称抢注攻击，利用 Agent 虚构技能名实施供应链劫持
arXiv
2607.12340
Published
Jul 14
Layer
Application layer
Setting
coding agent
Evaluated models
GLM-4.5-Air, GPT-5.4-mini, Claude Sonnet 4.6 and others12
- AttacksPoisoning and backdoors
- RiskHallucination and fabrication
- TechniqueSelection manipulation
- ComponentMCP and skills
Abstract研究证实智能体虚构技能名具有高发生率与可预测性，现有防御难以兼顾安全与效用，亟需注册表结构性防护。
本研究系统揭示了大模型智能体技能推荐场景下的虚构技能名称安全风险，将其形式化为一种新型供应链抢注攻击。
Paper analysisRead analysis
Evaluation and benchmarksarXiv 2606.18021Jun 16
LegalHalluLens：面向法律 AI 的分类幻觉审计与校准多智能体辩论
提出 LegalHalluLens，分类审计法律抽取幻觉并校准多智能体辩论
arXiv
2606.18021
Published
Jun 16
Layer
Application layer
Setting
LLM applications, Multi-agent systems
Evaluated models
gemini-3-flash, gpt-5.2, qwen3-32b and others5
- RiskHallucination and fabrication
Abstract作者称类型画像与 RDI 能区分聚合率相近的系统，辩论主要减少虚构检出。
LegalHalluLens 是一套法律条款抽取的审计框架，外加一个按审计结果校准的辩论缓解。。
Paper analysisRead analysis
Risky behaviorarXiv 2606.01637Jun 1
研究：LLM 在多智能体系统中更易被同伴共识误导而非纠正
比较多智能体同伴共识下有害修订与有益修订的强弱
arXiv
2606.01637
Published
Jun 1
Layer
Application layer
Setting
Multi-agent systems
Evaluated models
Qwen2.5-7B-Instruct, Mistral-7B-Instruct-v0.3, Gemma-2-9B-Instruct and others4
- RiskMisalignment and value divergence
Abstract全错比全对更能改写答案。
这篇工作在四个开源指令模型上测量从众修订的方向：错改对是有益修订，对改错是有害修订。模型先独立答选择题，再看到六个模拟同伴后重答。同伴可以全体一致、意见分裂，也可以带权威角色标签。
Paper analysisRead analysis
Evaluation and benchmarksarXiv 2605.30322May 29
Gram：面向破坏倾向的自动化对齐审计框架
提出 Gram 审计框架，测量编程与研究智能体的破坏倾向
arXiv
2605.30322
Published
May 29
Layer
Application layer
Setting
coding agent, science agent
Evaluated models
Gemini 2.5 Pro, Gemini 3.0 Pro Preview, Gemini 3.1 Pro Preview
- DefensesMonitoring and auditing
- RiskDeception and scheming
- ComponentMonitors
Paper analysisRead analysis
Evaluation and benchmarksarXiv 2605.17062May 17
研究复现五个前沿模型的包名幻觉：跨模型共同幻觉 127 个包名，53 个仍可被注册
复评前沿模型代码生成中的包名幻觉与抢注攻击面
arXiv
2605.17062
Published
May 17
Layer
Model layer
Setting
coding agent
Evaluated models
Claude Sonnet 4.6, Claude Haiku 4.5, GPT-5.4-mini and others5
- RiskHallucination and fabrication
Abstract评估显示前沿模型包幻觉率收窄但威胁仍在，作者识别出跨模型通用攻击面并呼吁联合防御。
Paper analysisRead analysis
Risky behaviorarXiv 2603.04904Mar 5
Alignment backlash: safety interventions reverse language-dependently across 16 languages in LLM multi-agent systems
展示对齐前缀在多智能体中按语言反转集体病理
arXiv
2603.04904
Published
Mar 5
Layer
Application layer
Setting
Multi-agent systems
Evaluated models
Llama 3.3 70B, GPT-4o-mini, Qwen3-Next-80B-A3B
- RiskMisalignment and value divergence
Paper analysisRead analysis
DefensesarXiv 2603.04474Mar 4
From Spark to Wildfire: Modeling and Mitigating Error Cascades in LLM-Based Multi-Agent Collaboration
Proposing a lineage-graph governance layer to suppress error cascades in multi-agent collaboration
arXiv
2603.04474
Published
Mar 4
Layer
Application layer
Setting
Multi-agent systems
- DefensesDetection and filtering
- AttacksPrompt injection
- RiskHallucination and fabrication
- ComponentGuardrails and judges
AbstractA single seed can solidify into a false consensus along a fixed collaboration graph.
The authors turn "small errors becoming collective false consensus" in LLM-MAS into a measurable propagation on a graph, and use a plug-in that only changes messages, not topology, to reduce the final infection.
Paper analysisRead analysis
Analysis and theoryarXiv 2602.09877Feb 10
The Devil Behind Moltbook: Safety Alignment Continuously Disappears in Self-Evolving AI Societies
Arguing that safety alignment continuously disappears in self-evolving AI societies
arXiv
2602.09877
Published
Feb 10
Layer
Application layer
Setting
Multi-agent systems
Evaluated models
Qwen3-8B
- RiskMisalignment and value divergence
AbstractThe authors claim that closed self-evolution makes safety alignment unsustainable.
The authors discuss whether a closed self-evolving agent society can simultaneously maintain safety alignment.
Paper analysisRead analysis

You have reached the end

AI Safety Daily

Read the latest issueOne issue per day, updated as events develop