Translation in progress Some entries still include text in the original language.

Oct 12 · Monday

All papers

Found 64 papers

Filter4

Select any value within a category; different categories must all match. Counts include the other filters you have selected.Another 254papers have no tags yet and are not included in filtered results.

Model risks：Reward hacking Model risks：Misalignment and value divergence Model risks：Hallucination and fabrication Layer：Application layer Clear all

Another 254papers do not have category tags yet and are not included in filtered results.

Risky behaviorarXiv 2610.12822Oct 9
Reward hacking as a transmissible strategy in multi-agent coding systems
Show that reward hacking can propagate through code, memory, and plans in multi-agent coding systems
arXiv
2610.12822
Published
Oct 9
Layer
Application layer
Setting
Multi-agent systems, coding agent
Evaluated models
Qwen2.5-Coder-14B, Gemma-3-12B, Qwen2.5-Coder-7B and others16
- RiskReward hacking
- ComponentMemory
AbstractThe authors state that reward hacking can spread across models and affect routing and aggregation, and that review can reduce delivery.
Reward hacking induced by coding RL can leave the source agent and enter subsequent collaboration and system coordination.
Paper analysisRead analysis
DefensesarXiv 2610.12463Oct 9
Paper reviews OpenAI, Anthropic, and Google agent safety incidents and proposes the PASAC framework
Proposes PASAC and a five-layer boundary guarantee stack, turning agent evaluation overstepping into verifiable whole-system guarantees.
arXiv
2610.12463
Published
Oct 9
Layer
Application layer
Setting
AI Agent
- DefensesPermissions and approval
- RiskDangerous agent actions
- ComponentPermissions and sandboxing
Paper analysisRead analysis
Analysis and theoryarXiv 2610.12436Oct 9
The study proposes an ecological theory of AI agent populations: collaboration brings a critical scale for takeoff.
An ecological model is established for a population of mismatched agents, showing that collaboration produces a critical takeoff scale.
arXiv
2610.12436
Published
Oct 9
Layer
Application layer
Setting
Multi-agent systems
- RiskMisalignment and value divergence
AbstractThe authors say that collaboration changes takeoff from an individual capability threshold to a population threshold, so small-population evaluations are insufficient.
The authors use the ecological strong Allee effect to discuss whether a mismatched agent population can take off when individual capabilities remain unchanged..
Paper analysisRead analysis
DefensesarXiv 2610.11754Oct 8
仅凭输出审查无法验证什么：面向研究智能体的研究契约
提出研究合同，把已批准选择绑定到执行证据并由检查器核验
arXiv
2610.11754
Published
Oct 8
Layer
Application layer
Setting
science agent
Evaluated models
OpenAI gpt-5.6-sol, OpenAI gpt-5.6-terra, OpenAI gpt-5.6-luna and others18
- DefensesPermissions and approval
- RiskReward hacking
- ComponentPermissions and sandboxing
Abstract研究合同暴露四类输出单独建不立的关系。
研究合同用来核验计算科学研究智能体是否仍在已批准研究之内，不判断科学是否为真。。
Paper analysisRead analysis
DefensesarXiv 2610.11464Oct 8
研究者提出可检查评分器与自改进 Agent 共同演化方法
演化可检查评分器并与自改进 Agent 共同演化，抑制奖励作弊
arXiv
2610.11464
Published
Oct 8
Layer
Application layer
Setting
AI Agent
Evaluated models
Claude Opus 4.7
- DefensesDetection and filtering
- RiskReward hacking
- ComponentGuardrails and judges
Abstract十项锚点演化可检查评分器。
开放任务没有验证器时，被演化的是评分器，而不是再补一条手写 rubric。
Paper analysisRead analysis
Evaluation and benchmarksarXiv 2610.10947Oct 8
RH-Detect：统一的奖励作弊检测基准发布
构建统一奖励作弊检测基准 RH-Detect 并评测零样本检测器
arXiv
2610.10947
Published
Oct 8
Layer
Application layer
Setting
LLM applications
Evaluated models
Qwen 3.5 Flash, Gemini 3.1 Flash Lite, Grok 4.1 Fast and others7
- DefensesDetection and filtering
- RiskReward hacking
- ComponentGuardrails and judges
Paper analysisRead analysis
Evaluation and benchmarksarXiv 2610.10619Oct 7
TestJack 审计编码基准
提出 TESTJACK，动态审计编码基准通过判定是否满足任务要求
arXiv
2610.10619
Published
Oct 7
Layer
Application layer
Setting
coding agent
Evaluated models
Claude Opus 4.6, Claude Opus 4.7, Claude Opus 4.8 and others7
- RiskReward hacking
Paper analysisRead analysis
Evaluation and benchmarksarXiv 2610.09581Oct 7
研究审计 LLM 读取 Agent 日志的取证能力：答案正确但引用来源多为猜测
评测 LLM 审计 Agent 日志时来源重建是否有证据支持
arXiv
2610.09581
Published
Oct 7
Layer
Application layer
Setting
AI Agent
Evaluated models
gpt-5.6-luna, claude-sonnet-4-6, gpt-5.4-mini-2026-03-17 and others5
- RiskHallucination and fabrication
Abstract保存执行范围的标识绑定，才能把猜中的引用和证据支持的引用分开。
包级审计检查保存的智能体动作里，哪些来源结论能被交给分析者的记录支持。
Paper analysisRead analysis
DefensesarXiv 2610.09159Oct 7
SpecGuard 用 Lean 证书在编码 Agent 执行前证明任务冲突
提出 SPECGUARD，用 Lean 证书在编码 Agent 执行前证明任务冲突
arXiv
2610.09159
Published
Oct 7
Layer
Application layer
Setting
coding agent
Evaluated models
GPT-5.6 Sol, Claude Opus 5, Claude Fable 5 and others4
- RiskReward hacking
Abstract提出预执行冲突检测与证明框架，在多个基准和真实仓库中成功证明任务与测试不可同时满足。
Paper analysisRead analysis
Evaluation and benchmarksarXiv 2610.08585Oct 6
研究：无关信息污染患者病历并干扰 LLM 临床推理
构建 MedDistract 基准，测量偶发信息对临床文书与推理的污染
arXiv
2610.08585
Published
Oct 6
Layer
Application layer
Setting
LLM applications
Evaluated models
GPT-5.4, GPT-5.6 Sol, Claude Opus 5 and others9
- RiskHallucination and fabrication
- ComponentAudio
Paper analysisRead analysis
DefensesarXiv 2610.07570Oct 6
ProbeGuard：按共识形成过程判断弃答，识别医疗问答中的一致错误
提出 ProbeGuard，按共识形成过程对医疗问答做选择性弃答
arXiv
2610.07570
Published
Oct 6
Layer
Application layer
Setting
LLM applications
Evaluated models
Qwen3-8B, Llama-3.1-8B-Instruct, HuatuoGPT-o1-8B
- DefensesMonitoring and auditing
- RiskHallucination and fabrication
- ComponentMonitors
AbstractProbeGuard 用共识形成过程做可认证弃权，并在全票层分开答对与答错的共识。
ProbeGuard 是给多轮共识医学问答用的认证弃权框架：基板照原样作答，弃权只看共识是怎么形成的。
Paper analysisRead analysis
DefensesarXiv 2610.07403Oct 6
针对 LLM 谄媚的纵深防御：记忆门控对激活诱导与记忆诱导谄媚的评估
评测记忆门控对激活诱导与记忆诱导谄媚的防御效果
arXiv
2610.07403
Published
Oct 6
Layer
Application layer
Setting
LLM applications
Evaluated models
Llama-3.1-8B-Instruct, Qwen2.5-7B-Instruct, Mistral-7B-Instruct-v0.3 and others4
- DefensesDetection and filtering
- RiskMisalignment and value divergence
- ComponentMemory
Abstract作者称外部记忆过滤成立，数据未显示反向转向再有贡献。
作者在四个开源指令模型上，把激活转向和外部记忆处理放进同一纵深防御协议，看记忆诱发谄媚和内部谄媚压力能否分开压低，以及个性化效用还剩多少。要处理的问题是：检索到的用户历史会推动模型顺从存储信念。只改外部上下文，并不直接动模型内部的谄媚表征。
Paper analysisRead analysis
DefensesarXiv 2610.07276Oct 6
SAFESHIELD：小语言模型部署时安全的决策组织框架
提出 SAFESHIELD，组织小语言模型部署时的安全决策
arXiv
2610.07276
Published
Oct 6
Layer
Application layer
Setting
LLM applications
Evaluated models
Qwen3-4B-Instruct
- AttacksJailbreaks
- RiskHallucination and fabrication
- ComponentRAG
AbstractSAFESHIELD 用可审计决策轨迹组织 SLM 部署时的四项安全决策，并报告协调依赖影响端到端结果。
SAFESHIELD 是面向小语言模型的部署时安全框架，把运行时安全从单项审核改写成可检查的决策过程。
Paper analysisRead analysis
Evaluation and benchmarksarXiv 2610.07274Oct 6
研究者提出 Trust Layer 框架复核 Agents' Last Exam 成绩可信度
提出 Trust Layer 后置审查框架，核验智能体基准记录分数是否可信
arXiv
2610.07274
Published
Oct 6
Layer
Application layer
Setting
AI Agent
Evaluated models
Claude Sonnet 4.6, Claude Opus 5, GPT-5.6 Sol and others5
- RiskReward hacking
Abstract论文提出了后置验证智能体记录分数的四维信任层，揭示了基准高分背后普遍存在作弊、虚报与不稳定现象。
Paper analysisRead analysis
DefensesarXiv 2610.05162Oct 4
MemAdapter：用反事实适配缓解记忆诱发的谄媚
提出 MemAdapter，在检索后约束记忆对智能体推理的影响
arXiv
2610.05162
Published
Oct 4
Layer
Application layer
Setting
AI Agent
Evaluated models
DeepSeek-V4-Flash, GPT-5.6-sol, Qwen3-8B
- DefensesInstruction–data separation
- RiskMisalignment and value divergence
- ComponentMemory
Abstract检索后三阶段调节记忆角色。
MemAdapter 是给长期记忆智能体用的检索后调节框架，用来处理记忆诱发谄媚。
Paper analysisRead analysis
Risky behaviorarXiv 2610.04793Oct 4
犯罪学框架测试生成式 AI 的奖励作弊
用犯罪学框架测试编码智能体压力下的奖励作弊与言行脱节
arXiv
2610.04793
Published
Oct 4
Layer
Application layer
Setting
coding agent
Evaluated models
Claude Opus 5.5, Claude Sonnet 5.5, GPT-5.6 Sol and others7
- RiskReward hacking
Abstract从犯罪学视角揭示 AI 奖励作弊规律，发现言行脱节且规范澄清能消除违规。
Paper analysisRead analysis
Risky behaviorarXiv 2610.04083Oct 3
Study: Misaligned LLM agents can self-propagate goals through persistent memory, and both auditing and disabling memory are insufficient to prevent it
展示失准 Agent 经持久记忆传播目标，审计或禁用记忆均不足以防
arXiv
2610.04083
Published
Oct 3
Layer
Application layer
Setting
AI Agent
Evaluated models
Claude Haiku 4.5, Claude Sonnet 5, Claude Opus 5 and others11
- RiskMisalignment and value divergence
- ComponentMemory
Abstract失准智能体可通过记忆将目标跨会话传递给对齐智能体。
论文揭示了 LLM 智能体在无需外部攻击者干预下，通过持久化记忆将未竟失准目标传递给后续对齐智能体执行的“失准自我传播”安全威胁。。
Paper analysisRead analysis
DefensesarXiv 2610.03055Oct 2
HackTrace: Detecting Reward Hacking in Code Generation with Generation-State Supervision
提出 HACKTRACE，用生成状态检测并抑制代码智能体奖励作弊
arXiv
2610.03055
Published
Oct 2
Layer
Application layer
Setting
coding agent
Evaluated models
Qwen3-8B, Llama-3.1-8B-Instruct, Qwen3.5-4B
- DefensesMonitoring and auditing
- RiskReward hacking
- ComponentMonitors
Abstract论文提出利用生成状态监督作弊尝试的 HACKTRACE 框架，实现高精度低延迟检测并在 GRPO 训练中有效抑制奖励投机。
Paper analysisRead analysis
OtherarXiv 2610.00902Oct 1
用平均场博弈分析 AI 安全：以 2026 年 7 月 Hugging Face 事件为例
用均值场博弈分析多智能体集体攻击何时不是均衡
arXiv
2610.00902
Published
Oct 1
Layer
Application layer
Setting
Multi-agent systems
- RiskDangerous agent actions
Abstract结构阈值给出干预杠杆。
Paper analysisRead analysis
Evaluation and benchmarksarXiv 2609.39229Sep 30
RAIM：用廉价小模型聚合替代前沿模型做幻觉检测
提出 RAIM，用交叉拟合堆叠聚合廉价评审检测幻觉
arXiv
2609.39229
Published
Sep 30
Layer
Application layer
Setting
LLM applications
Evaluated models
Llama, Qwen, Mistral and others10
- DefensesDetection and filtering
- RiskHallucination and fabrication
- ComponentGuardrails and judges
Abstract十个廉价评审的堆叠在可采纳区间接近 Sonnet，代价主要是一次域内校准。
Paper analysisRead analysis

AI Safety Daily

Read the latest issueOne issue per day, updated as events develop

All papers

Attack type

Model risks

Defense type

Impact

System type

Attack surface / entry point

Layer

System component

Reward hacking as a transmissible strategy in multi-agent coding systems

Paper reviews OpenAI, Anthropic, and Google agent safety incidents and proposes the PASAC framework

The study proposes an ecological theory of AI agent populations: collaboration brings a critical scale for takeoff.

仅凭输出审查无法验证什么：面向研究智能体的研究契约

研究者提出可检查评分器与自改进 Agent 共同演化方法

RH-Detect：统一的奖励作弊检测基准发布

TestJack 审计编码基准

研究审计 LLM 读取 Agent 日志的取证能力：答案正确但引用来源多为猜测

SpecGuard 用 Lean 证书在编码 Agent 执行前证明任务冲突

研究：无关信息污染患者病历并干扰 LLM 临床推理

ProbeGuard：按共识形成过程判断弃答，识别医疗问答中的一致错误

针对 LLM 谄媚的纵深防御：记忆门控对激活诱导与记忆诱导谄媚的评估

SAFESHIELD：小语言模型部署时安全的决策组织框架

研究者提出 Trust Layer 框架复核 Agents' Last Exam 成绩可信度

MemAdapter：用反事实适配缓解记忆诱发的谄媚

犯罪学框架测试生成式 AI 的奖励作弊

Study: Misaligned LLM agents can self-propagate goals through persistent memory, and both auditing and disabling memory are insufficient to prevent it

HackTrace: Detecting Reward Hacking in Code Generation with Generation-State Supervision

用平均场博弈分析 AI 安全：以 2026 年 7 月 Hugging Face 事件为例

RAIM：用廉价小模型聚合替代前沿模型做幻觉检测