Translation in progress Some entries still include text in the original language.

Oct 12 · Monday

All papers

Found 169 papers

Filter3

System type

Models and APIs94 LLM applications12 AI Agent64 coding agent19 science agent8 Multi-agent systems14 Embodied systems and robotics4

Attack surface / entry point

User input12

Layer

Model layer52 Application layer64 Training and data layer51

System component

MCP and skills3 Memory11 RAG3 Guardrails and judges23 Monitors20 Reasoning chains12 Permissions and sandboxing10 Fine-tuning and open weights3 Vision12 Audio4 Video3 Multimodal16

Select any value within a category; different categories must all match. Counts include the other filters you have selected.Another 261papers have no tags yet and are not included in filtered results.

Model risks：Reward hacking Model risks：Misalignment and value divergence Model risks：Hallucination and fabrication Clear all

Another 261papers do not have category tags yet and are not included in filtered results.

Risky behaviorarXiv 2610.12822Oct 9
Reward hacking as a transmissible strategy in multi-agent coding systems
Show that reward hacking can propagate through code, memory, and plans in multi-agent coding systems
arXiv
2610.12822
Published
Oct 9
Layer
Application layer
Setting
Multi-agent systems, coding agent
Evaluated models
Qwen2.5-Coder-14B, Gemma-3-12B, Qwen2.5-Coder-7B and others16
- RiskReward hacking
- ComponentMemory
AbstractThe authors state that reward hacking can spread across models and affect routing and aggregation, and that review can reduce delivery.
Reward hacking induced by coding RL can leave the source agent and enter subsequent collaboration and system coordination.
Paper analysisRead analysis
DefensesarXiv 2610.12463Oct 9
Paper reviews OpenAI, Anthropic, and Google agent safety incidents and proposes the PASAC framework
Proposes PASAC and a five-layer boundary guarantee stack, turning agent evaluation overstepping into verifiable whole-system guarantees.
arXiv
2610.12463
Published
Oct 9
Layer
Application layer
Setting
AI Agent
- DefensesPermissions and approval
- RiskDangerous agent actions
- ComponentPermissions and sandboxing
Paper analysisRead analysis
Analysis and theoryarXiv 2610.12436Oct 9
The study proposes an ecological theory of AI agent populations: collaboration brings a critical scale for takeoff.
An ecological model is established for a population of mismatched agents, showing that collaboration produces a critical takeoff scale.
arXiv
2610.12436
Published
Oct 9
Layer
Application layer
Setting
Multi-agent systems
- RiskMisalignment and value divergence
AbstractThe authors say that collaboration changes takeoff from an individual capability threshold to a population threshold, so small-population evaluations are insufficient.
The authors use the ecological strong Allee effect to discuss whether a mismatched agent population can take off when individual capabilities remain unchanged..
Paper analysisRead analysis
Analysis and theoryarXiv 2610.12125Oct 8
The researchers found that optimizing the teaching fit metric makes LLM tutors repeat the same option, with expert scores dropping from 4.46 to 3.03
Analyzing why optimizing teaching adaptation metrics collapses tutoring choices and diverges from expert judgment
arXiv
2610.12125
Published
Oct 8
Layer
Training and data layer
Setting
Models and APIs
Evaluated models
Gemini 3.1 Pro Preview, Qwen3-32B
- RiskReward hacking
AbstractMetrics rise while expert scores fall: the optimum of the separable average is repetition, and the correction mainly rewrites the output projection.
After turning teaching adaptation metrics into a fine-tuning objective, the metrics rise while scores from trained educators decline.
Paper analysisRead analysis
DefensesarXiv 2610.11754Oct 8
仅凭输出审查无法验证什么：面向研究智能体的研究契约
提出研究合同，把已批准选择绑定到执行证据并由检查器核验
arXiv
2610.11754
Published
Oct 8
Layer
Application layer
Setting
science agent
Evaluated models
OpenAI gpt-5.6-sol, OpenAI gpt-5.6-terra, OpenAI gpt-5.6-luna and others18
- DefensesPermissions and approval
- RiskReward hacking
- ComponentPermissions and sandboxing
Abstract研究合同暴露四类输出单独建不立的关系。
研究合同用来核验计算科学研究智能体是否仍在已批准研究之内，不判断科学是否为真。。
Paper analysisRead analysis
DefensesarXiv 2610.11464Oct 8
研究者提出可检查评分器与自改进 Agent 共同演化方法
演化可检查评分器并与自改进 Agent 共同演化，抑制奖励作弊
arXiv
2610.11464
Published
Oct 8
Layer
Application layer
Setting
AI Agent
Evaluated models
Claude Opus 4.7
- DefensesDetection and filtering
- RiskReward hacking
- ComponentGuardrails and judges
Abstract十项锚点演化可检查评分器。
开放任务没有验证器时，被演化的是评分器，而不是再补一条手写 rubric。
Paper analysisRead analysis
Evaluation and benchmarksarXiv 2610.11305Oct 8
BeliefScope：区分证据与用户压力对模型信念影响的受控框架
用 BeliefScope 分离固定命题上证据与压力对模型信念的影响
arXiv
2610.11305
Published
Oct 8
Layer
Model layer
Setting
Models and APIs
Evaluated models
Qwen3-8B, Llama3.1-8B, Gemma3-12B and others4
- RiskMisalignment and value divergence
AbstractBeliefScope 把证据修正与压力偏移拆成绑定评测条件的行为画像。
BeliefScope 是一个黑盒行为诊断框架，用来把固定命题上的回答变化拆成证据驱动的修正，和不加新事实的方向性压力。信念修正评测能看到信息基础变化后的更新，谄媚评测能看到用户影响下的顺从或漂移。作者认为，当两种输入都偏向同一结论时，表面偏移本身不能区分来源。
Paper analysisRead analysis
DefensesarXiv 2610.11281Oct 8
包络采样：用少量真值标注重校准 LLM 评委以缓解奖励作弊
提出包络采样，用少量真值标注重标定 LLM 评委以缓解奖励作弊
arXiv
2610.11281
Published
Oct 8
Layer
Training and data layer
Setting
Models and APIs
Evaluated models
Qwen3-4B, Qwen3.8-27B, Qwen2.5-14B-Instruct
- DefensesModel hardening
- RiskReward hacking
- ComponentGuardrails and judges
Abstract包络采样把真值标注打到代理尾部，同样预算下缓解 reward hacking。
作者研究后训练前如何选择真值标注的查询分布，使重标定后的廉价代理更适合被优化。真奖励昂贵时，实践用 LLM judge 代替人类。若失准只发生在基座很少生成的输出上，on-policy 标注修不到这些点，随后的 KL 正则优化会把质量集中到被高估的输出。
Paper analysisRead analysis
Evaluation and benchmarksarXiv 2610.10947Oct 8
RH-Detect：统一的奖励作弊检测基准发布
构建统一奖励作弊检测基准 RH-Detect 并评测零样本检测器
arXiv
2610.10947
Published
Oct 8
Layer
Application layer
Setting
LLM applications
Evaluated models
Qwen 3.5 Flash, Gemini 3.1 Flash Lite, Grok 4.1 Fast and others7
- DefensesDetection and filtering
- RiskReward hacking
- ComponentGuardrails and judges
Paper analysisRead analysis
Analysis and theoryarXiv 2610.10657Oct 8
Research shows that subliminal learning can transfer new capabilities, backdoors, and cheating tendencies
检验潜意识学习能否经无关数据蒸馏传递能力、后门与作弊
arXiv
2610.10657
Published
Oct 8
Layer
Training and data layer
Setting
Models and APIs
Evaluated models
Qwen3.6-35B-A3B, Qwen3.5-9B, Qwen3.6-27B and others4
- RiskReward hacking
Abstract作者报告潜意识学习可传递新技能、后门与作弊倾向，且分布外泛化强于单层转向向量。
Paper analysisRead analysis
Evaluation and benchmarksarXiv 2610.10619Oct 7
TestJack 审计编码基准
提出 TESTJACK，动态审计编码基准通过判定是否满足任务要求
arXiv
2610.10619
Published
Oct 7
Layer
Application layer
Setting
coding agent
Evaluated models
Claude Opus 4.6, Claude Opus 4.7, Claude Opus 4.8 and others7
- RiskReward hacking
Paper analysisRead analysis
Evaluation and benchmarksarXiv 2610.09581Oct 7
研究审计 LLM 读取 Agent 日志的取证能力：答案正确但引用来源多为猜测
评测 LLM 审计 Agent 日志时来源重建是否有证据支持
arXiv
2610.09581
Published
Oct 7
Layer
Application layer
Setting
AI Agent
Evaluated models
gpt-5.6-luna, claude-sonnet-4-6, gpt-5.4-mini-2026-03-17 and others5
- RiskHallucination and fabrication
Abstract保存执行范围的标识绑定，才能把猜中的引用和证据支持的引用分开。
包级审计检查保存的智能体动作里，哪些来源结论能被交给分析者的记录支持。
Paper analysisRead analysis
DefensesarXiv 2610.09496Oct 7
SOUL：用稀疏特征策略遗忘缓解 VLA 模型的状态幻觉
提出 SOUL 稀疏特征策略遗忘，缓解 VLA 状态幻觉
arXiv
2610.09496
Published
Oct 7
Layer
Training and data layer
Setting
Embodied systems and robotics
Evaluated models
OpenVLA, π0.5
- DefensesModel hardening
- RiskHallucination and fabrication
AbstractSOUL 以稀疏特征遗忘 VLA 状态幻觉。
SOUL用稀疏特征引导的策略 unlearning 处理 VLA 的 state hallucination。
Paper analysisRead analysis
InterpretabilityarXiv 2610.09384Oct 7
人格层级模型解释微调 LLM 的上下文泛化
提出人格层级模型并设计 PPR 抑制奖励作弊的跨上下文泛化
arXiv
2610.09384
Published
Oct 7
Layer
Model layer
Setting
Models and APIs
Evaluated models
Qwen3-4B, Llama-3.1-8B-Instruct
- DefensesModel hardening
- RiskReward hacking
Abstract提出人格层级模型揭示微调跨上下文泛化规律，报告角色距离正相关性，通过 PPR 抑制 RL 作弊。
Paper analysisRead analysis
DefensesarXiv 2610.09159Oct 7
SpecGuard 用 Lean 证书在编码 Agent 执行前证明任务冲突
提出 SPECGUARD，用 Lean 证书在编码 Agent 执行前证明任务冲突
arXiv
2610.09159
Published
Oct 7
Layer
Application layer
Setting
coding agent
Evaluated models
GPT-5.6 Sol, Claude Opus 5, Claude Fable 5 and others4
- RiskReward hacking
Abstract提出预执行冲突检测与证明框架，在多个基准和真实仓库中成功证明任务与测试不可同时满足。
Paper analysisRead analysis
DefensesarXiv 2610.08761Oct 7
VeriFine：通过扩展验证实现具身推理的自我改进
提出 VeriFine，用双环协同扩展验证以实现具身推理自我改进
arXiv
2610.08761
Published
Oct 7
Layer
Training and data layer
Setting
Embodied systems and robotics
Evaluated models
Alpamayo 1.5 8B, Qwen3-VL 2B, Claude Opus 5
- DefensesModel hardening
- RiskReward hacking
- ComponentGuardrails and judges
AbstractVeriFine 用双环协同演化具身推理的策略、课程和评审，驾驶与导航的策略分和评审对齐都上升。
VeriFine 把驾驶和导航推理的自改进写成策略、课程和无参考评审的协同演化，用来处理固定评审跟不上新失败模式的问题。
Paper analysisRead analysis
Risky behaviorarXiv 2610.08670Oct 7
Study: post-training recipes determine whether large models act against their own moral judgments
提出知行差距面板，测量压力下模型是否违背自身判断
arXiv
2610.08670
Published
Oct 7
Layer
Model layer
Setting
Models and APIs
Evaluated models
OLMo-3-7B-Instruct, Llama-3.1-8B-Instruct, Tulu 3 and others7
- RiskMisalignment and value divergence
Abstract评估发现后训练配方决定知行差距是否存在，行动前审思可有效缓解。
本文评估了开源语言模型作为智能体行动时违背自身道德判断的知行差距及其决定因素。
Paper analysisRead analysis
Evaluation and benchmarksarXiv 2610.08585Oct 6
研究：无关信息污染患者病历并干扰 LLM 临床推理
构建 MedDistract 基准，测量偶发信息对临床文书与推理的污染
arXiv
2610.08585
Published
Oct 6
Layer
Application layer
Setting
LLM applications
Evaluated models
GPT-5.4, GPT-5.6 Sol, Claude Opus 5 and others9
- RiskHallucination and fabrication
- ComponentAudio
Paper analysisRead analysis
Evaluation and benchmarksarXiv 2610.08540Oct 6
The paper proposes a scaling law framework for alignment measured by risk category.
提出分风险对齐标度律框架，测量各类风险的对齐负担如何随规模变化
arXiv
2610.08540
Published
Oct 6
Layer
Training and data layer
Setting
Models and APIs
Evaluated models
Pythia (14m–2.8b), Qwen2.5 (0.5B–72B), Qwen2.5-Instruct (0.5B–14B) and others4
- RiskMisalignment and value divergence
Abstract论文提出了分风险对齐标度律框架，测得真实性与倾向纠错呈标度有益，但盲后门仍存活，揭示隐藏风险的长期挑战。
本文提出了一个将大语言模型对齐难度形式化为可测量标度关系的理论框架与预注册评测协议。
Paper analysisRead analysis
DefensesarXiv 2610.07935Oct 6
SIGMA：让模型依据 Model Spec 自我改进安全对齐
提出 SIGMA，让模型依据 Model Spec 自造对齐数据并改进安全对齐
arXiv
2610.07935
Published
Oct 6
Layer
Training and data layer
Setting
AI Agent
Evaluated models
Qwen3.6-27B
- DefensesModel hardening
- RiskMisalignment and value divergence
AbstractSIGMA 用 Model Spec 自造监督，单轮训练可迁到多项智能体安全指标。
SIGMA 让 Qwen3.6-27B 只凭 Model Spec 和高层领域，自己生产对齐任务、评分细则和奖励。
Paper analysisRead analysis

AI Safety Daily

Read the latest issueOne issue per day, updated as events develop

All papers

Attack type

Model risks

Defense type

Impact

System type

Attack surface / entry point

Layer

System component

Reward hacking as a transmissible strategy in multi-agent coding systems

Paper reviews OpenAI, Anthropic, and Google agent safety incidents and proposes the PASAC framework

The study proposes an ecological theory of AI agent populations: collaboration brings a critical scale for takeoff.

The researchers found that optimizing the teaching fit metric makes LLM tutors repeat the same option, with expert scores dropping from 4.46 to 3.03

仅凭输出审查无法验证什么：面向研究智能体的研究契约

研究者提出可检查评分器与自改进 Agent 共同演化方法

BeliefScope：区分证据与用户压力对模型信念影响的受控框架

包络采样：用少量真值标注重校准 LLM 评委以缓解奖励作弊

RH-Detect：统一的奖励作弊检测基准发布

Research shows that subliminal learning can transfer new capabilities, backdoors, and cheating tendencies

TestJack 审计编码基准

研究审计 LLM 读取 Agent 日志的取证能力：答案正确但引用来源多为猜测

SOUL：用稀疏特征策略遗忘缓解 VLA 模型的状态幻觉

人格层级模型解释微调 LLM 的上下文泛化

SpecGuard 用 Lean 证书在编码 Agent 执行前证明任务冲突

VeriFine：通过扩展验证实现具身推理的自我改进

Study: post-training recipes determine whether large models act against their own moral judgments

研究：无关信息污染患者病历并干扰 LLM 临床推理

The paper proposes a scaling law framework for alignment measured by risk category.

SIGMA：让模型依据 Model Spec 自我改进安全对齐