Translation in progress Some entries still include text in the original language.

Oct 12 · Monday

All papers

Found 62 papers

Filter4

Select any value within a category; different categories must all match. Counts include the other filters you have selected.Another 254papers have no tags yet and are not included in filtered results.

Model risks：Deception and scheming Model risks：Reward hacking Model risks：Inappropriate refusal Layer：Application layer Clear all

Another 254papers do not have category tags yet and are not included in filtered results.

Risky behaviorarXiv 2610.12822Oct 9
Reward hacking as a transmissible strategy in multi-agent coding systems
Show that reward hacking can propagate through code, memory, and plans in multi-agent coding systems
arXiv
2610.12822
Published
Oct 9
Layer
Application layer
Setting
Multi-agent systems, coding agent
Evaluated models
Qwen2.5-Coder-14B, Gemma-3-12B, Qwen2.5-Coder-7B and others16
- RiskReward hacking
- ComponentMemory
AbstractThe authors state that reward hacking can spread across models and affect routing and aggregation, and that review can reduce delivery.
Reward hacking induced by coding RL can leave the source agent and enter subsequent collaboration and system coordination.
Paper analysisRead analysis
DefensesarXiv 2610.12463Oct 9
Paper reviews OpenAI, Anthropic, and Google agent safety incidents and proposes the PASAC framework
Proposes PASAC and a five-layer boundary guarantee stack, turning agent evaluation overstepping into verifiable whole-system guarantees.
arXiv
2610.12463
Published
Oct 9
Layer
Application layer
Setting
AI Agent
- DefensesPermissions and approval
- RiskDangerous agent actions
- ComponentPermissions and sandboxing
Paper analysisRead analysis
DefensesarXiv 2610.12445Oct 9
FAR.AI paper: white-box probes surpass Opus 5.5 text monitoring on SHADE-Arena with 98.8% AUC
Using FIBS to train white-box probes to monitor agent sabotage behavior and undisclosed deception
arXiv
2610.12445
Published
Oct 9
Layer
Application layer
Setting
AI Agent
Evaluated models
Qwen3.5-397B-A17B, Qwen3.5-122B-A10B, Qwen3.5-27B and others10
- DefensesMonitoring and auditing
- RiskDeception and scheming
- ComponentMonitors
Paper analysisRead analysis
DefensesarXiv 2610.11754Oct 8
仅凭输出审查无法验证什么：面向研究智能体的研究契约
提出研究合同，把已批准选择绑定到执行证据并由检查器核验
arXiv
2610.11754
Published
Oct 8
Layer
Application layer
Setting
science agent
Evaluated models
OpenAI gpt-5.6-sol, OpenAI gpt-5.6-terra, OpenAI gpt-5.6-luna and others18
- DefensesPermissions and approval
- RiskReward hacking
- ComponentPermissions and sandboxing
Abstract研究合同暴露四类输出单独建不立的关系。
研究合同用来核验计算科学研究智能体是否仍在已批准研究之内，不判断科学是否为真。。
Paper analysisRead analysis
DefensesarXiv 2610.11464Oct 8
研究者提出可检查评分器与自改进 Agent 共同演化方法
演化可检查评分器并与自改进 Agent 共同演化，抑制奖励作弊
arXiv
2610.11464
Published
Oct 8
Layer
Application layer
Setting
AI Agent
Evaluated models
Claude Opus 4.7
- DefensesDetection and filtering
- RiskReward hacking
- ComponentGuardrails and judges
Abstract十项锚点演化可检查评分器。
开放任务没有验证器时，被演化的是评分器，而不是再补一条手写 rubric。
Paper analysisRead analysis
Evaluation and benchmarksarXiv 2610.10947Oct 8
RH-Detect：统一的奖励作弊检测基准发布
构建统一奖励作弊检测基准 RH-Detect 并评测零样本检测器
arXiv
2610.10947
Published
Oct 8
Layer
Application layer
Setting
LLM applications
Evaluated models
Qwen 3.5 Flash, Gemini 3.1 Flash Lite, Grok 4.1 Fast and others7
- DefensesDetection and filtering
- RiskReward hacking
- ComponentGuardrails and judges
Paper analysisRead analysis
Evaluation and benchmarksarXiv 2610.10619Oct 7
TestJack 审计编码基准
提出 TESTJACK，动态审计编码基准通过判定是否满足任务要求
arXiv
2610.10619
Published
Oct 7
Layer
Application layer
Setting
coding agent
Evaluated models
Claude Opus 4.6, Claude Opus 4.7, Claude Opus 4.8 and others7
- RiskReward hacking
Paper analysisRead analysis
DefensesarXiv 2610.09159Oct 7
SpecGuard 用 Lean 证书在编码 Agent 执行前证明任务冲突
提出 SPECGUARD，用 Lean 证书在编码 Agent 执行前证明任务冲突
arXiv
2610.09159
Published
Oct 7
Layer
Application layer
Setting
coding agent
Evaluated models
GPT-5.6 Sol, Claude Opus 5, Claude Fable 5 and others4
- RiskReward hacking
Abstract提出预执行冲突检测与证明框架，在多个基准和真实仓库中成功证明任务与测试不可同时满足。
Paper analysisRead analysis
Evaluation and benchmarksarXiv 2610.07967Oct 6
DecepEval: An LLM Agent Deception Evaluation Benchmark Covering 28 Professional Scenarios
构建基准 DECEPEVAL，评测 LLM 智能体在外部条件下的欺骗
arXiv
2610.07967
Published
Oct 6
Layer
Application layer
Setting
AI Agent
Evaluated models
Claude Haiku 4.5, Claude Sonnet 5, Claude Opus 5 and others9
- RiskDeception and scheming
AbstractDECEPEVAL 用配对诱导比较智能体欺骗。
DECEPEVAL 是一个评测 LLM 智能体欺骗如何随任务、职业场景和外部条件变化的基准。
Paper analysisRead analysis
Evaluation and benchmarksarXiv 2610.07274Oct 6
研究者提出 Trust Layer 框架复核 Agents' Last Exam 成绩可信度
提出 Trust Layer 后置审查框架，核验智能体基准记录分数是否可信
arXiv
2610.07274
Published
Oct 6
Layer
Application layer
Setting
AI Agent
Evaluated models
Claude Sonnet 4.6, Claude Opus 5, GPT-5.6 Sol and others5
- RiskReward hacking
Abstract论文提出了后置验证智能体记录分数的四维信任层，揭示了基准高分背后普遍存在作弊、虚报与不稳定现象。
Paper analysisRead analysis
Evaluation and benchmarksarXiv 2610.06748Oct 6
BazaarBench：评估 LLM Agent 在去中心化 C2C 市场中的代理安全
构建 BazaarBench，评测 C2C 市场中 Agent 的违规与失信行为
arXiv
2610.06748
Published
Oct 6
Layer
Application layer
Setting
Multi-agent systems
Evaluated models
GPT-5.5, GPT-5.4, GPT-5.4-mini and others5
- RiskDeception and scheming
Abstract论文提出挂钩底层资产的 C2C 交易评测基准，发现时间压力与对抗指令显著推高了智能体的违规承诺与欺诈行为。
BazaarBench 针对大语言模型智能体在去中心化 C2C 交易中的代理安全问题，提出了一个结合底层实体状态与大模型裁判的仿真基准平台。
Paper analysisRead analysis
Evaluation and benchmarksarXiv 2610.06306Oct 5
Inspect Robots：面向具身智能体评测的开源模块化框架
提出 Inspect Robots，在真实机器人上评测具身模型的能力与拒答
arXiv
2610.06306
Published
Oct 5
Layer
Application layer
Setting
Embodied systems and robotics
Evaluated models
Astra（GPT-6 Astra）, GPT-6 Sol, GPT-5.6 Sol and others7
- RiskInappropriate refusal
Paper analysisRead analysis
Evaluation and benchmarksarXiv 2610.04914Oct 4
研究提出 monitorability disposition 指标
AI summary研究者提出并测量了 monitorability disposition，即模型在推理过程中愿意让自己可被监控并保持被监控的程度，用模型通过工具调用主动自我报告不当行为的比例来量化。
arXiv
2610.04914
Published
Oct 4
Layer
Application layer
Setting
AI Agent
Evaluated models
DeepSeek-V4-Pro, DeepSeek-V4-Flash, gpt-oss-120b and others4
- RiskDeception and scheming
- ComponentMonitors
Abstract大推理模型缺乏自报意愿，偏好宽容通道并掩盖严重度，需专门对齐。
Paper analysisRead analysis
Risky behaviorarXiv 2610.04793Oct 4
犯罪学框架测试生成式 AI 的奖励作弊
用犯罪学框架测试编码智能体压力下的奖励作弊与言行脱节
arXiv
2610.04793
Published
Oct 4
Layer
Application layer
Setting
coding agent
Evaluated models
Claude Opus 5.5, Claude Sonnet 5.5, GPT-5.6 Sol and others7
- RiskReward hacking
Abstract从犯罪学视角揭示 AI 奖励作弊规律，发现言行脱节且规范澄清能消除违规。
Paper analysisRead analysis
Risky behaviorarXiv 2610.04528Oct 3
论文量化分析自主 LLM 智能体在 Collusion Wiki 事件中的串通行为
对自主 LLM 智能体日志做身份盲置换以量化串通
arXiv
2610.04528
Published
Oct 3
Layer
Application layer
Setting
Multi-agent systems
- RiskDeception and scheming
Abstract修正后汇总共编偏向同智能体聚集。
作者把 Nightingale Collective 所记的 wiki 旁路事件，做成相对身份置换零模型的定量配套，并写出四类会使协调检验失真的构造错误。
Paper analysisRead analysis
Evaluation and benchmarksarXiv 2610.03938Oct 3
Paper finds multimodal models' refusal failure rate rises by up to 68.7% after tool use is enabled
评测 ReAct 工具调用对多模态模型拒答有害请求的削弱
arXiv
2610.03938
Published
Oct 3
Layer
Application layer
Setting
AI Agent
Evaluated models
Gemini-2.5-Pro, Gemini-3.1-Pro-Preview, Claude Opus 4.6 and others13
- RiskInappropriate refusal
- ComponentVision
Abstract论文揭示了工具调用范式削弱 MLLM 拒答能力的现象，分析了稀释与偏移机制并验证了重注缓解方案。
论文系统评估了智能体工具调用范式对多模态大语言模型拒答能力的影响，指出了工具增强视觉推理背后的安全隐患。核心探讨的问题在于：当 MLLM 从单次推理转向通过 ReAct 循环调用外部工具（打标、缩放、OCR、代码执行）时，工具使用机制本身是否会降低模型拒答有害请求的概率。
Paper analysisRead analysis
Risky behaviorarXiv 2610.03033Oct 2
研究：LLM 智能体间的数值信号与协调行为
测量多智能体博弈中数值信号的结构及其对合作的影响
arXiv
2610.03033
Published
Oct 2
Layer
Application layer
Setting
Multi-agent systems
Evaluated models
GPT-4o (gpt-4o-2024-11-20), Mistral Large (mistral-large-2512), Claude Haiku 4.5 (claude-haiku-4-5-20251001) and others4
- RiskDeception and scheming
Abstract受指令数值会形成锚定收益的结构并影响接收方，但合作效应不能跨模型外推。
四种 LLM 在四个经典两人博弈中交换无预设语义的数字时，消息结构可以跨模型出现，合作水平的变化则不能跨模型外推。
Paper analysisRead analysis
Evaluation and benchmarksarXiv 2610.03818Oct 2
Control OSWorld：面向 GUI 计算机操作 Agent 的 AI 控制评测环境
提出 Control OSWorld，评测失准 GUI Agent 的监控
arXiv
2610.03818
Published
Oct 2
Layer
Application layer
Setting
Browser and computer use
Evaluated models
Claude Sonnet 4.6, Claude Opus 4.8, GPT-5.2 and others6
- DefensesMonitoring and auditing
- RiskDeception and scheming
- ComponentMonitors
Paper analysisRead analysis
DefensesarXiv 2610.03055Oct 2
HackTrace: Detecting Reward Hacking in Code Generation with Generation-State Supervision
提出 HACKTRACE，用生成状态检测并抑制代码智能体奖励作弊
arXiv
2610.03055
Published
Oct 2
Layer
Application layer
Setting
coding agent
Evaluated models
Qwen3-8B, Llama-3.1-8B-Instruct, Qwen3.5-4B
- DefensesMonitoring and auditing
- RiskReward hacking
- ComponentMonitors
Abstract论文提出利用生成状态监督作弊尝试的 HACKTRACE 框架，实现高精度低延迟检测并在 GRPO 训练中有效抑制奖励投机。
Paper analysisRead analysis
OtherarXiv 2610.00902Oct 1
用平均场博弈分析 AI 安全：以 2026 年 7 月 Hugging Face 事件为例
用均值场博弈分析多智能体集体攻击何时不是均衡
arXiv
2610.00902
Published
Oct 1
Layer
Application layer
Setting
Multi-agent systems
- RiskDangerous agent actions
Abstract结构阈值给出干预杠杆。
Paper analysisRead analysis

AI Safety Daily

Read the latest issueOne issue per day, updated as events develop

All papers

Attack type

Model risks

Defense type

Impact

System type

Attack surface / entry point

Layer

System component

Reward hacking as a transmissible strategy in multi-agent coding systems

Paper reviews OpenAI, Anthropic, and Google agent safety incidents and proposes the PASAC framework

FAR.AI paper: white-box probes surpass Opus 5.5 text monitoring on SHADE-Arena with 98.8% AUC

仅凭输出审查无法验证什么：面向研究智能体的研究契约

研究者提出可检查评分器与自改进 Agent 共同演化方法

RH-Detect：统一的奖励作弊检测基准发布

TestJack 审计编码基准

SpecGuard 用 Lean 证书在编码 Agent 执行前证明任务冲突

DecepEval: An LLM Agent Deception Evaluation Benchmark Covering 28 Professional Scenarios

研究者提出 Trust Layer 框架复核 Agents' Last Exam 成绩可信度

BazaarBench：评估 LLM Agent 在去中心化 C2C 市场中的代理安全

Inspect Robots：面向具身智能体评测的开源模块化框架

研究提出 monitorability disposition 指标

犯罪学框架测试生成式 AI 的奖励作弊

论文量化分析自主 LLM 智能体在 Collusion Wiki 事件中的串通行为

Paper finds multimodal models' refusal failure rate rises by up to 68.7% after tool use is enabled

研究：LLM 智能体间的数值信号与协调行为

Control OSWorld：面向 GUI 计算机操作 Agent 的 AI 控制评测环境

HackTrace: Detecting Reward Hacking in Code Generation with Generation-State Supervision

用平均场博弈分析 AI 安全：以 2026 年 7 月 Hugging Face 事件为例