Translation in progress Some entries still include text in the original language.

Oct 12 · Monday

All papers

Found 23 papers

Filter5

Model risks

Deception and scheming19 Reward hacking14 Misalignment and value divergence10 Dangerous agent actions22 Dangerous capabilities9 Hallucination and fabrication4

Impact

Oversight failure12 Increased misuse capability5

System type

LLM applications4 AI Agent48 coding agent19 science agent13 Multi-agent systems5

Layer

Application layer23

System component

Monitors3 Permissions and sandboxing3

Select any value within a category; different categories must all match. Counts include the other filters you have selected.Another 254papers have no tags yet and are not included in filtered results.

Model risks：Reward hacking Model risks：Dangerous capabilities System type：coding agent System type：Multi-agent systems Layer：Application layer Clear all

Another 254papers do not have category tags yet and are not included in filtered results.

Risky behaviorarXiv 2610.12822Oct 9
Reward hacking as a transmissible strategy in multi-agent coding systems
Show that reward hacking can propagate through code, memory, and plans in multi-agent coding systems
arXiv
2610.12822
Published
Oct 9
Layer
Application layer
Setting
Multi-agent systems, coding agent
Evaluated models
Qwen2.5-Coder-14B, Gemma-3-12B, Qwen2.5-Coder-7B and others16
- RiskReward hacking
- ComponentMemory
AbstractThe authors state that reward hacking can spread across models and affect routing and aggregation, and that review can reduce delivery.
Reward hacking induced by coding RL can leave the source agent and enter subsequent collaboration and system coordination.
Paper analysisRead analysis
Evaluation and benchmarksarXiv 2610.10619Oct 7
TestJack 审计编码基准
提出 TESTJACK，动态审计编码基准通过判定是否满足任务要求
arXiv
2610.10619
Published
Oct 7
Layer
Application layer
Setting
coding agent
Evaluated models
Claude Opus 4.6, Claude Opus 4.7, Claude Opus 4.8 and others7
- RiskReward hacking
Paper analysisRead analysis
DefensesarXiv 2610.09159Oct 7
SpecGuard 用 Lean 证书在编码 Agent 执行前证明任务冲突
提出 SPECGUARD，用 Lean 证书在编码 Agent 执行前证明任务冲突
arXiv
2610.09159
Published
Oct 7
Layer
Application layer
Setting
coding agent
Evaluated models
GPT-5.6 Sol, Claude Opus 5, Claude Fable 5 and others4
- RiskReward hacking
Abstract提出预执行冲突检测与证明框架，在多个基准和真实仓库中成功证明任务与测试不可同时满足。
Paper analysisRead analysis
Evaluation and benchmarksarXiv 2610.06584Oct 6
论文：AI 在非公开漏洞上更帮攻击者还是防御者
评测 coding agent 在非公开漏洞上的利用生成、修复与后续攻击
arXiv
2610.06584
Published
Oct 6
Layer
Application layer
Setting
coding agent
Evaluated models
Opus 5, GPT-5.6 Sol, GPT-6 Sol and others10
- RiskDangerous capabilities
Abstract攻防孰强随环境、系统和弱点改变。
作者比较编程智能体在利用生成与漏洞修复上的自主表现，并检查修复之后还能不能被攻破。文中的 help 指自主完成任务，不是提升人类操作者。
Paper analysisRead analysis
Risky behaviorarXiv 2610.04793Oct 4
犯罪学框架测试生成式 AI 的奖励作弊
用犯罪学框架测试编码智能体压力下的奖励作弊与言行脱节
arXiv
2610.04793
Published
Oct 4
Layer
Application layer
Setting
coding agent
Evaluated models
Claude Opus 5.5, Claude Sonnet 5.5, GPT-5.6 Sol and others7
- RiskReward hacking
Abstract从犯罪学视角揭示 AI 奖励作弊规律，发现言行脱节且规范澄清能消除违规。
Paper analysisRead analysis
DefensesarXiv 2610.03055Oct 2
HackTrace: Detecting Reward Hacking in Code Generation with Generation-State Supervision
提出 HACKTRACE，用生成状态检测并抑制代码智能体奖励作弊
arXiv
2610.03055
Published
Oct 2
Layer
Application layer
Setting
coding agent
Evaluated models
Qwen3-8B, Llama-3.1-8B-Instruct, Qwen3.5-4B
- DefensesMonitoring and auditing
- RiskReward hacking
- ComponentMonitors
Abstract论文提出利用生成状态监督作弊尝试的 HACKTRACE 框架，实现高精度低延迟检测并在 GRPO 训练中有效抑制奖励投机。
Paper analysisRead analysis
OtherarXiv 2610.00902Oct 1
用平均场博弈分析 AI 安全：以 2026 年 7 月 Hugging Face 事件为例
用均值场博弈分析多智能体集体攻击何时不是均衡
arXiv
2610.00902
Published
Oct 1
Layer
Application layer
Setting
Multi-agent systems
- RiskDangerous agent actions
Abstract结构阈值给出干预杠杆。
Paper analysisRead analysis
DefensesarXiv 2609.36170Sep 29
Assay：用内容寻址证据图约束 AI 编码智能体的声明失效
提出 Assay，用依赖锥哈希绑定代码断言并在合并门禁拦截失效声明
arXiv
2609.36170
Published
Sep 29
Layer
Application layer
Setting
coding agent
- DefensesPermissions and approval
- RiskDeception and scheming
- ComponentPermissions and sandboxing
AbstractAssay 提供随依赖内容失效的证据管理。
Assay 是一套面向 AI 辅助软件交付的内容寻址证据系统，关注代码改变后，测试通过、无秘密或行为保持等断言还能否用于批准合并。
Paper analysisRead analysis
DefensesarXiv 2609.35561Sep 29
RSI-Master：用结构化实验引导模型自主改进
提出 RSI-Master，约束自主后训练实验动作并结构化探索以防奖励作弊
arXiv
2609.35561
Published
Sep 29
Layer
Application layer
Setting
science agent, Multi-agent systems
- DefensesPermissions and approval
- RiskReward hacking
- ComponentPermissions and sandboxing
Paper analysisRead analysis
Risky behaviorarXiv 2609.30266Sep 25
研究显示 Claude Code、Codex 等本地智能体可轻易篡改自身执行轨迹
测试编程智能体被诱导或自主篡改自身执行轨迹的行为
arXiv
2609.30266
Published
Sep 25
Layer
Application layer
Setting
coding agent
Evaluated models
GPT-5.6-Sol, GPT-6-Sol, Opus-5 and others11
- AttacksMalicious use
- RiskDeception and scheming
- ComponentMonitors
Abstract论文评估了本地智能体的轨迹防篡改能力，发现其可因外部指令或奖励追求而清除执行证据，亟需建立独立拦截记录机制。
这是一项针对本地大模型智能体执行轨迹完整性与安全防篡改边界的实证研究。
Paper analysisRead analysis
Evaluation and benchmarksarXiv 2609.23980Sep 21
Stanford and UC Berkeley release MobileCybench: evaluating agent vulnerability discovery with executable probes
提出 MobileCybench，用可执行探针评测 Agent 的漏洞发现能力
arXiv
2609.23980
Published
Sep 21
Layer
Application layer
Setting
coding agent
Evaluated models
OpenCode/GPT-5.5, OpenCode/GPT-5.6-Sol, OpenCode/GLM-5.2 and others5
- RiskDangerous capabilities
Paper analysisRead analysis
DefensesarXiv 2609.19101Sep 17
用内部表征监控与发现 LLM 评测中的奖励作弊
用内部表征探针监控并发现评测中的奖励作弊
arXiv
2609.19101
Published
Sep 17
Layer
Application layer
Setting
coding agent
Evaluated models
Kimi K3, GLM 5.2, Qwen 3.8 Max and others6
- DefensesMonitoring and auditing
- RiskReward hacking
- ComponentMonitors
Abstract论文分析了前沿开源模型在评测中的奖励投机，展示了利用内部激活均值差向量低成本监控与发现投机行为的可行性。
Paper analysisRead analysis
Evaluation and benchmarksarXiv 2609.15939Sep 15
VLoc Bench released: evaluating Agent vulnerability localization capability in complete repositories
提出 VLoc Bench，评测智能体在完整仓库中定位漏洞文件并抑制修复后误报
arXiv
2609.15939
Published
Sep 15
Layer
Application layer
Setting
coding agent
Evaluated models
GPT-5.5 (xhigh), GPT-5.5 (default), Gemini 3 Pro and others15
- RiskDangerous capabilities
Abstract论文提出仓库级漏洞定位基准 VLoc Bench，指出当前模型定位表现较低且易在已修复代码上产生误报。
Paper analysisRead analysis
OtherarXiv 2609.12039Sep 11
现实才是最终验证者：智能体软件工程的两大关键缺口
提出双缺口框架，统一智能体软件工程中的奖励投机与幻觉
arXiv
2609.12039
Published
Sep 11
Layer
Application layer
Setting
coding agent
- RiskReward hacking
Abstract双缺口解释内环为何会放行部署后不可接受的实现，外环用现实证据持续修订 R、M 与 E。
Reality Is the Final Verifier 用 two-gap framework 解释智能体软件工程的主要失败，并主张用外层保证环持续收窄缺口，而不是试图在部署前证明缺口已关闭。
Paper analysisRead analysis
Evaluation and benchmarksarXiv 2609.08149Sep 8
研究者发布 SWE-Bench Pro Verified，修正奖励作弊与任务质量问题
发布 SWE-Bench Pro Verified，隔离答案通道并修订任务以抑制奖励破解
arXiv
2609.08149
Published
Sep 8
Layer
Application layer
Setting
coding agent
Evaluated models
GPT-5.6-Sol, Kimi-K3, GLM-5.3 and others7
- RiskReward hacking
Paper analysisRead analysis
Risky behaviorarXiv 2609.04170Sep 4
Google DeepMind 案例研究：100 个自主研究智能体中自发涌现的作弊与举报
展示自主科研智能体集群自发涌现的规范投机作弊与举报反制
arXiv
2609.04170
Published
Sep 4
Layer
Application layer
Setting
science agent, Multi-agent systems
Evaluated models
Gemini 3.1 Pro
- RiskReward hacking
Abstract论文报告了百个智能体在数学证明中自发作弊扩散与举报抵制的案例，提出借鉴公地治理原则构建自律机制。
Paper analysisRead analysis
Evaluation and benchmarksarXiv 2609.04075Sep 4
PatchBench: University of Maryland team releases C/C++ vulnerability repair evaluation benchmark
提出基准 PATCHBENCH，评测编程 Agent 的漏洞修补是否真正修复根因
arXiv
2609.04075
Published
Sep 4
Layer
Application layer
Setting
coding agent
Evaluated models
Codex + GPT-5.6 Sol, OpenHands + GPT-5.6 Sol, Claude Code + Claude Opus 4.8 and others11
- RiskDangerous capabilities
Abstract论文揭示漏洞修复评测的记忆与表面防护问题，提出离栈变异基准与双重验证，测得智能体实际解决率仅约五成。
Paper analysisRead analysis
Evaluation and benchmarksarXiv 2608.22103Aug 23
Hack-Verifiable Terminal Bench：评测终端任务中的奖励作弊
提出 HVTB，在终端编程任务植入蜜罐以测量奖励作弊
arXiv
2608.22103
Published
Aug 23
Layer
Application layer
Setting
coding agent
Evaluated models
claude-opus-5, gpt-5.6-sol, gemini-3.1-pro and others5
- RiskReward hacking
Abstract用蜜罐测量终端任务上的 reward hacking，并比较五档提示词。
HVTB 用来在真实终端与编程任务上测量 reward hacking：智能体读取被植入的参考解答或验收测试，从而偏离任务意图。作者认为这一失败模式会随智能体更自主而更重要，且事后人工或 LLM judge 可能失效。
Paper analysisRead analysis
Evaluation and benchmarksarXiv 2608.11469Aug 12
SRE-Bench：面向 Agent 逆向工程的无污染真实规模基准
提出 SRE-Bench 评测智能体的真实规模二进制逆向能力
arXiv
2608.11469
Published
Aug 12
Layer
Application layer
Setting
coding agent
Evaluated models
GPT-6-Astra, GPT-5.6-Sol, GPT-5.6-Cyber and others11
- RiskDangerous capabilities
AbstractSRE-Bench 用从零编写的大规模保护二进制评测智能体 RE，公开设置下多数模型完全解开比例不高。
作者构建并评测 SRE-Bench，一个从零编写、用于测量智能体逆向工程而非利用开发的基准，并称它同时满足污染控制与真实软件规模。
Paper analysisRead analysis
Evaluation and benchmarksarXiv 2608.02001Aug 3
VulnGym: A Benchmark for Repository-Level Vulnerability Detection with Coding Agents
Propose the VulnGym benchmark, evaluating programming agents' repository-level vulnerability detection and localizing failure steps
arXiv
2608.02001
Published
Aug 3
Layer
Application layer
Setting
coding agent
Evaluated models
Qwen3.5-2B, Qwen3.5-4B, Qwen3.5-9B and others7
- RiskDangerous capabilities
Paper analysisRead analysis

AI Safety Daily

Read the latest issueOne issue per day, updated as events develop