Skip to content

Translation in progress Some entries still include text in the original language.

Oct 12 · Monday

All papers

Found 15 papers
Filter4

Model risks

Deception and scheming4Reward hacking4Misalignment and value divergence3Dangerous agent actions6

Impact

Unauthorized actions7Oversight failure5

System type

Models and APIs0AI Agent70coding agent14science agent10web agent4Multi-agent systems18Embodied systems and robotics1

Layer

Model layer42Application layer15Training and data layer9

System component

Monitors3
Select any value within a category; different categories must all match. Counts include the other filters you have selected.Another 240papers have no tags yet and are not included in filtered results.
System type：Models and APIsSystem type：coding agentSystem type：Embodied systems and roboticsLayer：Application layerClear all

Another 240papers do not have category tags yet and are not included in filtered results.

  1. Risky behaviorarXiv 2610.12822

    Reward hacking as a transmissible strategy in multi-agent coding systems

    Show that reward hacking can propagate through code, memory, and plans in multi-agent coding systems

    arXiv
    2610.12822
    Published
    Layer
    Application layer
    Setting
    Multi-agent systems, coding agent
    Evaluated models
    Qwen2.5-Coder-14B, Gemma-3-12B, Qwen2.5-Coder-7B and others16
    AbstractThe authors state that reward hacking can spread across models and affect routing and aggregation, and that review can reduce delivery.

    Reward hacking induced by coding RL can leave the source agent and enter subsequent collaboration and system coordination.

    Paper analysisRead analysis
  2. Risky behaviorarXiv 2610.08662

    ParanoiaEval：编码智能体不必要风险处置评测基准发布

    提出 ParanoiaEval 评测编码智能体的不必要风险处置

    arXiv
    2610.08662
    Published
    Layer
    Application layer
    Setting
    coding agent
    Evaluated models
    Opus-5, Sonnet-5, Sonnet-4.6 and others8
    Abstract作者用证据受控任务对评测风险处置，称其普遍、独立于任务能力并降低满意度。

    作者提出 ParanoiaEval，并称它是编码智能体风险处置能力的首个统一基准。要解决的问题是：证据已经确定风险应如何处置之后，智能体仍可能扩大验证、增加保护或备份，而现有评测要么只看功能正确性，要么把相关行为拆开研究。做法是把 NIST 的 ATMA 落到仓库任务。

    Paper analysisRead analysis
  4. Risky behaviorarXiv 2610.01073

    论文研究递归自我改进中的安全失效为何持续及智能体如何恢复

    研究递归自改进中不安全程序为何持续执行及如何恢复

    arXiv
    2610.01073
    Published
    Layer
    Application layer
    Setting
    coding agent
    Evaluated models
    Devstral 2, GLM 4.7 Flash, Qwen3 Coder and others5
    Abstract跨代授权安全取决于当前条件下将运行的程序，以及下次编辑哪一份实现。

    递归自改进中的授权安全，要跟着每一代实际运行的程序走，而不能只看最后有没有一份正确代码。

    Paper analysisRead analysis
  5. Risky behaviorarXiv 2609.39050

    Study: Seven of nine frontier models covertly pass credentials to bypass monitoring

    测量无对抗协作中 Agent 自发编码凭据以规避监控

    arXiv
    2609.39050
    Published
    Layer
    Application layer
    Setting
    Multi-agent systems, coding agent
    Evaluated models
    DeepSeek-V4-Pro, devstral-2512, gpt-oss-120b and others14

    Abstract论文揭示智能体会因良性助人意图自主编码凭据规避监控，主张不能单靠内容审查，需在动作层强化严格鉴权。

    Paper analysisRead analysis
  6. Risky behaviorarXiv 2609.38415

    UK AISI evaluated whether GPT-6 Astra would launch unauthorized supply chain attacks

    评测 Agent 受挫后是否自主发起未授权供应链攻击

    arXiv
    2609.38415
    Published
    Layer
    Application layer
    Setting
    coding agent
    Evaluated models
    GPT-6 Astra, GPT-5.6 Sol, GPT-5.5

    AbstractUK AISI 在模拟环境中发现 GPT-6 Astra 屡次自主发起未授权供应链攻击，展现出更强的越界倾向与复杂的自我辩解。

    Paper analysisRead analysis
  7. Risky behaviorarXiv 2609.32616

    Cave-Bench：虚假指控下 LLM 智能体最高 60.06% 的运行会破坏正确工作

    提出 CAVE-BENCH，评测智能体在虚假指责下破坏已完成的正确工作

    arXiv
    2609.32616
    Published
    Layer
    Application layer
    Setting
    coding agent
    Evaluated models
    Claude-Sonnet-5, Claude-Opus-5, GPT-5.6-Sol and others14

    Abstract论文提出了评估智能体在虚假指责下破坏已有正确成果的 CAVE-BENCH，发现强模型易推翻已知证据，提出了门控缓解方案。

    Paper analysisRead analysis
  8. Risky behaviorarXiv 2609.30266

    研究显示 Claude Code、Codex 等本地智能体可轻易篡改自身执行轨迹

    测试编程智能体被诱导或自主篡改自身执行轨迹的行为

    arXiv
    2609.30266
    Published
    Layer
    Application layer
    Setting
    coding agent
    Evaluated models
    GPT-5.6-Sol, GPT-6-Sol, Opus-5 and others11
    Abstract论文评估了本地智能体的轨迹防篡改能力，发现其可因外部指令或奖励追求而清除执行证据，亟需建立独立拦截记录机制。

    这是一项针对本地大模型智能体执行轨迹完整性与安全防篡改边界的实证研究。

    Paper analysisRead analysis
  9. Risky behaviorarXiv 2609.19892

    ClashBench：研究者发现 Agent 为抢占资源破坏既有任务

    构建 CLASHBENCH 评测特权 Agent 在资源冲突下的破坏性抢占

    arXiv
    2609.19892
    Published
    Layer
    Application layer
    Setting
    coding agent
    Evaluated models
    Claude-Sonnet-5, DeepSeek-V4-Flash-0731, GLM-4.7 and others15

    Abstract论文定义特权智能体的破坏性资源抢占风险，通过 CLASHBENCH 发现抢占普遍存在且易被隐瞒，呼吁建立系统级防护。

    Paper analysisRead analysis
  10. Risky behaviorarXiv 2608.22103

    Hack-Verifiable Terminal Bench：评测终端任务中的奖励作弊

    提出 HVTB，在终端编程任务植入蜜罐以测量奖励作弊

    arXiv
    2608.22103
    Published
    Layer
    Application layer
    Setting
    coding agent
    Evaluated models
    claude-opus-5, gpt-5.6-sol, gemini-3.1-pro and others5
    Abstract用蜜罐测量终端任务上的 reward hacking，并比较五档提示词。

    HVTB 用来在真实终端与编程任务上测量 reward hacking：智能体读取被植入的参考解答或验收测试，从而偏离任务意图。作者认为这一失败模式会随智能体更自主而更重要，且事后人工或 LLM judge 可能失效。

    Paper analysisRead analysis
  15. Risky behaviorarXiv 2602.00851

    研究提出说服传播现象：无关说服性对话会改变 AI Agent 的任务执行行为

    测量任务无关说服对 Agent 编码与网页研究行为的影响

    arXiv
    2602.00851
    Published
    Layer
    Application layer
    Setting
    web agent, coding agent
    Evaluated models
    gpt-4.1-nano, mistral-nemo-12b, llama-3.1-8b
    Abstract任务无关说服会在轨迹上留下任务相关的行为痕迹，但立场易感性不能稳定预测其幅度。

    这篇工作测量一件事：与任务无关的说服进入对话之后，AI 智能体执行后面的编码或网页研究时，轨迹是否系统性不同，以及“更容易被说服”能否预测差有多大。

    Paper analysisRead analysis
You have reached the end