Skip to content

Translation in progress Some entries still include text in the original language.

Oct 12 · Monday

All papers

Found 235 papers
Filter1
Select any value within a category; different categories must all match. Counts include the other filters you have selected.Another 240papers have no tags yet and are not included in filtered results.

Another 240papers do not have category tags yet and are not included in filtered results.

  1. SWE-Duel: Toward a Self-Expanding Arena for Coding Agents

    Proposes SWE-Duel, which lets coding agents mutually evaluate, create problems, and blindly review fixes during evaluation.

    Published
    Evaluated models
    GPT-5.5, Claude Opus 4.8, Kimi K2.7 Code and others11
    AbstractThe mutual evaluation arena creates questions during evaluation.

    SWE-Duel is a coding agent arena that generates tasks and judges only at evaluation time, to avoid leakage and saturation of static benchmarks.

    Paper analysisRead analysis
  2. The paper proposes the ISE framework to evaluate LLM agents' epistemic humility under knowledge conflicts

    Proposes the ISE framework to evaluate the epistemic humility of LLM agents under knowledge conflicts

    Published
    Setting
    AI Agent
    Evaluated models
    Nemotron-Orchestrator-8B, Claude Sonnet 4.6, GPT-5 and others5
    AbstractISE trajectory evaluations show that high task accuracy across four agents does not correspond to higher cognitive humility.

    What the authors want to test is not whether the agent's final answer is correct, but whether, when a knowledge conflict arises, it recognizes the gap in its trajectory, performs bounded tool follow-up, and writes the unresolved uncertainty to the user.

    Paper analysisRead analysis
  3. 研究提出 Agent 安全新维度:识别未履行的安全义务

    提出义务识别基准 ObligationBench,并训练护栏识别 Agent 未履行的安全义务

    Published
    Evaluated models
    ObligationGuard, Claude-Opus-4.8, GPT-5.6-Sol and others17
    Abstract义务是护栏缺口。

    作者认为编码智能体的安全不能只靠拦截禁止动作,还要找出终止时仍未做的安全关键动作,并将其称为义务。

    Paper analysisRead analysis
  4. Too many skills: how co-installed skills conflict in coding agents

    评测编码 Agent 中相似技能共同安装对已装技能核心功能的替代

    Published
    Evaluated models
    claude-sonnet-4-6, claude-haiku-4-5-20251001, claude-opus-5 and others6
    Abstract冲突藏在任务成功后面。

    作者对共同安装、做同一件事的良性 skill 做实证研究,用核心功能是否还在,而不是任务过不过,来看冲突。。

    Paper analysisRead analysis
  5. Workerville: Toward an Organizational Behavior Explanation of Agent Safety

    构建 Workerville 基准,测量组织条件如何改变 Agent 的安全行为

    Published
    Setting
    AI Agent
    Evaluated models
    GPT-5.2, Claude-Sonnet-4.6, Gemini-3.1-Pro and others6
    Abstract作者用 ACB 与 Workerville 测量组织条件对智能体安全的影响,并称结果支持可量化的组织行为框架。

    Workerville 用组织行为测量智能体安全如何随监督、同伴和记忆条件变化。。要处理的缺口是:持续运行的 LLM 智能体同时接收用户指令、同伴消息和长期记忆,但既有安全评测多把这些通道拆开,没有统一的定量测量。

    Paper analysisRead analysis
  6. SSCBench:评估工具调用 LLM Agent 故障注入测试证据有效性的测量协议与基准

    提出 SSCBench,评测工具调用 Agent 故障注入测试的证据有效性

    Published
    Setting
    AI Agent
    Evaluated models
    deepseek-flash, deepseek-v4-pro, gpt-5.6-luna and others5
    Abstract解释采纳前应报告该次执行的证据条件,以及支撑该主张的观察总体。

    SSCBench 研究工具使用智能体的故障注入结果,在什么证据条件下可以被解释。

    Paper analysisRead analysis
  7. The paper points out that the reliability of LLM vulnerability repair benchmarks is affected by evaluation design.

    用对照实验隔离评测设计对漏洞补丁基准得分的影响

    Published
    Evaluated models
    Claude Sonnet 4.0, Claude Sonnet 4.5, Claude Opus 4.7
    Abstract作者认为评测流水线会改变补丁基准分数,PoC 通过率高于开发者测试通过率。

    作者把 LLM 漏洞补丁基准的评测流水线当作会改变测量结果的对象,并明确说本文不是再做一个用于排名的基准。

    Paper analysisRead analysis
  8. 研究:Agent 更依赖工具报错通道

    测量工具调用 Agent 对显式报错与静默损坏的检测和恢复

    Published
    Setting
    AI Agent
    Evaluated models
    claude-haiku-4.5-thinking, claude-haiku-4.5, deepseek-r1 and others6
    Abstract作者称智能体看错误通道而非返回内容,静默损坏更少被当作问题。

    作者在 BFCL 多轮可执行环境上注入一次工具层故障,把注意到、改计划、恢复和重复分开记分,而不是只看任务有没有做完。

    Paper analysisRead analysis
  9. AgentTime 基准测试智能体时长遵循能力,GPT-6 Astra 出现 14 次做完后休眠

    提出 AgentTime 基准,评测智能体按时长工作与估时能力

    Published
    Evaluated models
    Fable 5.1, GPT-5.6 Sol, GPT-6 Astra and others7
    Abstract作者把完成任务和管理所用时间分开。

    AgentTime 是对智能体时间自控的评测:预测任务耗时、按要求时长工作、事后估计已用时间。

    Paper analysisRead analysis
  10. 研究审计 LLM 读取 Agent 日志的取证能力:答案正确但引用来源多为猜测

    评测 LLM 审计 Agent 日志时来源重建是否有证据支持

    Published
    Setting
    AI Agent
    Evaluated models
    gpt-5.6-luna, claude-sonnet-4-6, gpt-5.4-mini-2026-03-17 and others5
    Abstract保存执行范围的标识绑定,才能把猜中的引用和证据支持的引用分开。

    包级审计检查保存的智能体动作里,哪些来源结论能被交给分析者的记录支持。

    Paper analysisRead analysis
  11. 论文提出 agent plasticity 指标衡量智能体通过经验自我改进的效率

    提出 agent plasticity 指标衡量智能体通过经验自我改进的效率

    Published
    Evaluated models
    Claude Fable 5, Claude Opus 5, GPT-5.6 Sol and others10
    Abstract作者用固定权重协议比较前沿模型把经验写成制品后的留出增益与习得效率。

    作者用 agent plasticity 衡量一件事:权重冻结时,智能体把经验写成可继承制品,再换成留出能力增益,这个转换有多省。

    Paper analysisRead analysis
  12. ParanoiaEval:编码智能体不必要风险处置评测基准发布

    提出 ParanoiaEval 评测编码智能体的不必要风险处置

    Published
    Evaluated models
    Opus-5, Sonnet-5, Sonnet-4.6 and others8
    Abstract作者用证据受控任务对评测风险处置,称其普遍、独立于任务能力并降低满意度。

    作者提出 ParanoiaEval,并称它是编码智能体风险处置能力的首个统一基准。要解决的问题是:证据已经确定风险应如何处置之后,智能体仍可能扩大验证、增加保护或备份,而现有评测要么只看功能正确性,要么把相关行为拆开研究。做法是把 NIST 的 ATMA 落到仓库任务。

    Paper analysisRead analysis
  13. Learn2Play Bench 发布:用规则陌生的文本游戏评测 LLM Agent 从经验中学习的能力

    用 Learn2Play Bench 评测 Agent 从经验中学习陌生规则

    Published
    Setting
    AI Agent
    Evaluated models
    Claude Opus 5.5, GPT-6 Astra, Claude Opus 5 and others11
    Abstract用隐藏规则游戏把新学习与预训练知识分开,并比较模型、记忆方式和人类。

    Learn2Play Bench 用新设计的隐藏规则文本游戏,评测 LLM 智能体在不更新权重时从交互中学习的能力。

    Paper analysisRead analysis
  14. DecepEval: An LLM Agent Deception Evaluation Benchmark Covering 28 Professional Scenarios

    构建基准 DECEPEVAL,评测 LLM 智能体在外部条件下的欺骗

    Published
    Setting
    AI Agent
    Evaluated models
    Claude Haiku 4.5, Claude Sonnet 5, Claude Opus 5 and others9
    AbstractDECEPEVAL 用配对诱导比较智能体欺骗。

    DECEPEVAL 是一个评测 LLM 智能体欺骗如何随任务、职业场景和外部条件变化的基准。

    Paper analysisRead analysis