DelegationBench 评测 Agent 授权委托行为

1. 04:54
   论文追踪Selected

   DelegationBench: Gemini 3.5 Flash-Lite asks the user first 47.5% of the time when judging, but only 4.2% when actually executing with tools

   The researchers released DelegationBench, using 156 scenarios (including 48 pairs that change only one feature) to measure when AI agents should first ask for user permission, distinguishing four responses: direct execution, request permission, ask for missing information, and refusal. The study also found that a fixed simple "yes" reply can solve 97.5% of simple permission questions, but only about 33.3%–75.0% of mixed-type questions; when rules are explicitly written, the three tested models achieve balanced accuracy of 97.3%–100%. The authors recommend reporting judgment consistency, pair responsiveness, format sensitivity, and actual execution behavior separately, and have made the benchmark and evaluation tools public.

1. Date not given
   PieLord757/delegation-benchSelected

   DiscretionBench 发布：Agent 判断该不该先问用户，拿到工具后询问率大幅下降

   研究者发布 DiscretionBench，用 156 个场景测试 AI Agent 在执行动作前是否应先征求用户授权。每个场景给出用户指令、上下文和一个候选工具调用，模型需在直接执行、请求授权、追问缺失信息或拒绝之间选择；其中 48 组配对场景仅改变一个特征，例如文件是移入回收站还是永久删除，用于检验模型决策是否真的依赖可逆性。论文评估了 5 个家族的 10 款模型，报告了单一一致率掩盖的三个差距：关键词规则与标注者的一致率高于 10 款模型中的 8 款，但只在 48 组配对中的 9 组改变决策；同一问题换等价格式可使执行率变化最多 52.5 个百分点；所有模型在需要用工具实际执行任务时（Tool-Loop）比仅判断候选动作时更少询问用户。仓库包含基准、提示词、工具沙箱、分类结果和无需调用模型即可核验数字的脚本，代码用 MIT 许可，数据用 CC BY 4.0 许可。

Current attention58·Peak in the comparable period67（Oct 8 19:00）·Change over the comparable past 24 hours 0%