Translation in progress Some entries still include text in the original language.
BackSource Scale AI Research / SEAL
Vincent Siu, Manasi Sharma, Dawn Song, Daniel Yue Zhang, Chenguang Wang
Indexed by this site 2026-10-04 14:21 9 days ago
Originally published Jun 19
Selected 2026-10-04 14:21
Category Evaluation Scale AI Research / SEAL · Vincent Siu, Manasi Sharma, Dawn Song, Daniel Yue Zhang, Chenguang Wang · Indexed by this site 2026-10-04 14:21 · Originally publishedJun 19 SelectedAttention46
Scale AI 提出 ChainWorld,将 OSWorld 原子任务组合成长程桌面工作流评测 Agent ChainWorld: Composing Long-Horizon Desktop Workloads from Atomic OSWorld Tasks
AI summary
Scale AI 研究团队提出 ChainWorld,通过方向兼容性搜索把 OSWorld 的原子桌面任务组合成长程桌面工作流,同时保留原有评测器。该工作流包含 347 条长度为二到四的任务链,并对同一任务序列采用两种呈现方式:单轮评测把所有任务放在一个提示词中,多轮评测则逐个揭示任务。在四款现有 computer-use Agent 上,任务链最高完成率为 31%;多轮评测提升了其中三款模型的完成率,但两种协议都仍具挑战性。两种协议暴露出不同的失败特征:单轮失败集中在产物精度,多轮失败更多体现为会话管理问题,如进度碎片化和后续轮次脱离。
Why it was selected
ChainWorld 把原子桌面任务串成长程工作流,并对比单轮与多轮协议下四款 computer-use Agent 的完成率与失败模式。
Read original labs.scale.com Why it was selected SelectedAttention46
Attention 0–100: content value contributes up to 70 and circulation up to 30; circulation decays over time. How it was rated
ChainWorld 把原子桌面任务串成长程工作流,并对比单轮与多轮协议下四款 computer-use Agent 的完成率与失败模式。