Skip to content

Translation in progress Some entries still include text in the original language.

Scale AI Research Blog·Indexed by this site · Originally published

TERMINAL-BENCH 3.0 发布:面向更强智能体的更难任务

TERMINAL-BENCH 3.0: Harder Tasks for Better Agents

AI summary

Scale AI 参与发布开源基准 TERMINAL-BENCH 3.0,任务难度较 2.0 提升,前沿模型在其上的解决率不足 40%。该基准在隔离 Docker 沙箱中让 AI 智能体操作真实 shell,按最终结果而非自述评分,覆盖约 16 个类别,Scale 是单一机构中贡献任务最多的一方。

Read originallabs.scale.com