跳到正文
原文
Epoch AI· @EpochAIResearch · X·本站收录· 原文发表· 评测 →· 扫描记录里的位置 →

Epoch AI 实测:最强 LLM 均未达其员工工作标准

AI 导读

Epoch AI 用 Epoch Automation Reports 基准测试 GPT-6.2 Astra 等最新模型,无一达到 Epoch 员工工作标准。Fable 5.2 以平均 82% 居首;GPT-6.2 Astra 虽与 Fable 5.2 并列 ECI 最高分 174,平均 65%,排第四。在一项任务上 Fable 5.2 与 Opus 5.5 达到合同工水平。

正文 · 原文

Can the most capable LLMs match humans at real work? Not yet, at least not on the kind of work we do at Epoch.

We tested GPT-6.2 Astra and other recent models on Epoch Automation Reports, our benchmark of open-ended Epoch work tasks, ranging from making Epoch-style diagrams to designing basic research experiments.

An Epoch employee scores the models’ outputs based on our internal quality standards. A score of 100% represents the standard expected of an Epoch employee.

All models failed to meet the Epoch employee standard. GPT-6.2 Astra ranks fourth (averaging 65%), despite sharing the top ECI score of 174 with Fable 5.2.

Fable 5.2 scored highest, averaging 82%.

On one task, both Fable 5.2 and Opus 5.5 matched the level of a contractor. More on that below.

来源:Epoch AI · x.com