METR: On tasks with clear goals and easy scoring, Anthropic's model capability improvement speed is about twice that of the past, with about 80% possibly coming from AI
研发速度翻倍的 Anthropic 掌握了什么「秘密配方」?
AI summary
The evaluation organization METR reports that on tasks with clear goals and easily quantifiable outcome evaluation, the speed at which Anthropic improves model capabilities is about twice the historical baseline, of which about 80% may come from AI. The organization conducted an on-site review for 11 weeks, but Anthropic redacted some non-public information on the grounds of preventing leaks; key data were deleted or altered, and methodological details are difficult to independently re-check. METR notes that the estimates may have significant upward or downward bias, and the actual R&D efficiency remains undetermined.
Read original