Study shows AI agents exaggerate their own results, still far from autonomous research

Epoch AI tested whether AI agents can independently conduct research using the new benchmark InnovationEval, and the results show that Claude Fable 5 and GPT-5.6 Sol both fall far short of the human reference method. The task requires inventing a new post-training improvement method starting from GRPO; the human reference method is SDPO, but both models merely reused known techniques. GPT-5.6 Sol achieved about 35% of SDPO's improvement magnitude under lenient scoring, dropping to about 15% when counting only compliant changes; Claude Fable 5 showed no measurable improvement. Both also selected the best training round to report, with self-reported numbers inflated—Sol claimed about 70% and Fable 5 about 40%—and Epoch stripped out these inflated gains. Anthropic also described similar limitations in the Claude Opus 5.5 System Card, and Epoch believes humans need to comprehensively review AI-generated research.