Skip to content
The Decoder· Maximilian Schreiner·Indexed by this site · Originally published· AI updates →SelectedAttention48

Anthropic's Fable 5.2 completes 41% of real freelance tasks; a year earlier the top system had only 2.5%

Anthropic's Fable 5.2 completes 41 percent of real freelance jobs, up from 2.5 percent a year ago

AI summary

Anthropic's Fable 5.2 achieved a 41.3% completion rate on the RLI benchmark, which tests AI Agents on real freelance tasks, ranking first, more than 16 times the 2.5% of the top system a year earlier, Manus, and about 2.5 times Fable 5's July score (about 16%). According to Scale Labs, it improved most on audio and music editing, architecture and interior design, and 3D models; nearly 50 completed projects were tasks that no AI Agent had previously solved in testing, but industrial product design and animation remain almost unmanageable. Second and third on the leaderboard are Anthropic Opus 5.5 (21.25%) and OpenAI GPT 6 Astra (20.83%). The researchers said that when Agents fail, they usually deliver amateur-level, incomplete, corrupted-file or self-contradictory results.

Why it was selected

The benchmark uses real Upwork projects with acceptance by human experts; the top completion rate rose from 2.5% to 41.3% in one year, and can serve as a reference for observing the progress of Agent practicalization.

Read originalthe-decoder.com