Study: Multi-agent teams are costly but offer limited quality gains
Start here
Evaluation company Vals AI compared single-agent and multi-agent teams of GPT-6 Sol and Claude Opus 5.5 on Vibe Code Bench, finding that team costs were 1.8 to 5.1 times those of a single agent, with a statistically significant improvement in only one of four comparisons. That improvement occurred at GPT-6 Sol's medium reasoning intensity, where the team scored 7.3 points higher; at the highest reasoning intensity, the team setup offered no practical advantage for either model. Anthropic's own tests with Opus 5.5 also showed limited gains from adding more agents.
Generated by AI from reports · Updated 18 hr ago
Progress timeline
- The DecoderStudy finds AI agent teams consume large amounts of tokens yet bring almost no measurable quality improvement
Evaluation company Vals AI compared single-agent and multi-agent teams of GPT-6 Sol and Claude Opus 5.5 on Vibe Code Bench, finding that team costs are 1.8 to 5.1 times that of a single agent, with only one of four comparisons showing a statistically significant improvement. This improvement occurred at GPT-6 Sol's medium reasoning intensity, where the team scored 7.3 points higher; at the highest reasoning intensity, the team setup had no practical advantage for either model. Anthropic's own tests with Opus 5.5 also showed that quality gains shrink as agents are added, from 10 to 100 agents, with scores only slightly rising after 24 hours.
Related discussion
Attention trend
New sources per day
- Oct 11New1 sources (1 outlets,0 accounts)
- Oct 12New0 sources (0 outlets,0 accounts)