Skip to content
WeChat · AI Era (Xinzhiyuan)·Indexed by this site · Originally publishedSelectedAttention40

CLASHarness measures the performance of 15 frontier models in multi-agent conflict

当Agent遇上猪队友!实测多Agent冲突场景,前沿大模型如何应对

AI summary

A research team from Google and institutions including Emory and USC released CLASHarness, a multi-agent conflict benchmark, evaluating 15 mainstream models on 600 executable tasks; no model exceeded 30% end-to-end pass rate. The tasks cover 20 real-world domains, with 7,970 4K visual evidence images and 3,284 checkpoints; the generation pipeline first removes answer clues from the text, forcing the Agent to open monitoring screenshots or bill images to verify teammates' reports. Under a 20-step limit, Claude Opus 5 had a 25.2% pass rate and GPT-5.6 Sol 24.5%, leading the field; after relaxing the step limit, GPT-6 Astra reached 28.7%; with image input disabled, all models' pass rates dropped by 87%–90%. When only investigation reporting was required, the average pass rate was 28.1%; when required to actually modify system state, it fell to 11.0%; completing the full multi-agent collaboration workflow was only 4.4%. The higher the proportion of thinking tokens, the lower the success rate, with a correlation of ρ=-0.70.

Why it was selected

All 15 models had end-to-end pass rates below 30% on 600 multi-agent conflict tasks, which can be used to compare Agents' shortcomings in verifying teammates' reports and in the closing stage.

Related papers
Read originalmp.weixin.qq.com