Skip to content
机器之心 JIQIZHIXIN· @jiqizhixin · X·Indexed by this site · Originally published· AI updates →· Its place in the scan record →

Emory, USC and other institutions introduce the CLASHarness benchmark to evaluate how computer-operation agents handle multi-agent conflicts

AI summary

Researchers from Emory University, USC and partner institutions introduce the CLASHarness benchmark, which evaluates how agents handle multi-agent conflicts in executable computer environments; the paper is at https://arxiv.org/abs/2610.19128v1,代码开源于 https://github.com/google-research/CLASHarness。基准包含 600 multi-turn tasks targeting stale reports, contradictory instructions and shared-resource contention in collaboration, with scenarios constructed via a staged generation pipeline that first fixes decisive visual facts and then renders them, executes reference solutions and derives code and semantic checks from trajectories; the visual evidence in the tasks requires agents to inspect it themselves rather than infer it only from teammate messages. Evaluation of 15 frontier models exposed agent failures in evidence verification, proactive repair and coordination; in the step-budget ablation, Gemini 3.8 Flash's pass rate rose from 12.8% at 20 steps to 30.7% at 60 steps.

Related papers
Full text · Original

When a teammate's message contradicts the dashboard, which evidence should a computer-use agent trust?

Researchers from Emory University, USC and collaborating institutions introduce CLASHarness, an arXiv benchmark for resolving multi-agent conflicts in executable computer environments.

The bottleneck: Collaboration can involve stale reports, contradictory instructions and shared-resource contention. Detecting a contradiction is only the start; agents must repair state and complete the handoff.

The mechanism: A staged pipeline constructs scenarios, fixes decisive visual facts before rendering, executes reference solutions and derives code-based and semantic checks from those trajectories. Its benchmark contains 600 multi-round tasks, with visual evidence that agents must inspect rather than infer from peer messages alone.

The payoff: Evaluation of 15 frontier models exposes failures in evidence checking, active repair and coordination. In a step-budget ablation, Gemini 3.8 Flash rises from 12.8% pass rate at 20 steps to 30.7% at 60. These are results on generated sandbox scenarios; stronger checkpoint scores do not guarantee successful end-to-end coordination.

CLASHarness: Benchmarking Multi-Agent Conflict in Computer Use via a Controllable and Verifiable Generation Pipeline

Paper: https://arxiv.org/abs/2610.19128v1
Code: https://github.com/google-research/CLASHarness
Our Report: https://www.jiqizhixin.com/articles/2026-11-01

📬 #PapersAccepted by Jiqizhixin

Source: 机器之心 JIQIZHIXIN · x.com