Scale AI proposes the MRT framework to test the reliability of agent monitoring

1. 12:30
   WeChat · Weijiqi LixinSelected

   Scale AI proposes the MRT framework, red-teams LLM agent monitoring systems, and presents a hybrid monitoring architecture

   The Scale AI team proposes the Monitor Red Teaming (MRT) framework, which tests LLM agent monitoring systems with a unified threat model, evasion strategies, and evaluation metrics, and open-sources the code, datasets, and logs. The framework evaluates monitoring on two datasets: SHADE-Arena, covering tool-calling scenarios, and the newly added CUA-SHADE-Arena, covering computer-operation scenarios (15 main-subtask groups, extended from TheAgentCompany). Experiments show that whether the agent knows it is being monitored affects monitoring reliability more than how much the monitor knows about the task; under the monitor-hijacking strategy, the AUC of the baseline architecture can drop below 0.5, while the hybrid architecture still maintains 0.7-0.8. Under the hybrid architecture, when the weaker GPT-4o-mini monitors Claude 4 Sonnet, the AUC is still no lower than 0.85; combined with selective human feedback that reviews only suspicious trajectories, TPR@FPR=0.01 can be raised from 20.6% to 36.2%.

Current attention 38·Peak in the comparable period 38 (Nov 6 03:00)·Change over the comparable past 24 hours 0%