MAI-Gym: An Open-Source All-in-One Platform for GUI Agent Evaluation and RL Training in Mobile and Computer Environments

The core problem is how to address the high cost, poor state resettability, task scarcity, and protocol drift caused by divergent evaluation and training environment implementations for GUI agents in large-scale interactive environments.

• Scenario and importance: Online reinforcement learning has made progress in mathematics, programming, and tool use, but extending it to graphical user interfaces (GUI) faces major challenges. GUI instances require exclusive phone, desktop, or browser environments, and a single attempt involves multiple screenshots and long multimodal interactions, making concurrent sharing difficult unlike code sandboxes.
• Shortcomings of existing solutions: Existing GUI infrastructure typically targets only a single platform (e.g., web-only or mobile-only), and codebases supporting online RL have limited concurrency capacity (mostly within a few hundred instances); existing data-driven environments often do not release training code or cannot run standard evaluation benchmarks, and tasks often rely on isolated files or shared samples, lacking realistic coherent multi-application interaction states.
• Evaluation and training drift problem: Due to implementation differences across platforms and benchmarks, evaluation and training often use two separate agent harnesses, diverging in screenshot preprocessing, history truncation, and prompts, causing evaluation scores to be affected by harness drift rather than purely reflecting model capability.
• What the paper proposes: The paper proposes the open-source integrated platform MAI-Gym, which unifies evaluation, data collection, and online reinforcement learning across mobile, computer, and web, supports up to 20,000 concurrent environment instances, and builds an automated pipeline for synthesizing coherent cross-application worlds, resettable mocked apps, and executable verifiers.

The paper's related work mainly covers GUI agent reinforcement learning, GUI environment platforms and training infrastructure, and environment and task synthesis techniques:

• GUI Agent Reinforcement Learning: Early work focused on online RL recipes for a single platform. Android-side work includes DigiRL, DistRL, MobileRL, and MobileGUI-RL, exploring reward design and asynchronous collection; Web-side work includes WebRL (self-evolving curriculum learning) and WebAgent-R1; desktop-side work includes ComputerRL, DART-GUI, and Qwen-CUA. Later work such as EvoCUA-1.5, UI-Mate, and SCALECUA further optimized multi-round training pipelines. For cross-domain agents, UI-TARS-2, Qwen-UI-Agent, and UI-Venus-2 train unified policies, but often use a scheme of training experts separately per domain and then merging parameters by averaging.
• GUI Environments and Training Infrastructure: Existing open-source GUI platforms fall into two categories: infrastructure-type and data-type. Infrastructure-type platforms such as BrowserGym (unified Web benchmark), WebGym, MobileGym, and SCALECUA (supporting hundreds of concurrent online RL) and ClawGUI usually cover only a single operating system; data-type platforms such as CUA-Gym, Gym-Anything, GUI-Genesis, and PhoneWorld build new apps and tasks but do not release training code or have difficulty directly running existing benchmarks.
• Environment and Task Synthesis Methods: Because the number of hand-written tasks is small, recent research uses two types of synthesis methods: one relies on VLM judge scoring (e.g., ZeroGUI, AutoPlay, OS-Genesis), which scales quickly but is prone to false-positive rewards interfering with RL; the other uses code-execution verification (e.g., GUI-Genesis, AutoWebWorld, InfiniteWeb, AutoEval, OpenComputer), which is more precise in verification but often limited to a single file or simple states.
• Baseline Methods and Benchmarks: Frontier models compared in experiments include Claude Opus 4.8, GPT-5.5, Qwen3.8-Max, and open-source Qwen3.8-27B; GUI-specific agent baselines include UI-Venus-2-9B, SCALECUA-Qwen3.5-9B, GUI-Owl-1.5 (8B/32B), EvoCUA-1.5-32B, CUA-GYM, and Qwen-CUA; benchmarks cover MobileWorld, AndroidWorld, MobileGym, OSWorld-Verified, OSWorld 2.0, MacAgentBench, and MyPCBench.
• Positioning of This Paper: The authors note that MAI-Gym simultaneously supports mobile, desktop, and Web evaluation and ultra-large-scale online RL in a unified codebase, and solves the problem of large-scale trustworthy environments and synthetic task supply by building coherent cross-app worlds and dual-path independently synthesized execution verifiers.

MAI-Gym addresses the above problems through a unified system architecture, an efficient concurrent rollout mechanism, an automated environment and task synthesis pipeline, and cross-platform joint online RL training.

System Architecture and Unified Interface

• Single Harness and Single Rollout Path: Evaluation, data collection, and online RL share the same observation processing, context construction, and action execution path. Rollout Workers uniformly call the inference engine to generate trajectories, with only the downstream consumption differing (evaluation for aggregated scoring, data collection for SFT storage, RL for policy updates), eliminating component drift.
• Universal Environment Manager: The environment manager exposes 5 standard operations (request instance, observe, reset task, step, evaluate), connecting Android (Redroid containers), desktop VMs, and headless Chromium browsers through environment adapters, hiding underlying infrastructure details from the upper layers.

Scaled Rollout and Multi-Turn Inference Optimization

• Asynchronous Provisioning and Resource Isolation: Environment instances start asynchronously and are pre-warmed via a Warm Pool; screenshots are passed through object storage rather than control messages; each session is bound to an independent lease and timeout mechanism, with infrastructure failures and task failures recorded separately to avoid interfering with training signals.
• Windowed History and KV Cache Reuse: The traditional strategy of keeping the most recent k screenshots causes the prompt prefix to change at every new step, invalidating the KV Cache. MAI-Gym uses a windowed history of size K with overlap O (e.g., K=10, O=2). Within the window, only new observations are appended; when crossing window boundaries, old screenshots are collapsed into placeholders but text is retained, so the prefix changes only every K-O steps.

Automated Environment and Task Synthesis

• Coherent Cross-App World Construction: Starting from a set user persona and scenario, a cross-app virtual world is first generated with shared facts, characters, and linked files across multiple apps; composite tasks requiring cross-app evidence combination are then extracted from the world's facts.
• Independent Dual-Path Verifier Synthesis: For each task, the requirement specification (including user goal and internal reference answer) is first fixed, then two independent model calls generate the Golden Code that directly modifies the environment to the completed state and the Verifier Code that checks state changes; the verifier must pass state checks of 0 at the initial state, 1 after executing the Golden Code, and 0 again after environment reset.
• Mocked App Construction: 7 real clone apps covering real transaction flows (e.g., 12306, banking, food delivery) were built to support large-scale side-effect-free interaction, along with 42 new apps based on HTML5 state machines.

Joint Online Reinforcement Learning

• Cross-Platform Joint Optimization Objective: Based on the GRPO algorithm, mobile and desktop tasks are jointly optimized in a single training run, with G attempts per task to compute relative advantage:

A_i = (r_i − mean(r_1:G))/(std(r_1:G) + δ) where the objective uses clipped surrogate advantage: L(θ) = 1/(∑_i |Y_i|) ∑_i=1^G ∑_{u ∈ Yi} min(ρ_u A_i, clip(ρ_u, 1−ϵ, 1+ϵ) A_i)

• Policy-Adaptive Task Sampling: Task success rates are dynamically evaluated; mastered tasks (success rate above 0.8) and unsolved tasks receive reduced allocation budgets (set to 2 and 4 attempts respectively), focusing training on tasks where the policy is in the critical interval.

Experimental Setup

• Model under test: MAI-Gym-9B (based on Qwen3.5-9B, warmed up with SFT and trained with cross-platform joint GRPO).
• Comparison benchmarks: 7 evaluation benchmarks, divided into seen Task OOD (MobileWorld, OSWorld-Verified) and unseen Env OOD (AndroidWorld, MobileGym, OSWorld 2.0, MacAgentBench, MyPCBench).
• Baseline models: frontier models Claude Opus 4.8, GPT-5.5, Qwen3.8-Max, Qwen3.8-27B; same-scale GUI models UI-Venus-2-9B, SCALECUA-Qwen3.5-9B; large-parameter GUI models GUI-Owl-1.5 (8B/32B), Qwen-CUA, EvoCUA-1.5-32B, CUA-GYM, etc.; and the base model Qwen3.5-9B.
• Evaluation protocol and metrics: All evaluated uniformly via mai-gym eval, with 3 runs per task and decoding temperature 0.0. Metrics are each benchmark's native success rate (OSWorld 2.0 reports binary accuracy and partial score, MobileGym reports partial score, MyPCBench reports full-task completion rate).
• Training scale: 512 GPUs, peak concurrency of 20,000 environment instances, trained on 5,000 synthetic tasks for about 96 hours.

Main Results

Table 4 summarizes the performance of MAI-Gym-9B and baselines on mobile and computer benchmarks:

Swipe horizontally to read the table

Model	Parameters	MobileWorld (Task OOD)	AndroidWorld (Env OOD)	MobileGym (Env OOD)	OSWorld-Ver. (Task OOD)	OSWorld 2.0 (Env OOD)	MacAgent (Env OOD)	MyPCBench (Env OOD)
Claude Opus 4.8	Not disclosed	69.2	75.0	50.7	83.4	20.3 / 54.8	58.4	62.0
GPT-5.5	Not disclosed	62.4	74.1	48.1	78.7	13.9 / 47.5	66.7	47.3
Qwen3.8-Max	2.4T	77.8	85.3	61.3	86.1	19.4 / 46.7	74.3	48.6
GUI-Owl-1.5-8B	8B	38.2	71.6	15.1	52.9	0.0 / 1.6	10.2	1.8
UI-Venus-2-9B	9B	75.2	80.2	52.7	70.8	0.0 / 7.5	36.2	6.3
SCALECUA-9B	9B	12.8	47.4	13.5	68.7	0.0 / 2.2	3.9	1.1
Qwen3.5-9B (Base)	9B	18.0	57.8	9.0	41.8	0.0 / 2.5	20.7	6.5
MAI-Gym-9B (Ours)	9B	78.4	75.9	46.5	76.2	3.7 / 11.3	41.3	15.2

• Same-scale performance: Among open-source GUI models under 10B, MAI-Gym-9B outperforms UI-Venus-2-9B (75.2%/70.8%) and SCALECUA-9B (12.8%/68.7%) on both MobileWorld (78.4%) and OSWorld-Verified (76.2%).
• Env OOD transfer: It outperforms the base model on all 5 external benchmarks not used in training, and exceeds same-scale open-source baselines with reported results on OSWorld 2.0, MacAgentBench, and MyPCBench.
• Surpassing some frontier models: The authors state that MAI-Gym-9B's 78.4% on MobileWorld exceeds Claude Opus 4.8 (69.2%) and GPT-5.5 (62.4%).

Cross-Platform Joint Training vs. Expert Merging

• What was tested: Comparing joint RL training on mobile and desktop, versus training independent experts separately and then merging weights with weighted averaging (70/30, 50/50, 30/70).
• Results (per Table 7): Joint RL achieves 78.4% and 76.2% on MobileWorld and OSWorld-Verified, respectively; among merging schemes, the best 50/50 reaches 76.4% on MobileWorld (73.7% on desktop), and the best 30/70 reaches 74.8% on OSWorld-Verified (73.5% on mobile).
• Authors' interpretation: No single merging weight leads on both domains simultaneously; joint RL outperforms expert merging on both ends.

Transfer from Cloned Apps to Real Devices

• What was tested: After training on the 12306 train-ticket booking clone app, zero-shot transfer performance was tested on physical real phones using 52 paired tasks.
• Results (per Figure 9): Success rate in the clone app rose from 36.5% to 73.1% (a 36.5 percentage point improvement); success rate on the real phone app rose from 28.8% to 48.1% (a 19.2 percentage point improvement).
• Authors' interpretation: Capabilities learned via rewards in the clone app, such as relative date reasoning, can transfer to real devices, but the gap between clone and real device widened from 7.7 to 25.0 percentage points, so transfer remains incomplete.

Other Ablations and Analyses

• Synthetic task scaling (Figure 8): Increasing training tasks from 500 to 5,000 raises MobileWorld success from 69.2% to 78.4% and OSWorld-Verified from 69.7% to 76.2%.
• Zero-shot learning on entirely new unseen apps (Section 4.3.5): On 62 held-out tasks across 42 new apps, RL improves success rate from 25.8% to 45.2%, and the number of step-exhaustion timeouts drops from 27 to 5.
• Windowed history efficiency (Table 6): Compared to Step-wise, the Window-wise mechanism raises prefix cache hit rate from 42.5% to 85.9%, reduces per-trajectory training input tokens by 84.7%, and cuts single training update time from 2166.4 seconds to 810.9 seconds.
• Environment provisioning and reset efficiency (Table 5): Launching 1,000 mobile, computer, and browser environments takes 328.4, 88.9, and 94.2 seconds, respectively; median reset times are 165.0, 37.2, and 6.0 seconds; unscored abnormal interruption rates are 1.2%, 1.9%, and 0.0%, respectively.
• GRPO vs. SAO comparison (Figure 10): Although SAO reduces sampling wait time, its computational overhead of 2 Critic updates per Actor update offsets the benefit, and the two ultimately yield similar training results.

Limitations and Future Directions Noted by the Authors

• Clone-to-Real-Device Transfer Is Still Incomplete: The authors note in Section 4.3.4 that although training on cloned apps can transfer to physical phones, the performance gap between clones and real devices widened from 7.7 to 25.0 percentage points, so the benefit of the cloned environment cannot fully equal real app performance.
• Multi-Round Adaptive Synthesis Loop Not Evaluated: The authors note in Section 3.3 and Section 3.4 that the platform architecture supports organizing evaluation failure diagnosis and task synthesis into continuous multi-round iteration, but this paper has not experimentally evaluated multi-round closed-loop self-evolution.
• Real-Device Sensitive Operations Still Rely on Simulation: The authors note in Section 3.4.1 and Appendix A.7.1 that scenarios involving real fund flows, identity authentication, and order publishing cannot be explored tens of thousands of times directly in the real environment, and currently still rely on isolated simulation with cloned apps.

Experimental Coverage

• Policy Models Tested: RL training experiments were mainly validated on a single Qwen3.5-9B backbone model, without covering models of other sizes or architectures for cross-platform joint RL.
• Benchmark Evaluation Coverage: In the main evaluation of Table 4, 7 benchmarks were covered, and Web benchmarks such as WebVoyager, OnlineMind2Web, and WebArena were not included in the main comparison result table for MAI-Gym-9B.
• Real-Device Transfer Test Scope: Sim-to-Real transfer experiments on physical phones were tested only on the single 12306 railway travel app (52 paired tasks).
• New App Test Scale: Among the 42 synthetic HTML5 new apps, the held-out task evaluation set has 62 tasks, each run with 1 attempt.
• Hardware and Training Configuration: Distributed training was mainly completed on a 512-GPU configuration, using the GRPO algorithm with preset K=10, O=2 window hyperparameters.

MAI-Gym is an open-source cross-platform GUI agent evaluation and online reinforcement learning platform that addresses the high cost of building GUI interaction environments, the scarcity of resettable environments and coherent tasks, and the inconsistency between evaluation and training protocols.

• Core positioning and architecture: The platform uniformly supports mobile, desktop, and web, using a unified rollout path and Agent Harness shared by evaluation, data collection, and RL training to avoid evaluation drift caused by different environment implementations; the underlying system supports up to 20,000 concurrent environment instances.
• Coherent task and environment synthesis: It builds a cross-application world generation mechanism starting from roles and scenarios, producing files and records with factual associations; it proposes dual independent Golden Code and Verifier Code generation with a three-stage executable verification pipeline; and it develops 7 clones of real applications and 42 new synthetic applications.
• Efficient training mechanism: It designs a windowed history mechanism that reduces duplicate screenshot encoding through overlapping windows and placeholders, achieving an 85.9% KV cache prefix hit rate and a 62.6% reduction in single training update time; combined with policy-adaptive task sampling to balance exploration budget.
• Main experimental results: MAI-Gym-9B, trained on Qwen3.5-9B, reaches 78.4% on the Task OOD benchmark MobileWorld and 76.2% on OSWorld-Verified, both outperforming open-source GUI baselines of the same scale; it shows positive gains on all 5 unseen external benchmarks.
• Transfer and ablation findings: Training on the 12306 clone app improves success rate on the real phone app from 28.8% to 48.1%; zero-shot evaluation on synthetic new apps improves success rate from 25.8% to 45.2%; cross-domain joint RL outperforms averaging expert parameters from single domains.
• Authors' conclusion and released resources: The authors conclude that the platform validates the effectiveness of large-scale online RL in simulated/cloned environments with verifiable synthetic tasks, and they open-source all evaluation and training code, synthetic skills, 49 mocked apps, 2,000 executable verification tasks, and model weights.