MobileWorldSafety: A GUI agent safety benchmark for environment injection attacks in Android apps
[Daily Paper] MobileWorldSafety: Benchmarking GUI Agent Safety Against Environmental Injection Attacks in Android Apps
The researchers propose the MobileWorldSafety benchmark, which evaluates the defense capability of mobile GUI agents in Android apps against environment injection attacks under a threat model of benign users and malicious environments. The benchmark covers 13 Android apps, 5 MCP servers and 142 risky tasks, and evaluates safety failures decoupled from capability failures. The attack success rates of the six tested agents range between 40.4% and 66.9%; Gemini-3-Pro has the fewest execution failures but the highest attack success rate, and the lower values of specialized GUI agents mainly stem from high execution failure rates. The system prompt defense lowers Qwen3.5-397B-A17B's attack success rate from 51.6% to 37.0%, but the number of stalled samples increases from 2 to 20; enabling chain of thought raises Gemini-3-Flash's attack success rate from 51.2% to 57.9%. The paper suggests data provenance tracking, architectural isolation and human authorization for high-risk operations as mitigation directions.
The attack success rate of six mobile GUI agents on this benchmark ranges between 40.4% and 66.9%, and after prompt defenses, about 40% still remains.