Embodied First Exam: A Robot Control Evaluation Benchmark for Coding Agents

The paper proposes Embodied First Exam, a unified evaluation framework that lets coding agents choose direct control, code control, or trained policies to control robots. The benchmark contains 47 robots, 35 suites, and 1,156 tasks, of which 9 new suites cover spatial puzzles, dexterous manipulation, multi-robot interaction, dynamic whole-body control, and sim-to-sim engineering. Each task can run under up to three settings: standard, real-time, and privileged; the standard setting provides only sensor observations and by default does not allow environment resets. On 456 core tasks, the strongest of eight agents, GPT-6 Astra in Codex, achieved a 34% success rate under the standard setting.