OccuBench: Evaluating AI agents' performance on real occupational tasks through language environment simulation

Researchers release the OccuBench benchmark, which uses a language environment simulator (LES) to simulate domain environments through LLM-generated tool responses, constructing 100 real occupational task scenarios covering 10 industry categories and 65 professional domains, and automatically generating solvable, difficulty-calibrated evaluation instances via a multi-agent synthesis pipeline. The benchmark evaluates 15 frontier models from 8 model families along two dimensions, task completion and environmental robustness, and the results show that no single model leads across all industries, with each model having its own occupational capability profile; implicit failures (data truncation, missing fields) are harder to handle than explicit errors (timeouts, 500 errors) and mixed failures, because they lack obvious error signals and require the agent to detect data degradation on its own; larger, newer models and higher reasoning effort consistently improve performance, with GPT-5.2 improving by 27.5 points from the lowest to the highest reasoning effort. The authors also point out that a strong agent is not necessarily a strong environment simulator, and that simulator quality is crucial to the reliability of LES evaluation.