GPT-6-Astra solves 49 of 50 physics research problems at $2.40 per problem
GPT-6-Astra solves 49 of 50 physics research problems at $2.40 apiece
On the 50 theoretical physics research problems in TheoPhysEval, GPT-6-Astra running in the OpenAI Codex environment answered 49 correctly (98%), at an average cost of under $3 per problem, about $116 for the full run. Anthropic's Opus 5.5 in Claude Code scored 92%, Gemini 3.8 Flash scored 68%, and open-weight models DeepSeek-V4.1-Flash and GLM 5.3 scored 54% and 42%, respectively. The problems cover 11 subfields of theoretical physics and deliberately exclude major open questions, focusing on routine research problems that apply known methods with slight modifications. Web search had little impact: without web access, Astra and Opus each scored 92%, which the authors take as evidence that leading language models have memorized a large portion of the theoretical physics literature. The authors also say some AI errors are actually due to errors in the reference answers themselves, and argue that the strongest current models may already surpass the best human researchers in concrete computation, but do not yet have the research taste to judge whether a problem is interesting.
On the same set of theoretical physics research problems, the scores and per-problem costs of GPT-6-Astra, Opus 5.5, and several open-source models are listed together for easy horizontal comparison of each agent's scientific computing performance.