TheoPhysEval: GPT-6-Astra solves 49/50 physics problems
Start here
TheoPhysEval is a set of 50 unpublished, verifiable research-level theoretical physics problems used to test AI agents' ability to solve problems autonomously without human hints. The paper reports that when tools such as literature search and numerical and symbolic computation scripts are available, the best-performing GPT-6-Astra with OpenAI Codex achieves 98% accuracy, solving 49 problems, with an average cost per problem below $3 and a total round cost of about $116. Anthropic's Opus 5.5 in Claude Code scores 92%, Gemini 3.8 Flash 68%, and open-weight models DeepSeek-V4.1-Flash and GLM 5.3 score 54% and 42%, respectively. The paper also reports that after disabling web search and background research papers, the agents' accuracy changed only slightly on average, suggesting significant memorization of the literature. Other reports state that the benchmark involves Harvard, Google DeepMind, and other institutions, with problems drawn from experts' unpublished research notes covering quantum field theory, holography, and more, and includes both connected and disconnected test tracks; GPT-6 Astra initially scored about 88%, rising to 98% after expert review and correction of reference answers.
Generated by AI from reports · Updated 2026-10-18
Progress timeline
- X@leishen168888· Discussionhttps://mp.weixin.qq.com/s/W7wG-_3XdufChPXh2_XWtQ(opens the original post in a new tab)
- WebAI Era (Xinzhiyuan)· Media coverageShocking Harvard and Google research: GPT-6 Astra has solved 98% of the world's physics problems!(opens the original post in a new tab)
- WeChat · AI Era (Xinzhiyuan)Harvard and others release TheoPhysEval: GPT-6 Astra scores 98% after experts revise answers
The TheoPhysEval evaluation, involving institutions such as Harvard and Google DeepMind, shows that GPT-6 Astra scored about 88% in the initial assessment on 50 unpublished theoretical physics problems, rising to 98% after experts reviewed and corrected the reference answers, solving 49 problems. The problems were drawn from experts' unpublished research notes, covering quantum field theory, holographic theory, and more, and included two test tracks, one with internet access and one without; offline, both Astra and Claude Opus 5.5 scored 92%. The report states that the average API cost per problem was under $3. A concurrent MIT team study shows that frontier AI's problem-solving rate in theoretical science rose from 2% in mid-2024 to 35% in mid-2026.
- The DecoderSelectedGPT-6-Astra solves 49 of 50 physics research problems at $2.40 per problem
On the 50 theoretical physics research problems in TheoPhysEval, GPT-6-Astra running in the OpenAI Codex environment answered 49 correctly (98%), at an average cost of under $3 per problem, about $116 for the full run. Anthropic's Opus 5.5 in Claude Code scored 92%, Gemini 3.8 Flash scored 68%, and open-weight models DeepSeek-V4.1-Flash and GLM 5.3 scored 54% and 42%, respectively. The problems cover 11 subfields of theoretical physics and deliberately exclude major open questions, focusing on routine research problems that apply known methods with slight modifications. Web search had little impact: without web access, Astra and Opus each scored 92%, which the authors take as evidence that leading language models have memorized a large portion of the theoretical physics literature. The authors also say some AI errors are actually due to errors in the reference answers themselves, and argue that the strongest current models may already surpass the best human researchers in concrete computation, but do not yet have the research taste to judge whether a problem is interesting.
- XX @boazbaraktcs· DiscussionNice to see the model holding its ground when it’s right!(opens the original post in a new tab)
- RedditReddit r/ControlProblem· DiscussionGPT-6 Astra autonomously solves 49 out of 50 unpublished theoretical physics research problems, scoring 98% at under $3 per problem. Researchers initially marked several answers wrong, only to discover the AI had caught errors in the calculations of world-class physicists.(opens the original post in a new tab)
- arXivTheoPhysEval: Evaluating AI agents' ability to autonomously solve theoretical physics problems
TheoPhysEval proposes a set of 50 unpublished, answer-verifiable research-level theoretical physics problems to test off-the-shelf AI agents' ability to solve problems autonomously without human prompting. The paper states that when tools such as literature search and numerical and symbolic computation scripts are available, the best-performing agent, GPT-6-Astra with Codex, achieved 98% accuracy, with an average cost per problem below $3. After disabling web search and background research papers, the agents' accuracy changed only slightly on average, which the paper suggests indicates significant memorization of the literature. The authors also discuss the implications of these results and possible reasons why agents excel at problem-solving yet have not yet led to breakthroughs in theoretical physics.
Related discussion
- Older than 72 hours · Not included in the 72-hour figures
- X@aryaniyaps · Aryan IyappanGPT-6-Astra solved 49 of 50 unpublished physics problems for under $3 a problem man. what happens when someone points t…(opens the original post in a new tab)
1 likes·Oct 26 23:34Collect
- XX @rohanpaul_ai · Rohan PaulNew paper from Google Deep Mind, and many top Universities finds the best AI agent tested solved 49 of 50 unpublished r…(opens the original post in a new tab)
301 likes · 91 comments·Oct 27 18:45Collect
- X@leishen168888 · 雷神https://mp.weixin.qq.com/s/W7wG-_3XdufChPXh2_XWtQ(opens the original post in a new tab)
- XX @haider1no doubt GPT-6 Astra is a big breakthrough model for scientific research especially considering how much cheaper it is(opens the original post in a new tab)
- WebAI Era (Xinzhiyuan)Shocking Harvard and Google research: GPT-6 Astra has solved 98% of the world's physics problems!(opens the original post in a new tab)
- X@Tsubame33785667 · TsubameAfter mathematics, next is theoretical physics. When Astra was made to solve an unpublished research problem, it claimed that the model answer prepared by an expert was wrong, and that point was correct. This is clear evidence that AI has surpassed human experts. Are humans the side grading AI's answers…(opens the original post in a new tab)
185 likes · 14 comments·Oct 20 04:33Collect
- X@mcelia · Michael CeliaGPT 6 Astra in Codex got 49 of 50 unpublished research physics problems on TheoPhysEval. A few it first got marked wron…(opens the original post in a new tab)
1 likes · 1 comments·Oct 19 23:34Collect
- X@EveryMeetsAI · Everything Meets AIAI INDUSTRY: GPT-6-Astra solves 49 of 50 physics research problems at $2.40 apiece https://the-decoder.com/gpt-6-astra-…(opens the original post in a new tab)
- X@kabelsalat_info · Kabel-Salat | AI & Tech49 of 50 is not a theory of everything. TheoPhysEval grades final answers on unpublished physics problems: GPT-6-Astra…(opens the original post in a new tab)
1 comments·Oct 17 18:33Collect
- X@AnatoliKopadze · Anatoli KopadzePhysics is solved. 30+ physicists from Harvard, Google DeepMind and top research institutes just admitted AI now match…(opens the original post in a new tab)
48 likes · 7 comments·Oct 19 04:34Collect
- X@alex_verem · Alex VeremeyenkoGPT-6 Astra disagreed with expert physicists on several research problems. It insisted the expert results were wrong an…(opens the original post in a new tab)
115 likes · 12 comments·Oct 26 17:34Collect
- X@blockquery1234 · Goblin LabGPT-6 Astra scored 49/50 on a physics evaluation, and the 1 point lost was not because it answered incorrectly, but because the question setter's reference answer was wrong. The evaluation author later rechecked and admitted the model's answers were correct. The interesting part is not the score. Who wrote the question bank, who reviewed the answers, and whether a second person double-checked—the leaderboard never says. ...(opens the original post in a new tab)