GSM-Agent: Understanding Agent Reasoning with Controllable Environments

On the same problems, how much does static reasoning, where the prompt already contains all premises, differ from agentic reasoning, where the agent must collect premises using tools, and in which behavior does the difference lie?

• Scenario: The authors believe that when models act as agents, reasoning must shift from static reasoning within the prompt to agentic reasoning with tools, especially search: deciding what to read, what to ask next, when to verify, and how to leave unproductive directions. The authors state that existing agent benchmarks often mix this capability with difficult math and expert knowledge.
• Gap: The authors note that existing benchmarks cannot compare static and agentic settings on the same or similar tasks, and lack a framework to identify and quantify core skills.
• Benchmark: Built on GSM8K, GSM-AGENT. Questions and premises are separated; premises are written as documents and stored in a library; evaluation gives only the question, and the agent uses Search and NextPage to collect information before solving. The authors state that difficulty can be controlled by database construction, e.g., adding distracting documents.
• Task setup:
  • The goal is to find a numeric answer; the prompt contains only the question, not premises.
  • Tools are Search and NextPage. Search returns the 5 most relevant; NextPage at most 19 times per search, seeing at most top 100. Queries are chosen by the agent.
  • The environment is a document collection, each document corresponding to a premise of some problem.
  • Correctness is determined by answer equality to the standard numeric value.
• Analysis hypothesis: The authors use document embedding clustering to define an agentic reasoning graph, mapping tool calls to paths, distinguishing exploration, exploitation, and revisits. The authors state that revisits are often missing in many models' agentic trajectories and correlate with accuracy; accordingly, they add tools encouraging revisits for test-time scaling. The abstract writes GPT-5 has only 67% accuracy.

• Reasoning with incomplete information: Li et al. (2025)'s QuestBench evaluates whether models can ask appropriate questions, including GSM8K variants with missing information. The authors note its focus is whether models ask specific questions, not overall reasoning. Zhou et al. (2025b) compare passive and active reasoning, which the authors consider close to their static vs. agentic distinction, but the two settings use different tasks; this paper's dataset can be used in both settings.
• Agent reasoning benchmarks: The authors jointly discuss SWE-bench (Jimenez et al., 2023), τ-bench (Yao et al., 2024), ToolSandbox (Lu et al., 2024), AppWorld (Trivedi et al., 2024), and BFCL (Patil et al., 2025), stating they evaluate agentic reasoning. Their comparison is that this paper requires a controllable environment for direct comparison with static reasoning, rather than critiquing these benchmarks individually.
• Understanding reasoning: Theoretical lines include Zhu et al. (2024), Chen et al. (2024), Wang et al. (2025), Zhu et al. (2025), Huang et al. (2025). Empirical lines include Huang et al. (2024)'s self-correction, Zhou et al. (2025a)'s GSM-Infinite and Mirzadeh et al. (2025)'s GSM-Symbolic, and Yeo et al. (2025), Sun et al. (2025), Minegishi et al. (2025) on long-chain reasoning or reasoning behavior. The authors state that the exploration, exploitation, and revisit graph analysis is partly inspired by these works and extends to agentic settings with search tools, with graphs defined by document embeddings; the introduction credits inspiration from Minegishi et al. (2025).
• Baselines and data: Experimental comparisons include zero-shot ReAct, few-shot, prompt-only CoT, interaction round scaling, and tool combinations of Thinking, Explore, Revisit. The evaluation data is GSM-AGENT, adapted from GSM8K. The introduction also cites GSM8K, MATH, HotpotQA, etc., as examples of static reasoning tasks, not as baselines run in this paper.

The authors position this paper as: comparing static and agentic reasoning on the same task, and using a controllable environment and reasoning graph to analyze which tool-use behavior causes the gap.

The overall approach is to build a document environment with adjustable difficulty, allowing the same elementary math problem to be evaluated both statically and as an agentic task requiring retrieval, then discretize tool trajectories using an agentic reasoning graph.

Task Formulation

• The dataset is written as a task set, environment, and tool set. Each problem includes a question, several premises, and a numeric standard answer; the number of premises varies by problem.
• During evaluation, the agent sees only the question, must determine what is missing, call tools, then decide whether information is sufficient and when to answer.
• In the environment, each document is generated from a premise by a document generator and contains the information needed to solve the problem using that premise. The environment is referred to as the database later.

Data Construction

• Five stages: preprocessing, problem decomposition, document generation, filtering, and database construction. Unless otherwise stated, processing uses Claude-3.5-Sonnet. Appendix D gives full prompts for each stage.
• Preprocessing handles two types of ambiguity: shared character names across problems, and only generic entities. Steps are entity detection, assigning names to generic entities, and using entity graph coloring to assign year-month timestamps so that same-named entities have different times across problems.
• Problem decomposition splits the narrative into self-contained premises and rewrites the question; when sharing entity names with other problems, the question specifies the timestamp.
• Document generation has three steps: first write a high-level story for the whole problem, then generate documents per premise containing body text, unique ID, and metadata; then use a model for independence checks to avoid one document leaking information from other premises; then randomly rewrite 30% of documents so entity names do not appear in the body but move to metadata, avoiding retrieval only by character name. After successful retrieval, metadata can be viewed.
• Filtering rules: keep only problems that Claude-3.5-Sonnet can solve correctly when seeing the question and corresponding documents. Remaining 7323 problems, 32315 documents.
• Database construction uses Chroma, with body text (excluding ID and metadata) embedded using text-embedding-3-large. Three difficulty levels: Full is all filtered problems; Medium and Small are 25% and 6.25% respectively. Each library contains only documents for problems in that subset. Appendix Table 3: Full has mean documents per problem 4.41 (min 1, max 14), mean span 2.89.

Tools and Judgment

• Search(x) returns the 5 most relevant documents; queries can be sentences or keywords. NextPage returns the next 5; each Search allows at most 19 additional NextPage calls, i.e., at most top 100.
• The standard answer is numeric. The task is solved if and only if the model's final answer equals that numeric value.

Agentic Reasoning Graph

• K-means is applied to document embeddings in the library, K=250. Each centroid is a node. The caption of Table 2 states that t-SNE highlights three clusters, with centroids corresponding to nodes 29, 54, 100, and gives example cluster themes (Grocery Receipts, Fruit Counting, Youth Sports Stats). The text does not show point coordinates, so positions are not described.
• At step t, if Search(x) is called, the node is the centroid closest to the query embedding, p_t=argmin_{vk∈ V}|q(x)−c_k|₂. If NextPage is called, the node equals the previous step. The path is the sequence of node steps.
• The first step is always exploration. Thereafter, nodes never reached are exploration; nodes identical to the previous step and previously reached are exploitation; leaving and returning to a visited node is a revisit. Each proportion is the number of such steps divided by total tool call steps.

Test-Time Scaling with Tool Augmentation

• The authors do not define scaling as merely increasing interaction rounds, but as adding tools to encourage revisits. Thinking, when called, copies the model's previous tokens. Explore and Revisit have the same retrieval effect as Search, with system prompts requiring them to explore different queries or return to previous queries, respectively. Appendix E gives full prompts.
• Four experimental combinations: Thinking only, Explore only, Revisit only, and Explore+Revisit. There is also CoT, which only changes the prompt without adding tools.

实验设置

• 框架：全部评测用 LangChain ReAct。temperature 0.4，max tokens 4096，max search rounds 50。作者称零样本在多数模型上最稳，优于 few-shot 或多智能体；主表因此用零样本。
• 数据：Section 4 默认 GSM-AGENT-Full。每个结果是 100 题子样本、3 个随机种子的平均。Claude 因 API 不能指定种子只跑 1 次。
• 主表模型：o3、GPT-5、Grok-4、Claude-4-Sonnet、Gemini-2.5-Pro、Kimi-K2-Instruct、Gemini-2.5-Flash、GPT-4o、Llama-4-Maverick、DeepSeek-V3、Qwen3-235B、Llama-4-Scout。DeepSeek-R1、Claude-Opus 有附录结果，但未进主对比。
• 指标：Accuracy 为答案等于标准数值的比例。SR 为解题所用工具调用次数。SC 为找到全部相关文档的题目比例。ER 为找齐相关文档后的额外工具调用。FF 为按要求格式作答的比例；作者要求最终答案放在“####”之后。PremT 为最终决定前就给出过早答案的比例。另有总生成 token 与每轮平均 token。
• 判定：数值相等，论文未使用单独的 LLM 裁判。默认嵌入 text-embedding-3-large。
• 其他设置：few-shot、CoT、截断式与强制式交互扩展、四种工具组合、三种嵌入、三种库规模。

主结果

据 Table 1，零样本 ReAct，GSM-AGENT-Full 的 100 题子样本。Claude-4-Sonnet 只跑 1 次。

Swipe horizontally to read the table

模型	Acc	SR	SC	FF
o3	68.46%	13.33	53%	95%
GPT-5	66.78%	9.98	52%	100%
Claude-4-Sonnet	56.00%	7.39	42%	100%
Grok-4	53.00%	7.19	42%	100%
Gemini-2.5-Pro	38.33%	2.93	25%	82%
Kimi-K2-Instruct	37.42%	5.41	24%	92%
GPT-4o	22.67%	1.92	22%	94%
Llama-4-Scout	12.54%	2.07	9%	86%

省略 Gemini-2.5-Flash（25.33%）、Llama-4-Maverick（20.00%）、DeepSeek-V3（19.42%）、Qwen3-235B（19.30%），均来自 Table 1。

• 作者称 o3 与 GPT-5 最高，搜索轮次也多，并认为这对应更长的无人工流程；Grok-4 为 53.00%。Table 1 中 Claude-4-Sonnet 为 56.00%，高于 Grok-4，但只跑 1 次。作者称搜索轮次更多时准确率倾向更高。
• 作者称 Gemini-2.5-Pro 与 GPT-5 的差距主要来自更少搜索轮次。表中前者 FF 为 82%，o3 为 95%，GPT-5 与 Grok-4 为 100%。Gemini 系列 token 因 LangChain 格式不匹配为 Nan。
• 作者称环境只需小学数学与基本常识，最好准确率仍是 o3 的 68.46%。摘要写 GPT-5 为 67%，Table 1 为 66.78%。引言称 GPT-5 相对静态设定约损失 33% 绝对准确率，DeepSeek-V3 最多损失 80%。Table 1 另有一行 Solvable by any model，Acc 为 88.00%，其余为 Nan；表注未定义该行。

交互轮次扩展

• Figure 3 把零样本轨迹按工具调用次数截断：n 轮内答对，或 n 轮内找齐必要文档且最终答对，都算对。横轴为对数刻度。标注斜率与 R²：GPT-5 为 0.145、0.963；Kimi-K2 为 0.087、0.982；Llama-4-Maverick 为 0.028、0.921；Llama-4-Scout 为 0.046、0.961。图中未标出各点准确率。
• 作者称开源模型随额外搜索轮次仅小幅上升，交互时扩展弱于 GPT-5，因此要看每步质量，而不只看轮次。Figure 7 把同一截断协议扩到更多模型，作者称总体上看专有模型好于开源模型。
• Table 4 另有强制继续搜索的 interaction scaling，协议不同于 Figure 3。Kimi-K2-Instruct 为 40.0%（零样本 37.42%），Llama-4-Scout 为 25.51%（12.54%），Qwen3-235B 为 37.0%（19.3%），Llama-4-Maverick 为 13.0%（20.0%）。Maverick 低于其零样本。

推理图与回访

• 作者把每步标成探索、利用或回访，再与 GSM-AGENT 准确率作相关。Figure 4(b) 横轴为 Revisit Proportion (V 3)，标注 R² = 0.902、Slope = 2.002。作者称准确率与回访比例强相关，并认为回访可能是核心技能。探索比例 R² = 0.038、Slope = 0.281，作者称弱相关；利用比例 R² = 0.331、Slope = -0.630，作者称负相关。
• Table 5 的零样本总体回访比例：o3 为 24.56%，GPT-5 为 16.81%，Grok-4 为 17.77%，GPT-4o 为 0.62%，DeepSeek-V3 为 0.00% 且 V=1.00。按 V≥3 的 Rvst-V3+：o3 为 29.88%，GPT-5 为 26.40%，Llama-4-Scout 为 8.49%，Qwen3-235B 为 3.57%；DeepSeek-V3 该列为空。这与 Figure 4(b) 的横轴标签一致，不等于上列总体回访比例。

工具增强

• Figure 5a 比较零样本、CoT，以及 Thinking、Explore、Revisit、Explore+Revisit。图上无逐柱数字，下列数字来自 Table 4。作者称 Llama-4-Maverick 与 Qwen3-235B 上工具方法一律高于 CoT；Llama-4-Scout 与 Kimi-K2 上与 CoT 相当；多数情况下两者都高于零样本。
• Qwen3-235B 的 Revisit 为 45.68%，零样本 19.3%，CoT 31.0%。Kimi-K2-Instruct 的 think tool 与 CoT 均为 46.0%，零样本 37.42%。Llama-4-Scout 的 explore-revisit 为 19.39%，CoT 18.33%，零样本 12.54%。Llama-4-Maverick 仅 think tool 为 22.33%，高于零样本 20.0%；explore 15.67%、explore-revisit 16.0%、revisit 11.07%、CoT 8.67% 均低于零样本。
• 作者称 Figure 5b 显示回访比例增幅与准确率增幅强相关；该图标注 R² = 0.250、Slope = 0.869。Figure 8 对探索比例变化标注 R² = 0.735、Slope = 0.504，对利用比例变化标注 R² = 0.803、Slope = -0.453；图注称探索与回访的变化与准确率相关，利用比例变化为负相关。引言称工具增强优于 interaction-round scaling。Table 4 中 Llama-4-Scout 的强制扩展为 25.51%，高于其 explore-revisit 的 19.39%。

其他消融与分析

• 库规模（Table 7，整数百分比，full/medium/small）：o3 为 68%、81%、80%；Grok-4 为 53%、61%、69%。作者称库越小准确率越高。同表 Llama-4-Maverick 为 20%、17%、22%，medium 低于 full；o3 的 small 低于 medium。
• 嵌入（Table 6，text-embedding-3-large / small / all-MiniLM-L6-v2）：作者称影响很小且相对排序保持。o3 准确率 68.46%、65.00%、56.00%；Gemini-2.5-Pro 为 38.33%、38.00%、23.00%；Llama-4-Maverick 为 20.00%、11.00%、7.00%，FF 为 97%、27%、21%。
• few-shot（Table 4）：作者称零样本对多数模型更稳且优于 few-shot。GPT-5 few-shot 为 57.0%，低于零样本 66.78%；Grok-4 few-shot 为 56.33%，高于 53.0%；Qwen3-235B few-shot 为 35.0%，高于 19.3%。论文未给出多智能体系统的数字。
• DeepSeek-V3（Table 4）：explore-revisit 为 41.67%，零样本 19.42%，think tool 为 16.67%。DeepSeek-R1 零样本 0.31%，Claude-Opus 零样本 4.0%。正文称 DeepSeek 的工具格式不同，Claude 常向用户索取信息，故未纳入主对比。
• Figure 7 截断协议斜率：Qwen3-235B 为 0.000（R²=nan），GPT-4o 为 0.047（R²=0.942），Grok-4 为 0.133（R²=0.962），Gemini-2.5-Pro 为 0.136（R²=0.978）。
• 过滤后仍有模型找不到全部文档：Table 1 中 o3 的 SC 为 53%，Acc 为 68.46%。论文未解释 Acc 高于 SC 的原因。

作者指出的局限与后续方向

• 论文没有 Limitations、Discussion 或 Future Work 专节，未把某项结果写成不足。
• Conclusion 只写：希望 GSM-AGENT 成为后续研究智能体推理的可控环境，并希望 agentic reasoning graph 带来对智能体推理的理解与改进。摘要有相近的期望表述，不是单独的局限清单。

实验覆盖范围

• 主结果是 Table 1 的 12 个模型、零样本 ReAct。附录 Table 4 还有 DeepSeek-R1 与 Claude-Opus；正文说明因工具格式不同或频繁向用户索取信息，二者未进入主对比。
• 过滤后为 7323 题、32315 篇文档。Section 4 报告的是 GSM-AGENT-Full 的 100 题子样本，3 个随机种子平均；Claude 因 API 不能设种子只评测 1 次。
• 写明的运行参数为 temperature 0.4、max tokens 4096、max search rounds 50、LangChain ReAct；默认嵌入为 text-embedding-3-large，库存于 Chroma。成功判定是数值答案相等。论文未报告与人工判定的一致性。
• 另有 few-shot、CoT、Figure 3 的轨迹截断、Table 4 的强制继续搜索、四种工具组合、三种嵌入模型，以及 Full、Medium（25%）、Small（6.25%）三种库。
• 推理图用 K-means，K=250，并把 Search、NextPage 映到节点后统计探索、利用、回访。引言给出相对静态设定的两例降幅，未给出各模型静态准确率全表。Gemini 系列 token 因输出格式不匹配缺失（Table 1 注）。

GSM-AGENT places elementary math problems into a controllable document library to compare static reasoning where premises are already in the prompt with agentic reasoning where only the question is given and retrieval is required.

• Problem: The authors believe agents must simultaneously reason and make tool decisions, while existing benchmarks often mix in difficult math or expert knowledge, preventing same-question comparisons. This paper also aims to identify behaviors related to this gap and attempt improvements.
• Approach: GSM8K is disambiguated, split into questions and premises, expanded into documents, and stored in a library. Agents use Search and NextPage to retrieve documents; numeric answer equality counts as correct. Document embeddings are clustered into 250 nodes, tool calls are mapped to paths, and steps are classified as exploration, exploitation, or revisit. Additional tools are added that have the same retrieval effect as Search but prompt for revisits or exploration.
• Main results: On zero-shot, Full 100-question subsample, o3 is 68.46%, GPT-5 66.78%, Llama-4-Scout 12.54% (Table 1). The abstract writes GPT-5 as 67%. The introduction states that GPT-5 loses about 33% absolute accuracy relative to the static setting, and DeepSeek-V3 loses up to 80%.
• Behavior: In Figure 4, revisit proportion (x-axis labeled V 3) has R² = 0.902 with accuracy. The authors state that many models lack revisits and believe revisiting may be a key skill; exploration proportion has R² = 0.038. Open-source models have lower slopes than GPT-5's 0.145 on the added-rounds curves in Figure 3.
• Tool variants: Qwen3-235B's Revisit is 45.68%, zero-shot 19.3% (Table 4). On Llama-4-Maverick, several tool variants are lower than zero-shot 20.0%, only think tool is 22.33%. The authors state that tools encouraging revisits are better than merely increasing interaction rounds in most cases; in Table 4, Llama-4-Scout's forced scaling 25.51% is higher than its tool variants.
• Conclusion: The authors hope the benchmark and reasoning graph will be used for future understanding of agentic reasoning, and state that adding tools that encourage revisits can improve some models' performance on this benchmark.