Skip to content

OpenDS-World: a deep search benchmark in real web environments

Deep Search in Real-World Web Environments

Hugging Face Daily Papers· arXiv:2610.30181· Xiaomin Yu, Hongliang Chen, Zhe Huang, Shilin Yan, Shao-Rong Su, Xuqiao Ma, Feiyang Wang, Jiayu Chen, Jiwei Fang, Yu Yan, Yini Huang, Chen Chen, Ruoxi Cheng, Yuzhi He, Yuhui Zhang, Xiaoxing Hu, Chen Liu, Jianbo Lin, Yijiang Li, Xiaobin Hu, Menglong Li, Zhuoran Lu, Weishi Wang, Zhongqi Yue, Chengwei QinHKUST(GZ); FDU; UW; IC; NTU; Stanford; SJTU; Yale; NJU; UCSD; NUS; HKU; SAP; Microsoft ResearchIndexed by this site · 28 days ago· Originally publishedAI updates

Quick read

Evaluation and benchmarks

Based on the paper PDF (AI-generated); refer to the original

The authors built the OpenDS-World benchmark (200 real-web deep search tasks), evaluated 12 frontier models, with the highest accuracy at 39.0% (GPT-6 Astra), while independent human solving was 21.1%.

Questions
Existing Deep Search evaluations do not adequately cover the evidence acquisition process in real web environments where multimodality, heterogeneous formats, and interactive access are intertwined, making it difficult to systematically evaluate an agent's end-to-end deep search capability.
Methods
Proposes the OpenDS task and builds OpenDS-World: starting from real video anchors, 200 tasks are generated via open-web evidence expansion and question inversion, ten reviewers independently verify evidence validity and answer uniqueness, and 50 human solvers calibrate difficulty.
Experiments and results
Under a unified tool environment (16 tools, 3,600 seconds, 100-call limit), 12 frontier models were evaluated: GPT-6 Astra was highest at 39.0%, humans 21.1%; more tool calls and longer time did not correspond to higher accuracy; failed tool calls accounted for a low share but consumed 19.27%–61.90% of tool time.
Limitations and future work
The paper does not have a dedicated limitations section; Appendix E lists budget exhaustion, failure recovery, and others as directions for improvement. The evaluation covers only 12 models and 200 tasks, the judge is Qwen3.8-27B, and the paper does not report the consistency between the judge and human judgments, nor the number of repeated runs.
Experimental setupGPT-6 Astra、GPT-5.6 Sol et al. · OpenDS-World · ACC、total token consumption et al.
Evaluated models
GPT-6 AstraGPT-5.6 SolClaude Opus 5.5Claude Fable 5.1Gemini 3.8 FlashQwen3.8-MaxQwen3.8-FlashDeepSeek-V4.1-FlashDeepSeek-V4-ProKimi K3GLM-5.3-FlashGLM-5.3
Benchmarks
OpenDS-World
Metrics
ACCtotal token consumptionelapsed time
AI summaryThe researchers propose the OpenDS task and construct the OpenDS-World benchmark, used to evaluate agents' ability to autonomously complete multi-step deep search in the real-time open web. The benchmark contains 200 tasks, covering multiple information modalities, heterogeneous file formats, and sources that require interactive access, and the validity of evidence and the uniqueness of answers were independently verified by humans. Under a unified tool environment and execution protocol, 12 frontier models were evaluated; the best-performing model achieved an accuracy of 39.0%, while the accuracy of human evaluation in 1000 independent attempts was 21.1%. Analysis of tool use, search cost, termination behavior, and failure recovery shows that different models differ markedly in research strategies, and tool coordination and long-horizon search management remain persistent difficulties.

The researchers propose the OpenDS task and construct the OpenDS-World benchmark, used to evaluate agents' ability to autonomously complete multi-step deep search in the real-time open web. The benchmark contains 200 tasks, covering multiple information modalities, heterogeneous file formats, and sources that require interactive access, and the validity of evidence and the uniqueness of answers were independently verified by humans. Under a unified tool environment and execution protocol, 12 frontier models were evaluated; the best-performing model achieved an accuracy of 39.0%, while the accuracy of human evaluation in 1000 independent attempts was 21.1%. Analysis of tool use, search cost, termination behavior, and failure recovery shows that different models differ markedly in research strategies, and tool coordination and long-horizon search management remain persistent difficulties.

Related events and news

2 events · 2 news mentions

News that mentions this paper

OpenDS-World 基准:12 个前沿模型最高仅 39%

OpenDS-World基准发布,12模型最高准确率39%

  • 公众号 机器之心cites the paper's id or link
    OpenDS-World 基准:Deep Search 进入真实互联网,最强模型 GPT-6 Astra 准确率仅 39%
    围绕这一问题,来自香港科技大学(广州)、复旦大学、华盛顿大学、帝国理工大学、斯坦福大学等多家机构的研究团队,在论文《Deep Search in Real-World Web Environments》中提出了新的深度搜索任务范式:OpenDS(Open-Web Deep Search),并构建了面向真实开放网络环境的评测基准 OpenDS-World。

Paper analysis

6 questions; open one to read the full answer

  1. What problem does this paper attempt to solve?

    Existing Deep Search evaluations do not adequately cover the multimodal, heterogeneous-format and interactive-access challenges of the real web; the authors propose the OpenDS task and the OpenDS-World benchmark.

    Core problem: existing Deep Search evaluations are insufficient to systematically assess agents' ability to acquire and integrate evidence across heterogeneous information sources in real open-web environments.

    • Problem scenario: The authors state that the complexity of the real web is reflected in three aspects—diverse modalities (text, images, audio, video), heterogeneous formats (HTML, PDF, XML, Parquet, etc.), and interactive access requirements (web navigation, interface operations, parameter updates, artifact transfer)—requiring agents to autonomously search, access and integrate evidence.
    • Shortcomings of existing work: The authors state that existing benchmarks (Li et al., 2025b; Java et al., 2026; Liu et al., 2026a; Xu et al., 2026) do not adequately cover these intertwined challenges, limiting systematic analysis of deep search capability in real environments.
    • Core design: Propose the OpenDS (Open-Web Deep Search) task—given a factual question and initial clues, the agent must autonomously explore the open web, connect evidence across information sources with multiple modalities, multiple formats and multiple access mechanisms, and give a unique verifiable factual answer; the location and acquisition method of the evidence are not provided in advance.
    • Paper output: Build the OpenDS-World benchmark, containing 200 tasks, covering eight domains, four modalities, 42 formats and seven types of access requirements, independently manually verified to ensure evidence validity and answer uniqueness; and evaluate 12 frontier models under a unified tool environment and execution protocol, plus 1,000 independent human solves to calibrate difficulty.
  2. What related research exists?

    Related work falls into three groups: open-web deep search, multimodal evidence deep search, and interactive evidence acquisition; the authors state that existing benchmarks do not adequately cover the intertwining of the three types of challenges.

    Deep search in open web environments

    • Retrieval augmentation and learnable retrieval: Search-o1 (Li et al., 2025c) incorporates external retrieval into reasoning; Search-r1 (Jin et al., 2025a) and ReSearch (Chen et al., 2025a) use reinforcement learning to make retrieval decisions learnable; DeepResearcher (Zheng et al., 2025), WebDancer (Wu et al., 2026), and WebThinker (Li et al., 2026c) learn from web interaction trajectories.
    • Long-horizon search and task construction: DeepDiver (Shi et al., 2025) adjusts search frequency and depth according to evidence needs; HIRA (Jin et al., 2025b) and ManuSearch (Huang et al., 2025) use hierarchy and multi-agent coordination; ReSum (Wu et al., 2025d) and WebResearcher (Qiao et al., 2025) perform context summarization and state management; WebShaper (Tao et al., 2026), DeepDive (Lu et al., 2025), and WebExplorer (Liu et al., 2025) synthesize compositional training tasks.
    • Deep search evaluation benchmarks: BrowseComp (Wei et al., 2025) and LoHoSearch (Zhao et al., 2026) evaluate persistent fact finding and relational complexity; DeepSearchQA (Gupta et al., 2026) and DeepWideSearch (Lan et al., 2025) assess the depth and breadth of evidence collection; Java et al. (2026) characterize deep search in the form of claims; BrowseComp-Plus (Chen et al., 2025b) and SAGE (Hu et al., 2026) use controlled corpora; DeepResearch Bench (Du et al., 2026) and II (Li et al., 2026b) evaluate report quality.

    Multimodal evidence deep search

    • Multimodal search methods and evaluation: WebWatcher (Geng et al., 2026) and MMSearch-R1 (Wu et al., 2025b) coordinate visual reasoning with image-text retrieval; MM-BrowseComp (Li et al., 2025b) requires evidence from web images and videos and verifies a multimodal dependency checklist.
    • Video anchoring: Video-DeepResearch (Liu et al., 2026b) and Fang et al. (2026) extend clues to cross-frame distributions and open-web retrieval; Ego2Web (Yu et al., 2026) connects first-person video to online planning.

    Interactive evidence acquisition

    • Website traversal and unindexed information: WebWalker (Wu et al., 2025a) navigates beyond landing-page retrieval through site structure; UIS-Digger (Liu et al., 2026a) extends to dynamic interfaces and embedded documents.
    • Browser agents and cross-site evaluation: Avenir-Web (Li et al., 2026a) and Wuying-Browser-Agent (Chen et al., 2026) strengthen element localization and task state management; GTA (Huang et al., 2026) performs verified task generation; MMInA (Tian et al., 2025) and CAP (Xu et al., 2026) perform cross-site compositional evaluation; Xue et al. (2025) use intermediate visual evidence and action history to assess whether interactions complete the task.

    Positioning of this paper

    The authors state that the above benchmarks do not adequately cover the challenge of diverse modalities, heterogeneous formats, and interactive access requirements being intertwined; OpenDS-World unifies the three into an evidence path and supplements systematic evaluation with real video anchors, human verification, and difficulty calibration. This paper does not set a comparison baseline method; the experiment is a horizontal evaluation of 12 models in a unified environment.

  3. How does the paper address this problem?

    First define the OpenDS task, then construct 200 tasks via "video anchor → open-web evidence expansion → question inversion", with independent verification by ten people and difficulty calibration by testing with 50 people.

    Overall approach: define the OpenDS (Open-Web Deep Search) task formulation, build the OpenDS-World benchmark around three properties of real web pages—modality, format, and access mechanism—and use human verification and human testing to ensure quality and difficulty calibration.

    Task definition and properties

    • Task formulation: Given a factual question and initial clues, the agent autonomously explores the open web, searches, accesses, processes, and connects evidence across information sources with multiple modalities, multiple formats, and multiple access mechanisms, and determines a unique verifiable factual answer; the evidence location and acquisition method are not given in advance.
    • Three main properties: Diverse Modalities (text, image, audio, video), Heterogeneous Formats (HTML, PDF, WebP, MP3, MP4, RSS/XML, Parquet, etc.), Interactive Access Requirements (page navigation, parameter input, temporal indexing, graphical interface operation, structured query, resource download).
    • Access type taxonomy: seven access operations (Table 3): Immediate Observation (O), Hypermedia Transition (N), Parameterized State Update (P), Temporal Indexing (T), Graphical State Manipulation (G), Declarative Query (Q), Artifact Transfer (X); the labels are not mutually exclusive and are counted as a union per task.

    Three stages of task construction

    1. Real-world anchors: expert annotators select 500 candidate public videos from eight domains and analyze consecutive frames and audio to determine verifiable anchor entities; anchor identification must depend on the video's audiovisual content, and the core entity must uniquely correspond to a valid Wikidata entry; videos are archived and distributed with the benchmark and matched to tasks by ID.
    2. Open-web evidence expansion: annotators construct queries based on the current node, use Google Search to obtain candidate sources, check them one by one and select semantically relevant and verifiable sources, taking into account diversity of modality, format, and access type; each node must be corroborated by multiple independent web sources, and expansion proceeds recursively until a unique stable terminal node serves as the reference answer.
    3. Question inversion: the complete evidence path is rewritten in reverse into a set of interdependent constraints, intermediate nodes are not directly exposed, each clue alone does not reveal the target entity, and their combination uniquely determines the answer, so that direct retrieval of explicit clues cannot solve the task.

    Human verification and human testing

    • Human verification: ten independent reviewers review the complete evidence path of each candidate task, checking evidence validity (factually accurate, independently corroborated by multiple sources, anchor uniquely identifiable from the video and not directly obtainable from the video page text) and task validity (clear question, consistent constraints, unique answer); candidate tasks containing factual errors, insufficient evidence, inconsistent relations, ambiguous questions, or non-unique answers are directly excluded, and 200 tasks are ultimately retained.
    • Human testing: the 200 questions are divided into 10 groups of 20 questions each, and each group is answered independently by 5 solvers, for a total of 50 participants; solvers have only the video, audio, and question text, may use a regular browser but are prohibited from using any LLM system, and there is no unified time limit; they submit the final answer, time spent, and a subjective difficulty rating from 0–5, and answers are judged correct or incorrect by humans.
    • Difficulty grouping: grouped by the number of correct answers among the 5 solvers—Low (≥2 correct, 54 questions), Medium (1 correct, 61 questions), High (0 correct, 85 questions) (Table 1).

    Evaluation protocol

    • Unified harness: 16 tools in three categories (Table 4)—Search (text/image/video search, open page), GUI (navigation, click, form filling, scrolling, screenshot, reading downloaded artifacts, etc.), Other (video frame viewing and subtitle extraction); the input video is provided as a read-only file identified as input_video, and video reading samples 10 frames by default, with 1–100 frames per read.
    • Execution budget: the search phase is limited to 3,600 seconds per question, and all tools share an upper limit of 100 calls; context compression retains the most recent 4 tool observations, earlier search results keep only the first 2, and text is truncated to 2,000 characters; the image context limit defaults to 250 images (Table 5).
    • Termination and wrap-up: the research phase terminates when the model produces a complete answer, the budget is exhausted, the output is incomplete, the context overflows, or an execution error occurs; if no complete answer is produced, it enters a tool-free wrap-up phase and answers only based on the collected evidence, with at most 2 requests sharing a 5,000-token output budget.
    • Judging: Qwen3.8-27B is used as the LLM judge, and correctness is judged by semantic equivalence and completeness with respect to the reference answer; the main metric is answer accuracy (ACC), and total token consumption and time spent are also reported; the research and wrap-up prompts are in Appendix C.3, and the judge prompt is in Appendix C.4.
  4. What experiments did the paper conduct?

    Unified evaluation of 12 models: GPT-6 Astra is highest at 39.0%, humans 21.1%; analysis of cost, tool use, errors, and budget shows that more calls do not equal higher accuracy.

    Experimental setup

    • Models tested (12): GPT-6 Astra, GPT-5.6 Sol, Claude Opus 5.5, Claude Fable 5.1, Gemini 3.8 Flash, Qwen3.8-Max, Qwen3.8-Flash, DeepSeek-V4.1-Flash, DeepSeek-V4-Pro, Kimi K3, GLM-5.3-Flash, GLM-5.3; among them, DeepSeek-V4-Pro and GLM-5.3 are marked (T) as text-only models (Table 1).
    • Benchmark: OpenDS-World, 200 tasks, divided into three groups by the number of humans answering correctly: Low (54), Medium (61), High (85).
    • Execution conditions: unified system prompt, tool environment, and budget (3,600 seconds, 100 tool calls); the paper does not report the number of repeated runs.
    • Metrics and judging: the main metric is ACC; cost metrics are total token consumption and total time spent; the judge model is Qwen3.8-27B, judging by semantic equivalence; the paper does not report the consistency between the judge and human judgments.
    • Human comparison: 50 solvers, 1,000 independent attempts, answers judged by humans.

    Main results

    According to Table 1 (Overall is all 200 tasks; Â is total time in hours, Token unit is M):

    Swipe horizontally to read the table

    ModelLow Acc (%)Medium Acc (%)High Acc (%)Overall Acc (%)Â (h)Token (M)
    GPT-6 Astra59.2640.9824.7139.062187
    Claude Opus 5.553.7040.9825.8838.065339
    Gemini 3.8 Flash51.8534.4328.2436.559976
    Claude Fable 5.148.1532.7924.7133.5110481
    GPT-5.6 Sol46.3031.1514.1228.030186
    Qwen3.8-Max40.7426.2315.2925.585839
    GLM-5.3-Flash42.5916.3916.4723.5128389
    DeepSeek-V4.1-Flash33.3313.1117.6520.5120599
    Kimi K327.7814.7512.9417.5110310
    GLM-5.3 (T)22.226.5612.9413.5145345
    DeepSeek-V4-Pro (T)27.7811.483.5312.5148297
    Qwen3.8-Flash22.2213.114.7112.075752
    • Overall level: the authors state that current frontier models are still limited on real web deep search, and even the strongest, GPT-6 Astra, fails 122 of the 200 questions (failure rate 61.00%).
    • Difficulty trend: the average accuracy of the 12 models drops from 39.66% on Low to 23.50% on Medium and 16.76% on High; rankings change with difficulty—GPT-6 Astra leads on Low and ties with Claude Opus 5.5 on Medium, and Gemini 3.8 Flash is highest on High (28.24%).
    • Cost does not correspond to accuracy: GPT-6 Astra and GPT-5.6 Sol have close total tokens (187M vs. 186M) but their overall accuracy differs by 11.00 percentage points; Gemini 3.8 Flash's tokens are about 5.22 times those of GPT-6 Astra while its accuracy is lower.

    Human testing

    • Solvability: 211 of 1,000 attempts were correct, for an overall accuracy of 21.10%; 85 questions (42.50%) were solved by no one, only 1 question (0.50%) was solved by all 5 people, and 29 questions (14.50%) were solved by the majority (≥3 people) (Figure 4(a)).
    • Effort and subjective difficulty: the 50 participants took an average of 12.92 hours (median 9.44 hours) to complete their respective 20 questions, totaling about 646 hours, or about 38.8 minutes per attempt on average; the mean subjective difficulty was 3.884/5, 43.00% of attempts were rated the highest difficulty of 5, and no one rated 0 (Figure 4(b)(c)).

    Cost, tool use, and error analysis

    • Search cost (Figure 6): GPT-6 Astra averages 51.05 calls and 18.66 minutes per question; GPT-5.6 Sol has a similar number of calls (52.42) but only 8.99 minutes; Qwen3.8-Flash averages 89.94 calls, and DeepSeek-V4-Pro and GLM 5.3 average 44.47 and 43.36 minutes with lower accuracy; the authors therefore argue that cost depends on the scale of tool use and on whether low-yield trajectories can be terminated in time.
    • Tool use patterns (Figure 7): GPT-5.6 Sol most favors Search (74.50% of calls), the GUI shares of Claude Opus 5.5 and Qwen3.8-Flash are 51.52% and 49.42% respectively, but their overall accuracies are 38.00% vs. 12.00%; DeepSeek-V4.1-Flash averages 11.78 video/subtitle acquisition calls per question (Claude Opus 5.5 is 4.67), yet its accuracy is lower (20.50% vs. 38.00%); same-type consecutive transitions dominate, with Search→Search highest for Gemini 3.8 Flash at 84.63% and GUI→GUI highest for Qwen3.8-Flash at 84.97%.
    • Termination and failure recovery (Figure 8): natural termination rates are 69.0% for GPT-6 Astra, 65.0% for GPT-5.6 Sol, and 60.5% for Claude Opus 5.5, while Qwen3.8-Flash is only 6.5% (80.0% hit the call limit); failed tool calls account for 2.83%–9.42% of recorded calls but 19.27%–61.90% of tool execution time (GPT-6 Astra is 6.34% and 42.57%); the most common decision after a failure is to switch tools (about 83% for Claude Opus 5.5 and about 81% for Gemini 3.8 Flash), while Kimi K3 and GLM 5.3 retry with changed parameters at about 38% and 37%.

    Other ablations and analyses

    • Budget exhaustion (Figure 9): the proportion of tasks hitting at least one limit ranges from 16.5% for Claude Opus 5.5 to 81.5% for Qwen3.8-Flash; Gemini 3.8 Flash and Qwen3.8-Flash mainly hit the call limit (call-only 37.0%, 80.0%), GLM 5.3 mainly hits the time limit (time-only 41.0%); exhausting both budgets at once is at most 1.0% per model.
    • Domain distribution (Figure 13): GPT-6 Astra is at 51.5% on Society and 50.0% on Nature; Gemini 3.8 Flash is highest on History (36.8%); the cross-model standard deviations for Science and Society are the largest (13.2 and 12.9 percentage points).
    • Modality and format (Tables 6, 7): GPT-6 Astra is highest on Text (38.69%) and Image (48.31%), Gemini 3.8 Flash leads on Video (43.66%) and ties with Claude Fable 5.1 for highest on Audio (50.00%); accuracy on the ZIP format ranges across models from 0.00% to 57.14%.
    • Access types (Table 8): GPT-6 Astra is highest in six of the seven types (X 46.67%, Q 61.54%), Gemini 3.8 Flash is highest in type T (46.32%); type Q ranges across models from 0.00% (Kimi K3) to 61.54%.
    • Video duration (Figure 14): accuracy differs across five groups by duration (36/24/28/43/69 questions), and the authors explicitly state that these differences cannot establish a causal relationship between video duration and difficulty.
    • Long-horizon execution (Figure 15): the median number of tool calls for tasks containing 3, 4, 5, and ≥6 annotated sources is 74, 56, 72, and 82.5, and execution length does not grow monotonically with evidence chain length; models average 42.8–89.9 calls, far higher than the average of 3.94 annotated sources per question.
  5. What are some points that can be further explored?

    The paper does not have a dedicated limitations section; Appendix E gives directions for long-horizon execution and recursive self-improvement; the evaluation covers 12 models and 200 tasks, and judge consistency is not reported.

    Limitations and future directions noted by the authors

    • Long-horizon execution challenges (Appendix E "Natural Long-Horizon Tasks"): the authors state that a compact evidence chain unfolds into a long execution trajectory, and that long-horizon planning, context management, and adaptive tool use are core challenges of real deep search.
    • Recursive self-improvement (RSI) testbed (Appendix E "A Natural Testbed for Recursive Self-Improvement"): the authors state that OpenDS-World exposes diverse execution failures and produces actionable environment feedback that can be used to improve search strategies, tool selection, error recovery, and resource allocation; specific optimization objectives include reducing repeated failures, improving recovery decisions, and prioritizing more effective interactions.
    • Room for improvement at the execution level (Appendix E, Figure 16): termination due to tool budget exhaustion accounts for 15.5%–80.0%, and failed calls account for 19.3%–61.9% of cumulative tool execution latency; the authors state that this highlights the importance of adaptive exploration and termination strategies under a constrained interaction budget.
    • Outlook in the conclusion (Section 5): the authors state that OpenDS-World provides a foundation for studying long-horizon search and recursive self-improvement, with the goal of enabling agents to reliably discover, access, and integrate evidence in real web environments.
    • The paper does not have a separate Limitations subsection.

    Scope of the experiments

    • Models tested: 12 frontier models, of which DeepSeek-V4-Pro and GLM-5.3 are text-only (marked (T) in Table 1).
    • Task scale: 200 tasks and 787 annotated source entries, covering eight domains, four modalities, 42 formats, and seven access types; difficulty grouping is based on 1,000 attempts by 50 human solvers.
    • Execution protocol: a unified budget of 3,600 seconds and 100 tool calls; the paper does not report the number of repeated runs for each model.
    • Judging method: the LLM judge is Qwen3.8-27B, judging by semantic equivalence; the paper does not report an experiment on the consistency between the judge and human judgments.
    • Analysis dimensions: cost, tool use, termination and failure recovery, budget exhaustion, domain, modality, format, access type, video duration; the authors explicitly state that the video duration analysis cannot establish causality (Appendix D.3).
  6. Summarize the main content of the paper

    OpenDS-World: a benchmark of 200 real-web deep-search tasks; 12 frontier models reach a maximum of 39.0%, humans 21.1%, exposing shortcomings in tool coordination and long-horizon search management.

    A Deep Search evaluation benchmark paper addressing the lack of systematic evaluation of end-to-end research capability in real open web environments.

    • Problem: the authors state that evidence on the real web is distributed across sources with multiple modalities, heterogeneous formats, and interactive access requirements, and existing benchmarks do not adequately cover the challenge of these three being intertwined.
    • Method: define the OpenDS task and build OpenDS-World—starting from real video anchors, 200 tasks are generated via open-web evidence expansion and question inversion, ten reviewers independently verify evidence validity and answer uniqueness, and 50 human solvers (1,000 attempts) calibrate difficulty; evaluation uses a unified harness (16 tools, 3,600 seconds, 100-call limit), with Qwen3.8-27B as the judge.
    • Main results: GPT-6 Astra has the highest overall accuracy at 39.0% (200 questions), and Claude Opus 5.5 is at 38.0%; human independent solving accuracy is 21.1%, and 42.5% of tasks were solved by no one; the average accuracy of the 12 models drops with difficulty from 39.66% on Low to 16.76% on High.
    • Behavioral findings: more tool calls and longer time spent do not correspond to higher accuracy (Gemini 3.8 Flash's tokens are about 5.22 times those of GPT-6 Astra while its accuracy is lower); failed tool calls account for only 2.83%–9.42% but consume 19.27%–61.90% of tool time; budget exhaustion is common for some models (81.5% of tasks hit the limit for Qwen3.8-Flash).
    • Authors' conclusion: current agents still face persistent challenges in information source selection, tool coordination, search strategy adjustment, and long-horizon execution management; the authors state that OpenDS-World can serve as a unified evaluation platform for studying long-horizon search and recursive self-improvement.
Read originalhuggingface.co