Skip to content
机器之心 JIQIZHIXIN· @jiqizhixin · X·Indexed by this site · Originally published· AI updates →· Its place in the scan record →

Among 12 frontier models on OpenDS-World's 200 real-web research tasks, the best result was only 39%

AI summary

A team from institutions including the Hong Kong University of Science and Technology (Guangzhou), Fudan University, the University of Washington, and Imperial College London proposed the OpenDS-World benchmark in the arXiv preprint Deep Search in Real-World Web Environments; among the 12 frontier models tested, the best result was only 39% of 200 real-web research tasks answered correctly. Each task starts from a video clue and requires the agent to discover, access, and connect evidence across real websites, with information possibly hidden in interactive pages, audiovisual materials, scientific charts, or structured files; tasks come with independently verified evidence chains and unique factual answers, rather than preset tool-call sequences. All models share the same tool environment, with a search budget of 3600 seconds or 100 tool calls, whichever is exhausted first. The text states that this accuracy reflects only this benchmark and execution protocol and does not constitute a general ranking of research assistants. The paper is at https://arxiv.org/abs/2610.30181,代码开源于 https://github.com/Yu-xm/OpenDS。

Related papers
Full text · Original

The best of 12 evaluated frontier models answers just 39% of OpenDS-World's 200 real-Web research tasks correctly.

A team spanning HKUST (Guangzhou), Fudan University, the University of Washington, Imperial College London and other institutions introduces the benchmark in an arXiv preprint, Deep Search in Real-World Web Environments.

Each task starts with a video clue and requires agents to discover, access and connect evidence across live websites. Information can sit inside interactive pages, audiovisual material, scientific figures or structured files. Independently verified evidence chains support unique factual answers, rather than a predefined sequence of tool actions.

All models share one tool environment and a search budget of 3,600 seconds or 100 tool calls, whichever runs out first. Finding a page is only part of the problem: choosing useful sources, recovering from failed interactions and deciding when the evidence is sufficient remain difficult. The reported accuracy describes this benchmark and execution protocol; it does not establish a universal ranking of research assistants.

Deep Search in Real-World Web Environments

Paper: https://arxiv.org/abs/2610.30181
Code: https://github.com/Yu-xm/OpenDS
Our Report: https://www.jiqizhixin.com/articles/2026-10-30-12

📬 #PapersAccepted by Jiqizhixin

Source: 机器之心 JIQIZHIXIN · x.com