OpenDS-World benchmark: 12 frontier models score at most 39%
Start here
A team from the Hong Kong University of Science and Technology (Guangzhou), Fudan University and other institutions proposes the OpenDS-World benchmark in an arXiv preprint, containing 200 real-web research tasks: each task starts from a video clue and requires the agent to discover and connect evidence across real websites, with an independently verified evidence chain and a unique factual answer. The best of the 12 tested frontier models answers only 39% correctly, and the paper states that this does not constitute a general ranking of research assistants.
Generated by AI from reports · Updated 4 days ago
Attention and developments
Attention, hourly
Now 34, 0% in 24 hours·Peak 37 (Nov 15 21:00)
Storyline
- · 1 reports· Attention38OpenDS-World benchmark released, 12 models top out at 39% accuracy
- · 1 reports· Attention34
OpenDS-World benchmark: 12 frontier models score at most 39%This event
Progress timeline
- X @jiqizhixin (Synced)Among 12 frontier models on OpenDS-World's 200 real-web research tasks, the best result was only 39%
A team from institutions including the Hong Kong University of Science and Technology (Guangzhou), Fudan University, the University of Washington, and Imperial College London proposed the OpenDS-World benchmark in the arXiv preprint Deep Search in Real-World Web Environments; among the 12 frontier models tested, the best result was only 39% of 200 real-web research tasks answered correctly. Each task starts from a video clue and requires the agent to discover, access, and connect evidence across real websites, with information possibly hidden in interactive pages, audiovisual materials, scientific charts, or structured files; tasks come with independently verified evidence chains and unique factual answers, rather than preset tool-call sequences. All models share the same tool environment, with a search budget of 3600 seconds or 100 tool calls, whichever is exhausted first. The text states that this accuracy reflects only this benchmark and execution protocol and does not constitute a general ranking of research assistants. The paper is at https://arxiv.org/abs/2610.30181,代码开源于 https://github.com/Yu-xm/OpenDS。
Related discussion
- Older than 72 hours · Not included in the 72-hour figures
- X@SeiraHisadaNovember 15, 2026 (Sun.) summary from the XGBN AI News evening edition 【What does AI learn in the field?】 Tonight, research that “runs AI to verify it” stood out. Robots learn from another body's success cases, agents redo work to compare whether settings are good or bad…(opens the original post in a new tab)
Attention trend
New sources per day
- Nov 15: 2 new sources (1 outlets, 1 accounts)
- Nov 16: 0 new sources (0 outlets, 0 accounts)
- Nov 17: 0 new sources (0 outlets, 0 accounts)
- Nov 18: 0 new sources (0 outlets, 0 accounts)
- Nov 19: 0 new sources (0 outlets, 0 accounts)
- Nov 20: 0 new sources (0 outlets, 0 accounts)