Skip to content
WeChat · Synced (Jiqizhixin)·Indexed by this site · Originally published· AI updates →

OpenDS-World benchmark: Deep Search enters the real internet, strongest model GPT-6 Astra reaches only 39% accuracy

最强模型准确率仅39%!当Deep Search真正进入复杂的真实互联网

AI summary

A research team from institutions including The Hong Kong University of Science and Technology (Guangzhou), Fudan University and the University of Washington proposed OpenDS-World, a deep search benchmark for the real open web, in the arXiv paper "Deep Search in Real-World Web Environments"; among 12 evaluated frontier models, the strongest, GPT-6 Astra, reached only 39.0% accuracy. The benchmark contains 200 tasks covering 4 modalities, 42 resource formats and 7 types of evidence access requirements, and uses an approach of first constructing evidence chains and then reverse-generating questions; models must search autonomously within a budget of 1 hour and 100 tool calls and give verifiable factual answers. Claude Opus 5.5 and Gemini 3.8 Flash achieved 38.0% and 36.5% respectively; 50 human participants made 1,000 independent attempts in total, of which only 211 were correct, and 85 questions were solved by no one. The paper proposes that the benchmark can be used to study agents' long-horizon decision-making and recursive self-improvement (RSI).

Related papers
Read originalmp.weixin.qq.com