LLM capability uplift for novices on dual-use computational biology tasks
LLM Novice Uplift on Dual-Use, In Silico Biology Tasks
Quick read
Based on 论文全文 （AI 生成）; refer to the original
A multi-model, multi-benchmark human uplift study shows that novices with access to multiple frontier LLMs achieve 4.16 times higher accuracy (95% CI [2.63,6.87]) than an internet-only control group across eight biosecurity-related tasks. The study recruited both non-STEM and STEM novices, with tasks lasting up to 13 hours; the treatment group had access to models including o3, o4-mini, Gemini 2.5 Pro, Claude 3.7 Sonnet, Claude Opus 4, and Gemini Deep Research. On the four benchmarks with expert baselines, LLM-assisted novices exceeded experts on three of them; however, standalone LLMs often outperformed LLM-assisted novices, which the authors attribute to users not fully leveraging model capabilities. 89.6% of treatment-group participants reported encountering almost no guardrail obstruction when accessing dual-use information.
- 大语言模型在生物学基准上表现越来越好，但它们能否真正提升新手（novice）的能力、使其超过仅用互联网资源的水平，尚不清楚。这一问题关系到科学加速与两用（dual-use）生物风险，因为此前需要专家协助的任务可能被新手完成。
- 作者开展了一项多模型、多基准的人类 uplift 研究，比较新手在有 LLM 访问（Treatment）与仅互联网（Control）两种条件下的表现。参与者为生物学新手，任务涵盖八个生物安全相关基准，最长可达 13 小时，Treatment 组可使用 o3、Gemini 2.5 Pro、Claude Opus 4 等多个模型，并收集纵向数据与定性行为编码。
- 有 LLM 访问的新手准确率约为对照组的 4.16 倍（95% CI [2.63,6.87]），在 8 个基准中的 7 个上优于对照组，并在有专家基线的 4 个基准中的 3 个上超过专家。但独立 LLM 往往优于 LLM 辅助的新手，且 89.6% 的参与者表示几乎未遇到获取两用相关信息的困难。
- 研究期间可用模型集合不一致（Claude Opus 4 中途发布），部分参与者找到了 HLE 和 LAB-Bench 的在线题目，缺乏双盲可能引入受试者期望效应，Long-Form Virology 存在提示缺失与模型拒答问题。作者建议未来研究如何优化人类贡献、LLM 集成互相约束，以及这些动态如何迁移到真实湿实验室环境。
The study quantifies the capability uplift of LLMs for biology novices through human-machine collaboration experiments lasting up to 13 hours, and provides control baselines of experts and models answering alone.