Generality Labs audits LABBench2, finding most task sets saturated and graders unreliable

Generality Labs audited the 11 task sets of the biological capability benchmark LABBench2, concluding that most literature task sets are saturated and ProtocolQA2 scores are hard to trust. The benchmark has 1,912 tasks in total; five of the seven literature task sets are saturated or near-saturated, and the molecular biology SeqQA2 is also saturated. The audit points out that ProtocolQA2's construct validity is questionable: across four grading models, pass rates ranged from 47% for GPT-5.6 Terra to 59% for Gemini 3.7 Flash, a 12-percentage-point difference. When DbQA2 was moved from the benchmark's tools setting to an agent sandbox, GPT-5.6 Luna rose from 42% to 73% and Terra from 45% to 70%. CloningQA has only 14 questions, making it unusable for cross-model comparisons.