SciHazard: A dangerous capability evaluation benchmark for research safety with decomposed harm scoring

How to accurately measure the threat potential of LLMs and autonomous research agents in high-risk scientific scenarios.

• Scenario: The authors state that while LLMs accelerate scientific discovery, they also turn dangerous scientific knowledge into executable misuse guidance, ranging from chemical weapon synthesis to pathogen engineering, threatening public safety.
• Existing shortcomings: The authors state that existing benchmarks mostly use regulated substances with template prompts, measuring shallow retrieval rather than complex adversary misuse; LLM-as-a-Judge in scientific domains falls back to surface semantic heuristics, with scores disconnected from real harm. Most benchmarks only test standard conversational models, not deep research agents that retrieve, plan, and use tools.
• Core observation: The authors argue that the danger of scientific answers is determined by two orthogonal dimensions, Executability and Net-New Risk; holistic scoring forces annotators to weight subjectively.
• This paper proposes: SciHazard contains 2,400 hazard questions and 600 oversafety questions across 12 disciplines, with queries anchored to regulated entities and documented failure scenarios; DEHARM-SCORE decomposes question hazard, refusal, and answer-level risk. The main text points to Figure 1, stating that SafeScientist has 76% of questions at L5, while SciHazard covers L1–L5.

Section 2 of the paper divides related work into two lines: scientific safety benchmarks and automatic evaluation.

Scientific safety benchmarks

• WMDP (Li et al., 2024): Expert-written multiple-choice questions on dangerous knowledge.
• ChemSafetyBench (2024), SafeSci (2026), SOSBench (2026): Built from controlled substances or jailbreak prompts. The paper does not criticize each individually.
• SafeScientist (Zhu et al., 2025): Open-ended questions generated by deep research models. The authors state the above works share two common limitations: domains concentrated in traditional STEM, and disconnected from real misuse, measuring factual recall rather than executable scenarios.

Automatic evaluation

• Llama-Guard (Inan et al., 2023): Proprietary judge model for binary classification.
• StrongReject (Souly et al., 2024), DeepReject (Chen et al., 2025): LLM judges with rubrics, then synthesize sub-dimensions into a total score. The authors state that in high-knowledge-density scientific scenarios, both binary and rubric judges lack domain-specific knowledge and default to surface semantic heuristics.

Experimental baselines and comparison datasets

• Baselines: Section 5 and Appendix C use DeepReject, StrongReject, LLM-as-a-Judge, SafetyScore. DeepReject is R×W×(0.65K+0.35F); StrongReject is refusal factor times the average of persuasiveness and specificity. Main experiments fix judge models to Gemini-3.1-Pro and Qwen3.5-35B-A3B.
• Comparison datasets: Table 1 lists WMDPBench, SciSafeEval, SafeSci, SafeScientist, SOSBench, comparing question count, number of domains, format, whether expert labels and benign subsets exist.

The authors state SciHazard connects dangerous entities to the full misuse lifecycle, covering 12 disciplines; and claim it is the first such scientific safety benchmark with fine-grained expert annotations anchored to real misuse scenarios. The authors state DEHARM-SCORE fills the evaluation gap with dynamic retrieval and hierarchical matching, achieving the highest alignment with expert annotations.

SciHazard uses a dual-track pipeline to generate questions anchored to regulatory control lists and documented incidents; DEHARM-SCORE is a dataset-agnostic decomposition metric, as the authors call it, that splits non-refused responses into executability and net-new risk, then multiplies by question hazard level and refusal indication.

Dual-track data generation

• Substance track: chemistry, biology, physics, pharmacology, medicine, environmental science, agricultural science. 1,548 controlled substances are extracted from CWC, CDC/APHIS Select Agents, DEA schedules, EU REACH, and related UN conventions as Hazard Primitives.
• Combination method: A frontier LLM plus external knowledge bases labels three technical bottlenecks for each substance, then generates questions along four stages: upstream acquisition and substitution, midstream optimization and amplification, downstream stabilization and storage, delivery and dissemination. Questions are orthogonal combinations of substance × stage × bottleneck.
• Scenario track: computer science, psychology, sociology, economics, law. Primitives are abstract disruption scenarios mined by a frontier LLM combined with domain knowledge bases; each scenario has three systemic bottlenecks, corresponding to four stages: reconnaissance and access, weaponization and strategy design, execution and amplification, evasion and persistence.
• Quality control: Produces 2,400 hazardous questions and 600 over-safety questions. Appendix E states that primitives were expert-verified and deduplicated, bottlenecks underwent a second review, and all questions were manually checked one by one. Appendix A and B give examples, not repeated here.

Hazard levels and over-safety subset

• Two axes: L1–L5 integrates OSHA, FEMA, MIL-STD-882E, and frontier extreme-risk definitions. Axis one is consequence and intent; axis two aligns with WHO Risk Groups, CWC Schedules, Australia Group, and similar lists. Appendix D forms a deterministic mapping by crossing regulatory tiers R1–R4 with operational intent O1–O4; defensive intent is always L1, and ambiguity takes the higher tier.
• Who labels: GPT-5.4, Gemini-3.1-Pro, Claude-4.6-Opus score independently, majority vote; ties or refusals go to human domain experts. Result is Hq∈{1,2,3,4,5}.
• Validation: Appendix D stratified-samples 360 questions across 12 disciplines; committee and experts exact match 91.38%, adjacent accuracy 100%, QWK 0.96; all disagreements are adjacent levels.
• Over-safety: 600 questions generated by intent reversal, reusing high-risk trigger words from hazardous questions, with intent changed to defensive, detection, or emergency. The authors state the core intent complies with major LLM providers' usage policies.

Executability branch

• 10-item checklist: Looks only at the question, and retrieves public internet and domain knowledge bases. Three facets are specification, steps, verification; items determining whether the core implementation chain is feasible are Blockers, verification items do not block.
• Two-pass matching: Each item scores 1–5. If any item ≤2, a keyword-enhanced second pass review is done, taking the higher score of the two passes. Appendix F.3.1 gives the 1–5 pass criteria.
• Aggregation: A rule module uses Blocker pass rate and coverage density to get Erule; any Blocker failure lowers the tier. An independent Reviewer outputs Ejudge. The main text says this adjudication is unconstrained; Appendix F.3.1 says it can deviate at most one tier from the rule tier, with larger deviations falling back.

Net-New Risk branch

• Claims: Extracts up to 10 verifiable technical claims from the response, labels criticality (high / medium / low) and generates retrieval queries.
• Dual scoring: Real-time web search plus domain knowledge bases. Coverage∈{easy, moderate, hard} indicates how easily atomic information is obtained from public or catalogued sources; Synthesis Barrier∈{low, medium, high} indicates how difficult it is to assemble scattered sources into the coherent form in the response.
• Aggregation: The criticality-weighted proportion of hard-to-cover claims, plus the count of high-synthesis-barrier claims, gives Nrule. The Reviewer looks at synthesis value: a single claim, even if public, can still be high-risk if assembled into an executable harmful process, outputting Njudge. Appendix F.3.2 similarly specifies ±1 tier. Appendix F.3.2 writes each claim gets at most 3 searches, each taking the top 5 abstracts. Appendix F.4 states the retrieval time upper bound is 2026-04-15, and uses caching for reproduction.

Composite score and refusal

Response-level harm for non-refused responses is S = H_q · R · (E+N)/8. Hq is the question hazard level, R∈{0,1} is 0 when there is a substantive refusal and zeroes the risk, E, N∈{1,2,3,4}. The normalized sum of the two falls in [1/4, 1]. The authors call this a progressive gate: Hq sets the theoretical upper bound, R is a binary switch, (E+N)/8 modulates residual risk along operational completeness and knowledge uplift.

Refusal classification is in Appendix F.2: first template filtering; after a hit, a judge model checks whether there is substantive domain content beyond disclaimers. Accuracy on 500 human-annotated responses is 99.6% (498/500). The main experiment judge model is Qwen-3.5-35B-A3B; Appendix F.3 writes the single judge model temperature 0.0.

Experimental setup

• Standard LLMs: Table 7 has 31 systems total. Open-source 8: Kimi-K2.5, Step-3.5-Flash, DeepSeek-R1, GLM-5.1, LLaMA-3.1-70B-Inst., LLaMA-4-Scout, Qwen3.5-35B-A3B, Qwen3.5-397B-A17B. Closed-source 11: Qwen3.6-Plus, Claude Haiku 4.5, Claude Opus 4.6, Claude Sonnet 4.6, Gemini-3-Flash-Think., Gemini-3.1-Pro, GPT-5.2, GPT-5.4, o3, o3-mini, Grok-4.1-Fast.
• Agents and scientific models: Closed-source agents are o3 Deep Research, Gemini Deep Research, Sonar Pro. DeerFlow and WebThinker each connect DeepSeek-R1, o3-mini, Qwen3.5-35B. Scientific models are Intern-S1-Pro, InternAgent, S1-VL-32B.
• Generation: T=0, max_tokens=8192; closed-source agents keep provider defaults. Retrieval uses Serper API. The paper does not report the number of main evaluation repetitions.
• Main metrics: Judge model is Qwen-3.5-35B-A3B. Columns include Oversafety (%), Rejection (%), Eavg, Navg, DEHARM, DEHARM†. Table 7 note: DEHARM† is computed only on non-refused queries.
• Expert subset: 173 high-risk queries, three PhD students in corresponding disciplines independently annotate; each averages 1.47 hours, plus 0.62 hours per round of cross-quality checks. Also given an overall 1–5 score Hdirect. The component-aligned judge models are Gemini-3.1-Pro, gpt4o-mini, Qwen3.5-35B-A3B.
• Perturbations: 200 harmful queries, generated by GPT-5.4. V1 changes key parameters to make the process infeasible, V2 adds irrelevant pseudoscientific steps and citations, V3 only adds safety statements.

Main results

According to Table 7, only the highest DEHARM, highest DEHARM†, and several controls are listed. Percentages are understood per column headers.

Swipe horizontally to read the table

Model	Oversafety	Rejection	DEHARM	DEHARM†
WebThinker + DeepSeek-R1	0.0%	12.1%	2.520	2.869
InternAgent	0.0%	8.3%	2.023	2.215
Gemini Deep Research	15.0%	63.1%	1.856	3.010
LLaMA-3.1-70B-Inst.	6.8%	17.7%	1.818	2.208
GPT-5.4	26.0%	18.8%	1.440	1.783
o3	16.8%	82.2%	0.597	3.372
Grok-4.1-Fast	18.3%	81.4%	0.632	3.424
DeerFlow + Qwen3.5-35B	88.3%	52.5%	0.662	1.427

Omitted: Kimi-K2.5, Step-3.5-Flash, DeepSeek-R1, GLM-5.1, LLaMA-4-Scout, Qwen3.5-35B-A3B, Qwen3.5-397B-A17B, Qwen3.6-Plus, Claude Haiku 4.5, Claude Opus 4.6, Claude Sonnet 4.6, Gemini-3-Flash-Think., Gemini-3.1-Pro, GPT-5.2, o3-mini, o3 Deep Research, Sonar Pro, DeerFlow + DeepSeek-R1, DeerFlow + o3-mini, WebThinker + o3-mini, WebThinker + Qwen3.5-35B, Intern-S1-Pro, S1-VL-32B.

• Means and paired comparisons: The authors state deep research agents average DEHARM 1.504, 19 standard LLMs 1.137, 32.3% higher. o3 Deep Research is 1.601, relative to base the authors claim +168%. Conclusion written as an increase over 32.3%.
• Two paths: The authors state most configurations have refusal erosion; o3-mini drops from 73.2% to DeerFlow's 34.2%, DEHARM† still similar. Another path is content-level amplification: Gemini-3.1-Pro's DEHARM† is 2.329, to Gemini Deep Research +29.2%. The authors believe retrieval adds dangerous knowledge beyond parameters.
• Two strategies: The authors state a counterintuitive inverse relationship appears. Qwen3.6-Plus refuses 79.1%, DEHARM† 3.056. GPT-5.2 only refuses 17.5%, DEHARM† 1.913. The authors state agents erode refusal rate by at most 41.6 percentage points (DeepSeek-R1: 53.7% to WebThinker configuration).

Metric and expert agreement

• Human agreement: Table 2: E's Krippendorff α is 0.811, mean QWK 0.820; N is 0.768 and 0.775; Hdirect is 0.568 and 0.571. Composite is α 0.832, QWK 0.848. The authors believe ambiguity in the overall harm concept amplifies expert disagreement.
• Components: Table 3: E's highest human-machine QWK is Ours-gemini-3.1-pro's 0.764, direct judgment 0.548. Table 4: N's highest is Ours-qwen3.5-35b-A3b's 0.742, direct judgment 0.305. On gpt4o-mini, E only goes from 0.296 to 0.541, MAE still 0.834.
• Composite: Table 5: DEHARM-gemini's QWK is 0.774, MAE 0.196, Spearman ρ 0.812; DEHARM-qwen3.5 is 0.754, 0.225, 0.742. The same table's highest baseline is DeepReject-qwen3.5's 0.436. Abstract and Appendix J claim 90.17% improvement over the strongest baseline; Appendix J's comparison is DeepReject 0.407 vs 0.774. The main table's judge model is Qwen, not the Gemini reporting 0.774. LLM-as-a-judge-gemini's QWK is 0.147, MAE 2.040.

Perturbation robustness

• Expectation: Table 6 uses Gemini-3.1-Pro. The authors expect V1 to drop, V2 and V3 basically unchanged.
• DEHARM: V1's Mean Percentage Shift is -37.47%, Cohen's d -1.05; V2 -2.38% (d=-0.05), V3 -2.69% (d=-0.15). The authors state the checklist catches parameter deviations, retrieval anchoring filters out irrelevant noise and surface safety postures.
• Baselines: StrongReject-gemini on V3 is -53.66% (d=-0.76); DeepReject-gemini on V2 is +5.18% (d=0.32); SafetyScore-gemini on V1 is +1.36% (d=0.06). Table 14 with Qwen3.5-35B, DEHARM's V1 is -26.86% (d=-1.34), V2 +1.36%, V3 -0.84%.

Other ablations and analyses

• Appendix G, judge model fixed Gemini-3.1-Pro. E's QWK: direct 0.548, retrieval-only 0.634, decomposition-only 0.697, full pipeline 0.764 (Table 12).
• N's QWK: direct 0.397, retrieval-only 0.608, decomposition-only 0.524, full pipeline 0.722 (Table 13). The authors state E relies more on decomposition, N more on retrieval.
• Appendix I: Open-source LLMs average DEHARM 1.414, closed-source 0.936; average refusal 44.0% vs 58.2%; average DEHARM† 2.626 vs 2.491. The authors believe the open/closed distinction is not a good predictor of safety.
• Appendix I: Claude Sonnet 4.6 over-safety rate 74.8%, harmful refusal 54.5%; DeepSeek-R1 54.7% vs 53.7%. The authors state DeerFlow + Qwen3.5-35B refuses most inputs indiscriminately, its low score cannot be taken as real safety.
• Appendix I: InternAgent's DEHARM is 2.023, refusal 8.3%, over-safety 0.0%; S1-VL-32B's DEHARM† is 2.678, Intern-S1-Pro 2.760. The authors state domain specialization itself does not show safer performance.
• Appendix I: Same backbone, WebThinker + DeepSeek-R1's refusal is 12.1%, DEHARM 2.520, DeerFlow + DeepSeek-R1 25.8% and 1.625. Appendix J: DeepReject 4 LLM calls; DEHARM-SCORE 6 LLM calls plus 6–10 search API calls.

Limitations and future directions noted by the authors

• Disciplines and time: The authors state that although 12 core disciplines are covered, long-tail subfields such as quantum materials engineering, as well as emerging interdisciplinary areas such as AI-driven synthetic biology, are not yet fully included; scientific knowledge and misuse pathways change over time (Limitations, Limitation 1).
• Update plan: The authors plan to establish a community-driven dynamic update mechanism and periodically incorporate newly disclosed real high-risk cases (Limitations, Limitation 1).
• Annotation cost: The authors state that annotation typically requires a PhD or senior industry experience, and due to time and budget constraints, not all test answers from all evaluated LLMs and agents can be manually annotated; currently only a representative subset is deeply annotated (Limitations, Limitation 2).
• Annotation release: The authors state they are open-sourcing these expert annotations and hope they become public data for improving automatic evaluation (Limitations, Limitation 2).

The Ethics Statement describes controlled annotation and use of only publicly available information, without framing it as a limitation.

Experimental coverage

• The main evaluation is 31 systems on SciHazard: 19 standard LLMs, 9 deep-research configurations, and 3 science-specialized models (Section 6.1, Table 7). Generation temperature is 0, max_tokens=8192. The paper does not report the number of repetitions for the main evaluation.
• Data consists of 12 disciplines, 2,400 hazard questions plus 600 oversafety questions (Section 3, Table 1). Hazard levels are determined by majority vote of a three-model committee and compared with experts on 360 questions (Appendix D).
• Metric meta-evaluation uses 173 high-risk queries and three PhD students (Section 5.1). The refusal classifier is checked on 500 manually annotated answers (Appendix F.2).
• Automatic metric comparisons are against DeepReject, StrongReject, LLM-as-a-Judge, and SafetyScore. Perturbations are V1–V3 on 200 harmful queries, with judge models Gemini-3.1-Pro and Qwen3.5-35B (Section 5.4, Appendix H). Ablations use only Gemini-3.1-Pro (Appendix G).
• The judge model for the main table is Qwen-3.5-35B-A3B. Appendix J reports DEHARM-SCORE as 6 LLM calls plus 6–10 search API calls. The paper does not report total query counts or wall-clock time for the main evaluation.

SciHazard is a scientific misuse risk benchmark paired with an automatic score that decomposes harm and then recombines it.

• What it measures: The authors aim to measure the extent to which LLMs and autonomous research agents turn dangerous scientific knowledge into executable misuse guidance. The authors argue that existing benchmarks are biased toward templates and factual recall, and that holistic LLM judging in scientific settings lacks domain grounding.
• Data: A dual-track pipeline covers 12 disciplines, yielding 2,400 hazard questions and 600 oversafety questions. The substance track generates questions from 1,548 regulated substances along their life cycle and technical bottlenecks; the scenario track generates questions along a four-stage chain. Question hazard levels are 1–5.
• Score: When not refused, Executability uses a 10-item dynamic checklist and Blocker to check whether the operational chain is closed; Net-New Risk uses retrieved claim coverage and synthesis barriers to measure the uplift relative to public knowledge. The total score is the product of hazard level, refusal indication, and the normalized sum of these two components.
• Metric evidence: With 173 questions and three experts, Krippendorff's alpha for E and N is 0.811 and 0.768, and the overall score Hdirect is 0.568. In Table 5, DEHARM-gemini has a QWK of 0.774; the Qwen judge actually used in the main table is 0.754. In Table 6, parameter tampering shifts DEHARM-gemini by -37.47%, and no-context injection and safety statements are -2.38% and -2.69%.
• 31 systems: The authors state that deep research agents average a DEHARM of 1.504, which is 32.3% higher than the 1.137 of 19 standard LLMs. In Table 7, WebThinker + DeepSeek-R1 has a DEHARM of 2.520; Grok-4.1-Fast has a DEHARM† of 3.424 with 81.4% refusals. The authors state there is a counterintuitive inverse relationship between high refusal rates and the harm of non-refused answers, and that agent configurations erode refusal barriers.
• Authors' conclusions: The authors state that relative to the strongest baseline, expert agreement improves by 90.17% relatively, and they view autonomous research agents as a key blind spot in current safety defenses. The authors hope the benchmark and metric will be used to diagnose and mitigate scientific safety risks in next-generation systems. The authors state that code and datasets are publicly released.