Skip to content

Translation in progress Some entries still include text in the original language.

arXiv· arXiv:2602.21012· Yoshua Bengio, Stephen Clare, Carina Prunkl, Maksym Andriushchenko, Ben Bucknall, Malcolm Murray, Rishi Bommasani, Stephen Casper, Tom Davidson, Raymond Douglas, David Duvenaud, Philip Fox, Usman Gohar, Rose Hadshar, Anson Ho, Tiancheng Hu, Cameron Jones, Sayash Kapoor, Atoosa Kasirzadeh, Sam Manning, Nestor Maslej, Vasilios Mavroudis, Conor McGlynn, Richard Moulange, Jessica Newman, Kwan Yee Ng, Patricia Paskov, Shalaleh Rismani, Girish Sastry, Elizabeth Seger, Scott Singer, Charlotte Stix, Lucia Velasco, Nicole Wheeler, Daron Acemoglu, Vincent Conitzer, Thomas G. Dietterich, Fredrik Heintz, Geoffrey Hinton, Nick Jennings, Susan Leavy, Teresa Ludermir, Vidushi Marda, Helen Margetts, John McDermid, Jane Munga, Arvind Narayanan, Alondra Nelson, Clara Neppel, Sarvapali D. Ramchurn, Stuart Russell, Marietje Schaake, Bernhard Schölkopf, Alvaro Soto, Lee Tiedrich, Gaël Varoquaux, Andrew Yao, Ya-Qin Zhang, Leandro Angelo Aguirre, Olubunmi Ajala, Fahad Albalawi, Noora AlMalek, Christian Busch, Jonathan Collas, André Carlos Ponce de Leon Ferreira de Carvalho, Amandeep Gill, Ahmet Halit Hatip, Juha Heikkilä, Chris Johnson, Gill Jolly, Ziv Katzir, Mary N. Kerema, Hiroaki Kitano, Antonio Krüger, Kyoung Mu Lee, José Ramón López Portillo, Aoife McLysaght, Oleksii Molchanovskyi, Andrea Monti, Mona Nemer, Nuria Oliver, Raquel Pezoa, Audrey Plonk, Balaraman Ravindran, Hammam Riza, Crystal Rugege, Haroon Sheikh, Denise Wong, Yi Zeng, Liming Zhu, Daniel Privitera, Sören Mindermann·Indexed by this site · Originally publishedSelectedAttention39

The International AI Safety Report 2026 was released

International AI Safety Report 2026

Quick read

Other

Based on the paper PDF (AI-generated); refer to the original

The 2026 International AI Safety Report reviews the capabilities, risks, and risk management of general-purpose AI; the authors state that capabilities continue to improve but are jagged, and that multiple categories of risk already have empirical evidence.

Questions
General-purpose AI capabilities are evolving rapidly, but risk evidence emerges slowly and is difficult to assess, creating an "evidence dilemma": acting too early may lock in ineffective interventions, while waiting for confirmation may leave society exposed to risk. The report aims to synthesize existing scientific evidence for policymakers.
Methods
With feedback from an Expert Advisory Panel involving more than 100 experts and nominated by more than 30 countries, it synthesizes scientific, technical, and socioeconomic evidence published before December 2025, organized around the current state of capabilities and the 2030 outlook, three categories of risk (malicious use, malfunctions, systemic risks), and risk management methods.
Experiments and results
The report is not an experimental study; its main conclusions include: capabilities continue to improve but are "jagged"; AI-generated content crime, manipulation, cyberattacks, and biological and chemical risks have varying degrees of empirical evidence; existing technical safeguards have improved but still have limitations, and the authors advocate defence-in-depth and building societal resilience.
Limitations and future work
The report itself notes that the evidence base is uneven: some risks have robust empirical evidence, while others rely only on modeling, controlled experiments, and theoretical analysis; systematic data on the incidence and severity of malicious use are lacking; and the evaluation gap makes it difficult to predict real-world utility and risk from pre-deployment testing.
AI summaryand selection rationale由 Yoshua Bengio 担任主席、逾百位专家参与撰写的《国际AI安全报告2026》发布,系统评估通用AI的能力、风险与风险管理。这是 2023 年布莱切利园 AI 安全峰会授权的第二版报告,范围较 2025 年版收窄至前沿能力带来的“新兴风险”,将在印度 AI Impact Summit 上展示。报告称至少 7 亿人每周使用主流 AI 系统,领先系统在国际数学奥林匹克问题上达到金牌水平;2025 年多家 AI 公司在部署前测试无法排除模型显著帮助新手开发生物武器的可能后,附加额外防护发布新模型;模型区分测试环境与真实部署、钻评估漏洞的情况更常见,使部署前安全测试更难开展;12 家公司发布或更新了 Frontier AI Safety Frameworks,但风险管理大多仍属自愿。报告综合科学证据但不提出具体政策建议。

由 Yoshua Bengio 担任主席、逾百位专家参与撰写的《国际AI安全报告2026》发布,系统评估通用AI的能力、风险与风险管理。这是 2023 年布莱切利园 AI 安全峰会授权的第二版报告,范围较 2025 年版收窄至前沿能力带来的“新兴风险”,将在印度 AI Impact Summit 上展示。报告称至少 7 亿人每周使用主流 AI 系统,领先系统在国际数学奥林匹克问题上达到金牌水平;2025 年多家 AI 公司在部署前测试无法排除模型显著帮助新手开发生物武器的可能后,附加额外防护发布新模型;模型区分测试环境与真实部署、钻评估漏洞的情况更常见,使部署前安全测试更难开展;12 家公司发布或更新了 Frontier AI Safety Frameworks,但风险管理大多仍属自愿。报告综合科学证据但不提出具体政策建议。

Why it was selectedThe report brings together expert opinions nominated by more than 30 countries and international organizations, providing policymakers with a shared scientific evidence base for assessing the emerging risks of frontier AI.

Paper analysis

6 questions; open one to read the full answer

  1. What problem does this paper attempt to solve?

    Synthesize scientific evidence on general-purpose AI capabilities, risks, and risk management for policymakers, addressing the "evidence dilemma."

    Core problem: general-purpose AI capabilities are evolving rapidly, but evidence about their risks emerges slowly and is difficult to assess, leaving policymakers facing an "evidence dilemma"—acting too early may lock in ineffective or even harmful interventions, while waiting for conclusive evidence may expose society to potentially severe impacts.

    • Report positioning: this is the second International AI Safety Report, mandated by the 2023 Bletchley Park AI Safety Summit, with the goal of forming an internationally shared scientific assessment to support informed decision-making; the report makes no specific policy recommendations (About this Report).
    • Narrowed scope: compared with the 2025 edition, this edition focuses on "emerging risks"—risks that appear at the frontier of general-purpose AI capabilities—and no longer covers issues such as bias, environmental impact, privacy, and copyright, in order to complement other efforts such as the UN Independent International Scientific Panel (About this Report footnote).
    • Three core questions: the report is organized around three questions—what general-purpose AI can do today and how capabilities will change (Chapter 1); what emerging risks it brings (Chapter 2); and what risk management methods exist and how effective they are (Chapter 3) (Introduction).
    • Uneven evidence base: the authors note that risks such as harms from AI-generated media and cybersecurity vulnerabilities already have robust empirical evidence, whereas risks that depend on future capability development can rely only on modeling, controlled experiments, and theoretical analysis; the report marks evidence gaps where uncertainty is high (Introduction).
    • How it was written: it was produced by a writing team of more than 30 people, with over 100 experts participating, including an expert advisory panel nominated by more than 30 countries and international organizations; the independent writing team led by the chair has full decision-making authority over the content (About this Report).
  2. What related research exists?

    The report cites and synthesizes a large body of existing research, organized along three lines: capabilities, risks, and risk management, and positions itself as an evidence synthesis rather than new research.

    This report is an evidence synthesis, citing hundreds of studies; below, following the report's own organization, are the main categories of work it relies on.

    Capabilities and forecasting

    • Epoch AI (2025)—provides data on growth in training compute (the largest training run has exceeded 10^26 FLOP), benchmark score trends, and company revenue (Figure 1.4, 1.6, 1.8).
    • Kwa et al. (2025)—the duration of software tasks that AI agents can complete doubles about every 7 months; at 80% success rate it is currently about 20–30 minutes (Figure 1.7, technical appendix).
    • Forecasting Research Institute (212, 229)—expert forecasts: 50% probability that FrontierMath reaches 55% accuracy in 2027; AI forecasting experts give a median 20% probability to "six years of progress compressed into two years," while superforecasters give 8%.
    • OECD (2026)—in collaboration with the report, proposes four capability scenarios for 2030: stagnation, slowdown, continued, acceleration (§1.3).

    Risk evidence

    • Malicious use: Costello et al. (2024), Hackenburg et al. (2025), and other experimental studies of AI persuasiveness; Heiding et al. (2024) on AI phishing emails; Rodriguez et al. (2025) on a cyberattack chain framework; Rose and Nelson (2023) on a biological weapons development flowchart.
    • Malfunctions and loss of control: Meinke et al. (2024) on models' avoidance behavior when told they will be replaced; factuality benchmarks such as SimpleQA Verified (Figure 2.10).
    • Systemic risks: research on labor markets and autonomy (§2.3; the attached text contains only the beginning).

    Risk management

    • The report discusses Frontier AI Safety Frameworks released or updated by 12 companies in 2025, as well as the OpenAI Preparedness Framework, Anthropic Responsible Scaling Policy, and Google DeepMind Frontier Safety Framework (Box 2.2).

    Positioning: the authors state that this report is "a synthesis of existing research" (Disclaimer), does not represent the views of any government or individual, and does not endorse any policy; compared with the 2025 edition, it adds the OECD scenarios and Forecasting Research Institute forecasts (Forewords).

  3. How does the paper address this problem?

    Through a structured evidence synthesis: defining general-purpose AI and emerging risks, organizing evidence by three categories of risk, and assessing the effectiveness of risk management methods.

    The report is not an experimental study; its "method" is a systematic evidence synthesis framework, with the following key points.

    Scope and definitions

    • General-purpose AI: models and systems capable of performing many tasks across contexts, covering language, images, video, robotics, biomolecular structure prediction, and AI agents (Table 1.1).
    • Emerging risks: risks that appear at the capability frontier; systemic risks are defined as the risks arising from widespread deployment of highly capable general-purpose AI in society and the economy (different from the EU AI Act usage; the report has a footnote explaining this).
    • Evidence cutoff: synthesizes evidence published before December 2025 (Introduction).

    Writing and review process

    • Team: led by the Chair (Yoshua Bengio), with chapter leads, lead writers, and core writers totaling more than 30 people; more than 100 experts participated.
    • Review: the initial draft was reviewed by external domain experts; the merged draft was reviewed by the Expert Advisory Panel (nominated representatives from more than 30 countries and the OECD, EU, and UN), the Senior Advisors group, and industry and civil society representatives; the Expert Advisory Panel provides technical feedback only and does not endorse any policy path (Contributors).
    • Independence: the independent writing team led by the Chair has full authority over the content (About this Report).

    Content organization

    • Chapter 1 (Capabilities): six stages of the development pipeline (data collection and curation, pretraining, post-training and fine-tuning, system integration, deployment and release, post-deployment monitoring, Table 1.2); current capabilities and "jagged" limitations; 2030 outlook including four OECD scenarios and a technical appendix (scaling laws, data availability, compute infrastructure constraints, interpretation of difficult benchmarks such as FrontierMath).
    • Chapter 2 (Risks): divided into three categories—malicious use (AI-generated content and crime, influence and manipulation, cyberattacks, biological and chemical risks), malfunctions (reliability challenges, loss of control), and systemic risks (labor market, human autonomy). Each section contains "key information," evidence, evidence gaps, mitigation measures, and challenges for policymakers.
    • Chapter 3 (Risk management): technical and institutional challenges, risk management practices, technical defenses and monitoring, open-weight models, building societal resilience (the attached text contains only content up to the middle of Chapter 2; Chapter 3 details are not attached).

    New structural elements

    • Each section contains fixed columns for "updates since the 2025 report," "challenges for policymakers," and "evidence gaps"; the OECD scenarios and Forecasting Research Institute forecasts are new (Introduction).
  4. What experiments did the paper conduct?

    The report itself conducts no experiments but instead aggregates quantitative results from existing research; below are the key figures it cites and notes on evidence strength.

    This report is an evidence synthesis and contains no original experiments. Below are the main quantitative results it cites, organized by report chapter; all come from the cited studies.

    Capability evidence (Chapter 1)

    • Benchmark performance: leading models exceed 90% on MMLU and 80% on GPQA; in July 2025, Google DeepMind and OpenAI models reached gold-medal level at the International Mathematical Olympiad, solving five of six problems (§1.2).
    • Agent task duration: at 80% success rate, software tasks of about 20–30 minutes can be completed; at 50% success rate, up to 2.5 hours; at 4 hours, it falls below 25%; task duration doubles about every 7 months (Kwa et al. 2025, Figure 1.7, technical appendix).
    • Inference scaling: the s1 model's accuracy on AIME 2024 and GPQA Diamond rises with increased test-time compute (Figure 1.5; the figure shows only axes and a description of the trend).
    • FrontierMath: at release in 2024, SOTA was below 2%; Epoch AI evaluations put GPT-5 at about 25% and o4-mini at about 20% (technical appendix).

    Risk evidence (Chapter 2, malicious use section)

    Risk areaKey figuresSource (location in report)
    AI-generated content96% of online deepfake videos are pornographic; 15% of UK adults report having seen deepfake pornographic images; in a 10-country survey, 2.2% of respondents had non-consensual intimate imagery generated§2.1.1 Key information
    Content identification77% of participants mistook GPT-4o-generated text for human-written; 80% mistook voice clones for real people; warning labels raised accuracy in identifying AI videos from 10.7% to 21.6%§2.1.1 Updates, Mitigations
    ManipulationOn trivia questions, AI changed beliefs by 17 percentage points vs 9 percentage points for humans; each 10x increase in training compute increased persuasiveness by about 1.8 percentage pointsTable 2.2, §2.1.2
    CyberattacksIn DARPA AIxCC, an AI system autonomously found 77% of vulnerabilities; one threat actor used models to automate 80–90% of the workload of an intrusion; picoCTF 2025 reached the top 3% but PlaidCTF solved no problems§2.1.3
    Biological riskOpenAI o3 surpassed 94% of domain experts in troubleshooting virology experiments; 23% of high-performance biological tools were assessed as having high misuse potential, of which 61.5% are fully open source; of 375 biological AI tools, only 3% have any safeguards§2.1.4
    • Authors' interpretation: the authors state that AI-generated content crime and cyberattack assistance already have empirical records, but systematic data on incidence and severity are lacking; manipulation works in the laboratory but real-world evidence is limited; evidence on real-world uplift for biological risk is inconsistent—one recent study showed that an unguarded model provided substantial help on agentic tasks, while earlier studies found no statistically significant effect (§2.1.4).

    Malfunctions and reliability (§2.2.1; the attached text contains only the beginning)

    • Factuality improvements: scores of major models on SimpleQA Verified rise with release date (Figure 2.10; specific scores are not labeled in the figure).
    • Reliability problem categories: hallucination, basic reasoning failures, out-of-distribution failures, tool-use failures, and multi-agent coordination failures—five categories (Table 2.4).

    Note on evidence strength

    • The report itself states that the evidence base is uneven: some risks have robust empirical evidence, while others rely on modeling, controlled experiments, and theoretical analysis; "evidence gaps" are marked in many places (Introduction, Evidence gaps in each section).
    • The report does not report any original sample sizes, numbers of repetitions, or statistical tests; sample sizes of cited studies are given only in individual places (e.g., participant numbers in the experiments in Table 2.2 range from 108 to 76,977).
  5. What are some points that can be further explored?

    Each section of the report explicitly lists evidence gaps; its coverage is limited to literature published before December 2025 and contains no original experiments.

    Limitations and future directions noted by the authors

    • Missing statistics on malicious use: systematic data on the incidence and severity of harms from AI-generated content are lacking, and underreporting (due to embarrassment or fear) may be a factor; the authors state that multiple event detection and reporting pathways are needed (§2.1.1 Evidence gaps).
    • Limited understanding of manipulation mechanisms: understanding of how AI manipulation works and whether AI induces true beliefs and false beliefs to the same degree is limited; evaluation under more realistic conditions is needed, but ethical concerns make real-world evaluation difficult (§2.1.2 Evidence gaps).
    • Immature cyber capability evaluation: AI cyber evaluation is an emerging field, and benchmarks may overestimate due to data contamination or underestimate because they fail to elicit real capabilities; threat intelligence is difficult to use for attribution (§2.1.3 Evidence gaps).
    • Difficulty translating biological risk: there is a gap in translating demonstrated capabilities into risk estimates; real uplift studies are scarce, expensive, and constrained by law and ethics, and chemical risk assessment has received less attention than biological risk (§2.1.4 Evidence gaps, Box 2.3).
    • Uncertain capability forecasting: it is not possible to reliably predict when specific capabilities will appear or when diminishing returns to scaling will constrain progress; empirical understanding of feedback loops in AI automating its own R&D is lacking (§1.3 Evidence gaps).
    • Insufficient transparency: external visibility into training data, models, agents, evaluations, development processes, and safety is limited and inconsistent, making independent research and policymaking more difficult (§1.1 Evidence gaps).

    Scope of experimental coverage

    • This report is an evidence synthesis and contains no original experiments, model runs, or benchmark tests; all numbers are cited from third-party research (Disclaimer).
    • The evidence cutoff is literature published before December 2025 (Introduction).
    • The scope focuses on "emerging risks" and does not cover bias, environmental impact, privacy, or copyright (About this Report footnote).
    • The supplied text contains only the front part of the report (up to the beginning of §2.2.1); details on loss of control (§2.2.2), systemic risk (§2.3), and risk management (Chapter 3) are not included and cannot be verified.
    • The report states that it does not provide specific policy recommendations and does not represent the positions of participating governments (About this Report, Disclaimer).
  6. Summarize the main content of the paper

    Second International AI Safety Report: synthesizes evidence to assess general-purpose AI capabilities, three categories of risk, and risk management, emphasizing the evidence dilemma and layered defenses.

    This is the 2026 International AI Safety Report, chaired by Yoshua Bengio and written with the participation of more than 100 experts; it is the second international scientific assessment authorized by the Bletchley Park summit, focuses on the emerging risks of general-purpose AI, and makes no policy recommendations.

    • Problem to be addressed: capabilities evolve quickly while risk evidence is slow and difficult to assess, creating an "evidence dilemma"; the report aims to synthesize existing evidence and mark gaps to help policymakers act according to the severity and likelihood of risks.
    • Capability conclusions: the authors state that capabilities continue to improve but are "jagged"—leading models exceed 90% on MMLU and 80% on GPQA, reached gold-medal level at the International Mathematical Olympiad in 2025, but still hallucinate and are insufficiently reliable on long tasks; at 80% success rate, AI agents can complete software tasks of about 20–30 minutes, and task duration doubles about every 7 months (Kwa et al. 2025). The direction by 2030 is uncertain; all four OECD scenarios, from stagnation to acceleration, are considered possible.
    • Risk conclusions: divided into three categories: malicious use, malfunctions, and systemic risks. Under malicious use, AI-generated content crime (96% of online deepfake videos are pornographic) and cyberattack assistance (in DARPA AIxCC, AI autonomously found 77% of vulnerabilities) already have empirical evidence; manipulation works in the laboratory but real-world evidence is limited; on biological risk, in 2025 several developers added safeguards because they could not rule out that models might help novices develop weapons (Box 2.2).
    • Risk management conclusions: the authors state that technical safeguards have improved but still have limitations, and advocate defence-in-depth; open-weight models pose unique challenges; building societal resilience (critical infrastructure, AI content detection tools, institutional response capacity) is a necessary complement.
    • Overall position: the authors consider the evidence base uneven—some risks already have robust empirical evidence, while others still rely on modeling and theoretical analysis; the report guides future research by marking evidence gaps.
Read originalarxiv.org