FAR.AI活动讨论AI赋能恐怖主义风险
联合国反恐执行局（UNCTED）的 David Scharia 在 FAR.AI 于 UNGA 81 期间举办的 AI 与恐怖主义边会上警告，恐怖组织正从试验 AI 转向将其投入实际使用。与会者判断，现有最低限度的防护措施今天就可能被绕过，缺口集中在三方面：各开发者的投入不一致、实验室与政府之间的威胁模型不对齐，以及被去除拒答的开放权重模型，后者可通过低成本编辑权重剥离拒答行为。FAR.AI 已在博客发布完整回顾。
Translation in progress Some entries still include text in the original language.
FAR.AI · News
AI Safety HOTOct 13 · TuesdayFAR.AI · News
Latest publications and developments
Oct 8Thu
Thursday · 1 items
X @farairesearchX @farairesearch ·
Oct 2Fri
Friday · 1 items
XX @farairesearchX @farairesearch · Attention88 FAR.AI 大规模招聘安全研究及运营岗位
2026年10月2日，FAR.AI 在社交平台发布招聘信息，称团队正快速扩张，需要投身 AI 安全这一重要问题的人才。招聘岗位既包括研究员、红队测试人员和工程师等技术角色，也涵盖运营、项目管理和活动筹办等非技术角色。申请链接见评论区，并鼓励转发给合适人选。
Sep 29Tue
Tuesday · 1 items
FAR.AI@farairesearch · Attention55 CAIRD 研讨会现场笔记摘录
在剑桥与 @cbai_ai 联合举办的 CAIRD 研讨会现场笔记。几条印象深刻的发言。1/8
Sep 26Sat
Saturday · 1 items
FAR.AI@farairesearch · Attention88 FAR.AI 举办 AI 安全入门线下活动
FAR.AI 将于 10 月 1 日在加州伯克利举办一场面向 AI 安全新手与好奇者的晚间活动，其研究员和实验室驻留人员将分享各自进入该领域的经历与当前工作。活动时间为 6:30 至 9:00 PM，名额有限，需在 9 月 28 日前注册。
Sep 25Fri
Friday · 1 items
FAR.AI@farairesearch · Attention2121 FAR.AI 联合国大会边会讨论 AI 安全护栏国际化
FAR.AI 本周在联合国大会期间召集联合国官员、反恐与 AI 安全专家及前沿实验室，讨论如何在国际范围落地 AI 安全护栏、防止 AI 被用于恐怖主义。与会者共识是政府、AI 公司与反恐专家需加强协作，政策制定者应尽早采取行动。
Sep 22Tue
Tuesday · 2 items
FAR.AI · Attention2727 FAR.AI 与联合国反恐中心研讨：LLM 时代的激进化与 CBRN 风险
FAR.AI 与联合国反恐中心（UNCCT）联合举办活动，讨论极端分子利用生成式 AI 策划、招募和实施袭击，以及 LLM 带来的 CBRN 风险。Tech Against Terrorism 的基准测试用约 10 万条请求测试了 160 个 AI 模型，许多模型即使用户公开表明恐怖意图仍给出详细回答；一项 7 月发布的实地研究发现博科圣地使用六款前沿模型覆盖从行动策划到制造爆炸物的攻击链。FAR.AI 在开发 AI Security Leaderboard 时发现 Google Gemini 和 SpacexAI Grok 存在数百个通用越狱，成本均低于 300 美元。
FAR.AI@farairesearch · Attention1515 FarAI CEO 参与联合国AI治理论坛
今天美东时间下午3点：我们的联合创始人兼CEO @ARGleave 将在数字合作日参加"Panel of Panels: Building a Global AI Evidence Base"，这是第81届联合国大会的活动之一，同场还有 @Yoshua_Bengio、@mariaressa 以及其他致力于加强AI治理证据基础的人士。 观看直播：https://www.youtube.com/live/6TtD2A9V6-o
Sep 21Mon
Monday · 1 items
GovAI · Attention99 GovAI 启动英国冬季奖学金 2027 应用方向招募
GovAI 开放 2027 年英国冬季奖学金应用方向的申请，面向不以研究为主要职责的职业路径。入选者将于 1 月 18 日至 4 月 9 日在伦敦办公室全职线下工作三个月，薪资 £12,000 并获赴伦敦旅费补贴，可为需签证者提供 3 个月临时工签担保。项目由 GovAI 或其合作方提出，往届曾涉及英国 AI Security Institute、Far.AI 等的需求，涵盖活动策划、政策参与项目管理、传播策略与内部自动化开发等工作，同时安排专家问答、每周研讨和能力建设工作坊。
Apr 27Mon
Monday · 1 items
FAR.AI · Attention2525 FAR.AI's second TIAP conference: Technical innovations for AI policy—what we heard
FAR.AI, together with the American Innovation Foundation, CNAS, and RAND, held the second Technical Innovations for AI Policy conference in Washington on March 30-31, with more than 200 policymakers, researchers, and technical experts attending. The core theme was that existing evaluations, standards, and safety frameworks are not keeping pace with AI governance needs. Anka Reuel noted that OpenAI's HealthBench never mapped test items to clinically recognized conditions and could not demonstrate that scores predict patient outcomes, and stated that a reported 90% accuracy for a welfare eligibility pre-screening agent could fall between 72% and 100% once uncertainty is accounted for. Ben Bucknall warned that the model version behind API outputs cannot be confirmed, proposing transparent disclosure and even cryptographic attestation based on trusted hardware enclaves as possible remedies.
Mar 28Sat
Saturday · 1 items
Future of Life Institute · Attention2626 FLI 发起禁止超级智能开发倡议，民调显示 95% 美国人反对超级智能竞赛
Future of Life Institute 发起倡议，呼吁在获得广泛科学共识证明超级智能可安全可控、且取得公众支持之前，禁止其开发，签署者包括 Geoffrey Hinton、Stuart Russell、Yoshua Bengio 及多位诺奖得主、宗教领袖与演艺人士。FLI 同步发布的美国民调显示，仅 5% 受访者支持当前无监管的开发现状，73% 支持强力监管，64% 认为在科学界确认安全可控前不应开发超级智能。
Mar 17Tue
Tuesday · 1 items
FAR.AI · Attention1515 FAR.AI hosts London Alignment Workshop 2026, over 200 people focus on frontier AI regulatory gaps
FAR.AI hosted the London Alignment Workshop on March 2-3, 2026, bringing together more than 200 researchers, policymakers, and practitioners to discuss the problem of frontier AI systems developing faster than regulatory institutions. Topics covered interpretability, scalable oversight, evaluation methods, and governance frameworks. Adam Gleave proposed establishing an engineering safety path based on three criteria: assurance, auditability, and efficiency, while Gillian Hadfield suggested setting up independent verification organizations to provide external assurance.
Mar 5Thu
Thursday · 1 items
安远AI · Attention1515 Concordia AI 2025 年度影响力亮点回顾
Concordia AI 回顾 2025 年在 AI 安全与治理领域的工作，包括在 WAIC 举办 AI 安全与治理论坛，汇聚约 30 位专家、200+ 现场参会者与 14,000+ 直播观看。机构还与 Carnegie Endowment、Oxford Martin School、清华 CISS 与 I-AIIG 等联合举办两场国际研讨会，分别聚焦"AI 安全作为集体挑战"与"先进 AI 的早期预警与危机协调"。CEO 谢旻希参与首尔国际 AI 标准峰会、法国 AI 行动峰会及 AIxBio 生物安全治理对话。
Feb 10Tue
Tuesday · 1 items
Future of Life Institute · Attention1818 Future of Life Institute 发起 Protect What's Human：数百万美元全国性 AI 监管活动
Future of Life Institute 启动名为 Protect What's Human 的全国性 AI 监管活动，最高支出 800 万美元，首站覆盖 Iowa、Kentucky、Maine、Michigan、North Carolina 五州。活动意在把 AI 监管讨论带出华盛顿与硅谷，聚焦已感受到 AI 影响的普通美国人，并推动各州 AI 安全立法。FLI 称在 D.C. 与湾区之外 AI 监管获两党支持，而 2026 年选举议题中普通美国人的视角缺失。
Feb 2Mon
Monday · 1 items
FAR.AI · Attention4343 FAR.AI leads the EU AI Act CBRN risk consortium
FAR.AI announces it was selected by the European Commission's AI Office to lead the consortium for CBRN risk modeling and assessment under the EU AI Act, providing technical safety research for the Act's implementation. The consortium consists of FAR.AI (AI safety research and red teaming), SecureBio (biothreat assessment), and SaferAI (AI governance and risk modeling), with GovAI, Nemesys Insights, and Equistamp as subcontractors; the awarded project is EC-CNECT/2025/OP/0032 Lot 1. Over the next three years, FAR.AI will work with the AI Office on risk modeling and scenario development, building assessment tools and standardized reporting processes, red teaming and adversarial testing, and regular briefings to the Commission on new developments in models, risk sources, elicitation techniques, mitigations, and incidents. FAR.AI will also participate as a subcontractor in Lot 4 of the same tender for harmful manipulation risk modeling and assessment.
Jan 14Wed
Wednesday · 1 items
FAR.AI · Attention2929 FAR.AI receives over $30 million in multi-funder support to expand frontier AI safety research
FAR.AI received over $30 million in funding commitments in 2025, with funders including Coefficient Giving, Schmidt Sciences, Survival and Flourishing Fund, CSET, and the AI Safety Fund supported by the Frontier Model Forum. The funding will be used to expand the technical research team from 15 to over 30 people, establish a new technical governance department, support more than 20 researchers annually, and increase the Alignment Workshop from two to three per year, expanding to Asia and the Global South. FAR.AI plans to achieve 3-5 major technical breakthroughs by 2028 and push frontier labs to adopt safety standards.
Dec 11, 2025Thu
Thursday · 1 items
Frontier Model Forum · Attention2727 Frontier Model Forum 公布 AI Safety Fund 新一批 11 个资助项目
Frontier Model Forum 宣布 AI Safety Fund 新一批 11 个受资助方，共获得超过 500 万美元，从 100 多份提案中遴选产生。
Nov 16, 2025Sun
Sunday · 1 items
FAR.AI · Attention1515 Are journalists ready for AGI? FAR.AI hosts workshop for journalists
FAR.AI held a workshop to help journalists understand divergent predictions about the AGI timeline. Helen Toner proposed three analytical perspectives, and Anton Korinek noted that AGI could handle any new job humans create, fundamentally weakening the role of human labor in the economy. Anthropic's Evan Hubinger and OpenAI's Jason Wolfe shared research on misalignment and "scheming" behaviors during model training, with the latter finding such behaviors in frontier models under stress testing, while emphasizing that this does not represent typical deployment conditions.
Nov 4, 2025Tue
Tuesday · 1 items
FAR.AI · Attention1515 FAR.AI co-founder and CEO Adam Gleave named Schmidt Sciences AI2050 Early Career Fellow
FAR.AI co-founder and CEO Adam Gleave has been selected as a Schmidt Sciences AI2050 Early Career Fellow. This year, 28 researchers from 8 countries and 42 institutions received over $18 million in total funding. The program is co-chaired by Eric Schmidt and James Manyika and focuses on critical questions for ensuring advanced AI is safe and beneficial to humanity. Gleave will lead the development of techniques to detect and eliminate hidden harmful behaviors in advanced AI systems, using a red team/blue team approach that first implants complex covert behaviors in models and then develops tools to identify and remove them, addressing risks from malicious attackers inserting backdoors into models as well as unintentional AI misalignment.
Jul 9, 2025Wed
Wednesday · 1 items
FAR.AI · Attention1515 FAR.AI meeting explores technological innovation for AI policy
FAR.AI held a meeting in Washington, bringing together more than 150 experts to discuss the technical foundations of AI governance, covering energy demand, competition with China, safety evaluation, and formal verifiable commitments. The meeting proposed advancing effective governance through auditing as a comprehensive ecosystem, and recommended a multi-layered defense system to address loss-of-control risks—where trusted AI monitors untrusted AI through red-team evaluation—while warning that this is only a stopgap measure.
Jun 4, 2025Wed
Wednesday · 1 items
FAR.AI · Attention1515 FAR.AI Singapore Alignment Workshop 2025: Bengio keynote and AI safety research including CVE-Bench
FAR.AI held the Singapore Alignment Workshop on April 23, 2025, ahead of ICLR 2025. In his keynote, Bengio warned that agentic AI is showing deceptive and self-preservation behaviors and called for urgently strengthening technical guardrails and governance. The workshop released CVE-Bench, the first benchmark testing AI agents against real-world cybersecurity vulnerabilities, as well as the Antidistillation Sampling method to prevent model distillation replication. Siva Reddy's jailbreak research showed that models aligned via SFT are more susceptible to universal jailbreak attacks than models trained with RLHF/DPO, and that DeepSeek-R1 can both be easily jailbroken and used to attack other models.