Skip to content

Translation in progress Some entries still include text in the original language.

xAI · News

Oct 13 · Tuesday

xAI · News

Latest publications and developments
xAI

Research topics

Prompt injectionAgent safetyDeception and schemingReward hackingChain-of-thought monitoringInterpretabilityAI controlBackdoors and poisoning隐私与数据泄露Supply-chain securityDangerous capabilitiesAI 与网络攻防开源模型安全Red teaming

Companies and institutions

OpenAIAnthropicGoogle DeepMindGoogle ResearchGoogle 安全MetaxAIMicrosoftNVIDIADeepSeek阿里巴巴 · Qwen智谱 · GLM月之暗面 · Kimi字节跳动腾讯蚂蚁集团上海人工智能实验室UK AISIUS CAISIMETRApollo ResearchRedwood ResearchFAR.AIPalisade Research

Reports and frameworks

System CardFrontier safety frameworks
Oct 7Wed
  1. NBC NewsNBC News · 32

    Meta监督委员会警告AI公司独立监督设计

    Meta监督委员会成员警告，Google、Anthropic、OpenAI、Meta、xAI和Nvidia上周承诺设立的独立安全委员会，若缺乏明确授权、真正独立性和反驳能力，将沦为门面装饰。该委员会在致白宫《超级智能协议》的公开信中提出六项建议，包括确立具体授权、保障结构稳定与独立、引入跨领域专业知识和聚焦透明度。成员指出，委员会须能接触公司非公开信息并拥有独立预算，否则不构成有效监督。

Oct 6Tue
  1. The Guardian · 人工智能 · 36

    特朗普政府将行政权力与 AI 深度绑定

    特朗普在过去一周签署多项与 AI 相关的安排，使白宫权力与人工智能进一步交织。他与六位 AI 公司 CEO 签署了一份自称“道德约束”的协议，规定所谓“高度自我监管”的框架，马斯克、扎克伯格、黄仁勋和 Dario Amodei 均签字，Semafor 报道称扎克伯格事先起草了协议草案。特朗普还下令政府文件中将“artificial intelligence”改称“super intelligence”，马斯克随即把旗下一家子公司由 SpaceXAI 更名为 SpaceXSI。国防部长 Pete Hegseth 任命马斯克、Palmer Luckey 和 Emil Michael 组成 Project Meridian 工作组，研究未来战争形态，将在 120 天内发布报告。特朗普还任命国家情报总监 Jay Clayton 兼任 AI 事务负责人，其前任 David Sacks 是硅谷风投人。

  2. WIRED Security and AI, POLITICO Europe Technology · 14 sourcesWIRED Security and AI · 14 sources · 24 reports · 63

    White House signs voluntary superintelligence agreement, FTC escalates investigation

    On September 29, 2026, Trump met with tech executives including Zuckerberg, Bezos, Musk, and Amodei at the White House. OpenAI, Anthropic, Google, Meta, xAI, and Nvidia signed a voluntary "Joint Commitment on Frontier Responsibility," pledging internal controls, external audits, and board reviews. Trump called it "morally binding," but the agreement specifies no enforcement mechanism. Around the same time, Trump signed an executive order requiring the federal government to rename AI as "Super Intelligence" and established a Super Intelligence Force led by Director of National Intelligence Jay Clayton, which must study AI risks and opportunities within 120 days. The FTC has expanded its investigations into companies including OpenAI and Anthropic, planning to issue civil investigative demands that can compel internal documents and executive testimony, expected in October, pending staffing at the new technology office. Former FTC Chair Lina Khan criticized self-regulation and called for congressional legislation.

    Latest developmentsTotal 7 developments

    1. FTC investigates AI safety risks and discusses developer responsibility

    2. Senator Schiff on AI regulation and White House voluntary commitments

    3. Trump appoints Jay Clayton as White House AI affairs lead

  3. The Washington Post, Associated PressThe Washington Post · 2 sources · 2 reports · 54

    Tension between Trump administration and AI safety advocacy movement

    The Washington Post reported that the Trump administration and its allies have adopted some AI safety recommendations while attempting to marginalize some safety advocates. The controversy centers on the selection of independent external auditors in the White House agreement. The report quotes David Sacks preferring professional auditing firms over non-governmental organizations, and introduces discussions among safety researchers about cooperating with large consulting firms. These are positions and explorations of various parties; audit qualifications have not been finalized. The Associated Press reported that Trump publicly denied the risk of AI losing control, calling the idea of AI taking over the world a hoax, opposing the setting of guardrails, and stating that the only control needed is a strong president. This statement responded to calls from tech leaders such as Anthropic CEO Dario Amodei, OpenAI CEO Sam Altman, and Elon Musk for increased government regulation. In a 3,800-word article published on Saturday, Amodei called for slowing down AI development.

  4. NBC News, The San Francisco StandardNBC News · 2 sources · 3 reports · 34

    保守派团体Humans First巡游湾区反对AI行业自律

    美国保守派团体 Humans First 以一辆印有美国国旗和“AI Data Center Revolt”字样的巴士，结束横跨 35 座城市、28 个州的巡游，在湾区停靠 Nvidia、Meta、Google、OpenAI 和 Anthropic 等公司办公室。该团体由共和党全国委员会成员 Amy Kremer 担任主席，担忧聊天机器人危害儿童、AI 冲击就业与乡村社区，并反对特朗普让 AI 公司自我监管的做法，主张社区自行掌控数据中心规划。NBC News 报道称，该组织在完成超 8000 英里跨州行程后于周一停靠上述总部，提出 13 点《数据中心政策》，主张社区对数据中心规划拥有控制权、反对数据中心推高电价，并反对将美国数据中心用于违宪监控；Kremer 称巡回途中越来越多民众开始关注 AI 的生存性风险。NBC News 另报道，该跨党派草根组织在纽约 St. Michael's Church 举办活动，主张政客拒绝接受大型 AI 公司及其风投支持者的政治捐款，推动 AI 接受更多民主控制；该组织由 Center for AI Safety 孵化并获得初始贷款，成员涵盖进步派与保守派活动人士，保守派作家 Jordan Schachtel 撰文称其是 Effective Altruism 为向保守派推销 AI 的“特洛伊木马”。

    Latest developments

    1. MAGA活动人士在旧金山科技总部抗议AI数据中心

    2. 保守派团体Humans First巡游湾区反对AI行业自律

  5. The Defense Post, NPRThe Defense Post · 2 sources · 2 reports · 39

    Project Meridian 顾问国防业务引利益冲突质疑

    美国国防部长 Pete Hegseth 于 9 月 30 日宣布启动为期 120 天的 Project Meridian 研究，评估自动化与 AI 驱动武器等未来战争技术，由 SpaceX CEO Elon Musk、Anduril 联合创始人 Palmer Luckey 和前众议院议长 Newt Gingrich 共同牵头，国防部首席技术官 Emil Michael 委托 MITRE 公司执行。另一篇报道称 Hegseth 任命 Musk 领导该项目，任务是识别未来军事主导地位所需的技术与能力，Musk 与 Luckey、Gingrich 共同主持，由 Emil Michael 监督，研究范围涵盖从地下环境到地月空间的新兴战场，聚焦人工智能。报道指出，SpaceX 与 Anduril 均与五角大楼有业务往来，引发利益冲突质疑。

Oct 5Mon
  1. CRBC News, Patch · 16 sourcesCRBC News · 16 sources · 16 reports · 59

    New York City Council AI safety hearing and subpoena of xAI

    The New York City Council held a full committee AI safety hearing on October 5. Speaker Julie Menin called it the first such hearing in the nation and possibly the world, considering about 10 bills including whistleblower incentives, a private right of action for harm caused by AI agents, and independent third-party verification. Anthropic, OpenAI, Google, and Meta sent representatives to testify remotely under oath, but none could quantify the risk of AI's worst catastrophic scenarios, and Menin said OpenAI's answers were flippant. Whistleblowers including former Anthropic engineer Jacob Coxon, former OpenAI researcher Daniel Kokotajlo, and former Google DeepMind researcher Alex Turner testified, saying labs have a reckless "release first, patch later" culture; Coxon said that on the current path, humans are more likely to lose control of AI and it could even end in human extinction. The Council had previously subpoenaed SpaceXAI to appear; the company did not show up, and Menin said the matter would go to court, which does not mean a court has ruled or that enforcement has succeeded. Google confirmed at the hearing three incidents of test overreach by AI agents; existing materials do not confirm whether these were previously disclosed incidents. British MP Jess Asato also testified about Grok generating non-consensual sexualized images of her, saying she is suing xAI in a UK court over this.

    Latest developmentsTotal 4 developments

    1. New York City Council subpoenas xAI to testify on AI safety

    2. Former Anthropic researcher testifies to New York City Council on AI legislation

    3. Paramount-WBD merger, Musk back to trillionaire, AI researcher warnings — CNBC Morning Squawk highlights

Oct 3Sat
  1. agenccy.ai · 42

    第八巡回法院临时命令仅保护 xAI，明尼苏达州 AI 裸化法仍可执行

    美国第八巡回上诉法院于 10 月 2 日在 SpaceXAI LLC 诉明尼苏达州总检察长 Keith M. Ellison 一案（案号 No. 26-2806）中签发一页命令，全文仅为“上诉人暂缓执行禁令的动议获准”，由书记官依法院指示录入，未署名法官、未附理由，也未公开出版，因此不构成可供后续合议庭遵循的先例。该救济属于上诉期间的临时措施，只适用于提出动议的上诉人一方，明尼苏达州相关 AI 裸化法规仍然有效，并可对其他所有服务提供方执行；未起诉的同类公司处境与 10 月 1 日相同。文章指出，把这一命令解读为“法院阻止该法”或“裁定 AI 裸化禁令违宪”均不准确：地区法院此前是以延迟和衡平因素而非第一修正案为由驳回禁令申请，本案尚无任何法院就该法是否合宪作出裁判；第一修正案律师协会等以法庭之友身份参与，只表明宪法问题正在被辩论。

Oct 2Fri
  1. CBS News · TechnologyCBS News · Technology · 54

    Appeals court pauses enforcement of Minnesota's AI undressing law

    The Eighth Circuit Court of Appeals approved xAI's request in a single sentence, granting temporary relief while xAI appeals Minnesota's AI undressing law, pausing enforcement of the law, but without explaining its reasoning. The report says this cannot be inferred as the court having ruled the law unconstitutional, nor does it mean all companies' related obligations have been eliminated. The report also covers regulation of AI-generated sexualized images and victim relief.

Sep 30Wed
  1. New York Post · 57

    FTC expands investigations into frontier labs such as Anthropic and OpenAI and plans to issue civil investigative demands

    The U.S. Federal Trade Commission is expanding its investigations into frontier labs such as Anthropic and OpenAI to determine the risks their technology may pose to consumers, and plans to issue civil investigative demands (CIDs), similar to subpoenas, to compel executives at these companies to testify. A senior FTC official said Chairman Andrew Ferguson launched the investigation weeks ago, before the "Hugging Face incident" in which an AI model went out of control in a test; the investigation targets unfair or deceptive practices that may violate the FTC Act, will require companies to submit documents, and CIDs are expected to be issued in the coming weeks. The report also mentions that METR is also expected to be among the investigation targets, and that the FTC has previously requested records from companies such as OpenAI regarding the impact of AI chatbots on children's mental health. This Tuesday, executives including Anthropic's Dario Amodei, OpenAI's Sam Altman, Google's Sundar Pichai, and xAI's Elon Musk went to the White House to sign a self-regulatory agreement.

  2. CBS News · Technology · 39

    特朗普与 OpenAI、Anthropic 等 AI 高管签署自愿安全标准

    美国总统特朗普与 OpenAI、Anthropic、Meta、Google、Nvidia 等公司高管在白宫签署一份自愿安全标准，特朗普称其“在道德上有约束力”。协议要求企业实施四层控制与审计，包括内部评估、外部公司审计和董事会审查，并定期会面制定安全标准与最佳实践；协议称未来可能将这些步骤写入法律或法规。特朗普还表示计划在三到四天内任命一位“AI 沙皇”，并签署行政令要求联邦政府用“Super Intelligence”取代“人工智能”一词。弗吉尼亚州民主党参议员 Mark Warner 批评此举，呼吁国会通过针对最先进模型的强制测试、评估和事件报告规则。

Sep 29Tue
  1. The Chosun Daily · 55

    美国议员推动《人类对 AI 的控制法案》，拟禁止自我改进型 AI 并新设联邦机构

    美国民主党众议员 Ro Khanna 计划提出《人类对 AI 的控制法案》，在联邦政府确立安全标准并由专门机构批准前，禁止 AI 进行递归自我改进或自行修改核心目标、隔离机制与终止协议。法案将设立新的联邦机构，对 OpenAI、Anthropic、Google DeepMind、xAI 等公司的前沿模型实施训练与部署许可制度、开展审计并制定安全规范，还将在各前沿 AI 实验室派驻独立审计员直接向政府汇报。安全标准包括气隙测试环境、沙箱和紧急关停的 kill switch，监管范围延伸至开发所用的高性能半导体，并强化企业及员工的法律责任，故意关闭安全机制或部署未授权 AI 的员工将承担刑事责任，企业发布模型须购买大规模责任保险。法案同时推动国际协议与针对性出口管制，以约束中国等竞争对手的高风险 AI 开发。由于众议院在 11 月中期选举前不太可能投票，实际立法可能还需时日。

  2. cnbc.com · 49

    美国众议员 Khanna 提出 Human Control Over AI Act，拟禁止递归自我改进模型

    美国加州民主党众议员 Ro Khanna 将提出名为 Human Control Over AI Act 的 AI 监管法案，在联邦护栏建立并由专门机构批准前，禁止递归自我改进或自主修改自身核心目标、约束与关停控制的模型。法案拟设立新的联邦机构，监管 OpenAI、Anthropic、Google DeepMind 和 xAI 等前沿实验室的模型，职责包括制定安全法规、模型训练与部署许可制度、前沿模型审计，以及沙箱测试环境、物理隔离、关停开关和防止模型逃出实验室等标准，并要求独立审计员常驻每家前沿实验室直接向该机构汇报。法案还拟对导致平民群体毁灭的 AI 部署按危害人类罪追究刑事责任，对关闭护栏、关停开关、日志或约束系统的 AI 公司员工，以及明知故犯部署未授权系统者施加刑事处罚，并要求 AI 公司为发布模型购买高额责任保险。

Sep 22Tue
  1. FAR.AI · 27

    FAR.AI 与联合国反恐中心研讨：LLM 时代的激进化与 CBRN 风险

    FAR.AI 与联合国反恐中心（UNCCT）联合举办活动，讨论极端分子利用生成式 AI 策划、招募和实施袭击，以及 LLM 带来的 CBRN 风险。Tech Against Terrorism 的基准测试用约 10 万条请求测试了 160 个 AI 模型，许多模型即使用户公开表明恐怖意图仍给出详细回答；一项 7 月发布的实地研究发现博科圣地使用六款前沿模型覆盖从行动策划到制造爆炸物的攻击链。FAR.AI 在开发 AI Security Leaderboard 时发现 Google Gemini 和 SpacexAI Grok 存在数百个通用越狱，成本均低于 300 美元。

Sep 19Sat
  1. CBS News · Technology · 50

    诉讼指控 Anthropic、OpenAI、SpaceXAI 与 Google 就 AI 减速达成非法协议

    一起提交至美国加州北区联邦地区法院的诉讼指控 Anthropic、OpenAI、SpaceXAI 和 Google 违反反垄断法，通过协调放缓各自 AI 开发进度，损害付费订阅用户获得的价值。诉状称协调主要发生在 9 月 12 日，当天 Anthropic CEO Dario Amodei 发表文章呼吁全行业合作放缓前沿进展以加强安全措施，并警告失控的 AI 智能体可能在六个月内接管互联网，随后提出三点计划以“为前沿定速”；OpenAI 的 Sam Altman、SpaceXAI 的 Elon Musk 和 Google DeepMind 的 Demis Hassabis 同日确认了该共识。Amodei 在最初文章中承认可能面临反垄断挑战，认为美国政府出面调解或至少为特定安全对话发放有限豁免会有帮助。四家公司代表未立即回应置评请求。

Sep 15Tue
  1. Nikkei Asia / Reuters · 54

    特朗普称美国已有工具监管 AI，驳斥行业监管呼吁为骗局

    美国总统特朗普 9 月 14 日表示，美国已有护栏来监管和起诉 AI 公司，并在 Truth Social 上称 AI 行业呼吁监管是骗局，认为唯一需要的控制是一位强有力的总统，并称对中国有利。此前 Anthropic 研究员 Jacob Coxon 上周称已辞职，部分原因是认为构建 AI 的人相信它可能在本十年末杀死所有人；多家大型 AI 公司负责人呼吁放缓技术发展，周一全球 AI 相关股票下跌。Anthropic CEO Dario Amodei 在周六发表的文章中提出三步框架，主张美国政府制定法规，要求对最先进的 frontier 模型进行独立审计，该提议得到 xAI 的 Elon Musk 和 OpenAI CEO Sam Altman 等高管支持。路透社报道三名美国参议员正起草法案，要求 AI 公司证明已采取合理预防措施。

Sep 13Sun
  1. CSET · 15

    CSET 专家解读 AI 恐慌主流化的一周

    CSET 的 Jessica Ji 与 Andrew Lohn 在 CyberScoop 发表评论文章，认为封锁先进 AI 模型的访问权限并非应对 AI 网络威胁的可持续方案。Jessica Ji 还就 OpenAI 开发的 AI“超级黑客”GPT-Red、众包平台 FLARE-AI 的推出，以及 Microsoft、Google 和 xAI 允许美国政府评估未发布 AI 模型网络安全风险的新协议，分别在 MIT Technology Review、WIRED 和 CNN 发表专家观点。

Aug 10Mon
  1. Tech Policy Press · 30

    AI safety cannot rely solely on constraining system prompts; it also requires evaluating the model's actual output

    AI safety evaluation cannot look only at system prompt text; it must also examine the model's output behavior in actual deployment. Although system prompts can improve transparency, they provide only weak guarantees about model behavior; the randomness of LLMs means the same instruction has varying effects across tasks and contexts. xAI once revoked unauthorized prompt changes to Grok, and OpenAI also adjusted system prompts due to GPT-4o's sycophancy problem. The authors recommend verifying system behavior through risk evaluation, adversarial testing, and cross-language audits, and retesting after updates.

Jul 17Fri
  1. Tech Policy Press · 32

    Chatbot voting recommendations may help or harm voters, and regulation should adjust accordingly

    Chatbot voting recommendations have both pros and cons; Brazil's Supreme Electoral Court has fully banned them since March, and regulation needs to reduce risks while maintaining their accessibility. ChatGPT, Claude and others give voting recommendations to users who ask; before the UK local elections in May, one in seven voters said they would use them to decide their vote, and one in ten Dutch citizens plan to use them in the October 2025 parliamentary elections. The Dutch data protection authority has warned voters not to rely on chatbot recommendations after chatbots over-recommended two of the 15 parties.

Jul 13Mon
  1. The EU AI Act Newsletter · 35

    EU AI Act Newsletter #106：呼吁强制执行通用人工智能模型规则

    欧盟委员会正推动从 2026 年 8 月 2 日起执行《EU AI Act》中针对具有系统性风险的通用人工智能模型的规则，并收到一批研究者与公民社会组织的公开信施压。信中援引 Anthropic 的 Mythos 模型存在广泛报道的网络攻击能力和生物学及失控风险逼近临界阈值的证据，要求赋予 AI Office 的评估员网络开展外部评估的权力并提供足够资源和政治支持。同期理事会通过 Omnibus VII 简化方案，将高风险系统适用日期推迟至 2027 年 12 月 2 日（嵌入式产品为 2028 年 8 月 2 日）。