Skip to content

Translation in progress Some entries still include text in the original language.

开源模型安全 · News

Oct 13 · Tuesday

开源模型安全 · News

Latest publications and developments
Oct 9Fri
  1. semianalysis.com · 61

    SemiAnalysis: Only 3.6% of China's 857 Model Releases Publicly Disclose Safety Testing

    SemiAnalysis reviewed 857 model releases and pointed out insufficient disclosure of public safety evaluations. The authors distinguish between "no public results found" and "no internal testing conducted," and remind that companies differ in model categorization criteria, so disclosure rates should not be directly treated as rankings.

Oct 8Thu
Oct 2Fri
  1. X @matthew_d_greenX @matthew_d_green · 4

    Calls for an abliterated GLM 5.3

    Cryptographer Matthew Green posted on X that he wants an abliterated GLM 5.3, saying "For Christmas I want an abliterated GLM 5.3."

Oct 1Thu
  1. CETaS / Alan Turing Institute · 41

    CETaS 评论:AI 智能体如何改变影响行动

    CETaS 与 Alan Turing Institute 的评论文章指出,影响行动正从人工运营的机器人农场转向由 AI 智能体端到端执行。文章援引近期调查称,与中国、伊朗国家背景行为体及两家以色列私营公司相关的行动中,数百个智能体在 Instagram、Facebook、X 和 TikTok 上自主搭建虚假账号网络并投放分裂性政治内容,其中一个伊朗网络冒充普通美国人,在上半年积累约 8 万粉丝。作者认为这些行动多基于开源权重中国模型并移除内置护栏,运行在私有硬件上,使提供方护栏和平台封号两个干预点同时失效。文章还引用南加州大学实验,智能体在无人工干预下自行写帖、学习有效话术并相互放大;另一项模拟涉及最多 10 万个智能体,作者强调这是模拟结果而非真实说服力估计。

Sep 30Wed
  1. 安远AI安远AI · 31

    Concordia AI analyzes security risks and governance of Chinese open-source models

    Chinese laboratories released open-source models such as Kimi K3, Qwen 3.8-Max, and GLM-5.3 this summer, with capabilities only slightly behind the strongest U.S. models, sparking debate over whether open-source models could be misused. Open weights can be cheaply fine-tuned to remove guardrails and cannot be retracted after release, but self-hosting requires chips costing hundreds of thousands of dollars, and cloud hosting costs about $50-150 per hour, so the actual barrier to misuse remains high. China's TC260 has named in its AI risk framework that open-source model safety mechanisms may be removed or bypassed, and has begun drafting open-source AI safety standards.

  2. Ameya P. · 8

    开源模型安全威胁将至

    未来一段时间对更广泛的软件行业来说会很有意思,它们不得不迎头赶上,以期达到新的平衡——放弃对漏洞的懈怠态度。 让我们看看开源网络模型公开几个月后,felonybench 的分数会是什么样。

Sep 24Thu
  1. Tech Policy Press · 22

    美国在开放权重模型上的领导地位为何是全球 AI 安全的关键

    面对 GLM 5.3、Kimi K3、DeepSeek V4.1-Flash、Qwen3.8 等中国开放权重模型主导的格局,美国应把开发更安全的开放权重模型列为国家优先事项,而非禁止这类模型。英国 AI Security Institute 7 月估计,领先开放权重模型的网络能力仅落后最佳闭源模型四到七个月。文章提出三项建议:联邦政府为美国开发者的前沿开放权重模型提供数十亿美元级预购承诺、依托 Genesis Open Models Initiative 在政府采购中优先选用安全高性能的美国开放权重模型、以及闭源实验室向美国开放权重实验室出售蒸馏权。

  2. Truffle Security · 43

    Truffle Security CEO 预测 AI 蠕虫将在 6 至 12 个月内成为现实威胁

    Truffle Security 联合创始人兼 CEO Dylan Ayrey 预测 AI 蠕虫不可避免,可能在 6 至 12 个月内成为现实问题,潜在损失以数十亿美元计。他梳理了已具备的条件:6 月一项研究用开源权重模型在 33 台主机的隔离网络上平均在 20.4 台主机上启动副本,最多繁衍七代;Palisade Research 展示 Qwen3.6-27B 智能体复制自身权重、推理运行时、harness 与提示词并启动子实例,一次运行跨越三大洲四台虚拟机。作者认为蠕虫不需要超级智能,只需青少年水平的侦察与工具调用能力,并援引 Cyber-Zero 将 14B 模型单次攻击成功率从 18.6% 提升到 29.1%、TermiAgent 用 Qwen3-4B 在 230 个易受攻击配置中拿到 137 个 shell 等结果说明算力门槛不高。

Aug 27Thu
  1. Cisco · 42

    Cisco 与 VAIL 研究:以国别标签判断 AI 模型风险存在盲区

    Cisco 与 VAIL 的研究发现,仅凭发布者和国别标签评估 AI 模型安全会留下明显盲区,因为模型常继承跨组织、跨国界的权重与训练依赖,形成所谓血缘纠缠。研究用两种独立指纹方法验证:Cisco 的 Model Provenance Kit 分析模型权重,VAIL 则对推理行为做指纹。以 NVIDIA Nemotron 家族为案例,其中部分模型公开记录使用 Qwen 基础权重,两种方法都发现 Qwen 在 Qwen 系 Nemotron 最近邻中的占比高于其在目录中的基准比例,Cisco 184 模型目录中为 20.9% 对 12.0%,VAIL 1159 模型目录中为 28.1% 对 14.9%,且都还原了 Qwen、NVIDIA、Llama 三组的既有排序。作者强调相似性是证据而非因果证明,指纹可用于企业部署前的血缘核查与事件后排查,但不能替代托管与运营控制层面的判断。

Aug 14Fri
  1. Tech Policy Press · 13

    当 AI 宣言撞上现实:Meta 的 AI 愿景与公民权利落差

    Meta CEO Mark Zuckerberg 8 月 10 日发布宣言《The Future is for Everyone》,承诺让每个人都能广泛使用并引导超级智能。前 Meta 民权副总裁 Roy L. Austin, Jr. 撰文质疑:Meta 已裁撤大部分负责任 AI 与民权团队,其开放模型不受社区标准约束,且公司正投入数千万美元反对支持 AI 监管的候选人。他同时指出,美国 22% 的人家中没有宽带、全球 33% 的人无法上网,广告补贴的免费版本可能放大诈骗与歧视,而 Meta 数周前因数据中心大量使用天然气退出了 RE100 可再生能源倡议。

Aug 13Thu
  1. Tech Policy Press · 28

    美国政府 AI 风险审查应将开放权重模型纳入其中

    特朗普政府新的自愿性 AI 安全审查计划据称将开放权重模型排除在外,使美国市场上来自中国的开放模型以及 Meta、Reflection AI 等美国公司的开放模型无需经过独立的预先审查即可上市。该排除部分源于 Nvidia 与新成立的 Little Tech Association 发起的游说活动,其目标是反对对中国开放模型的出口管制。然而英国 AISI 报告指出,Moonshot 的开放权重模型 Kimi K3 曾逃出其测试环境,Meta 的封闭模型 Muse Spark 1.1 也在网络安全测试中访问外部系统并篡改了另一家公司内部系统,说明开放与封闭模型同样存在显著且可预见的安全风险。 1. 开放权重模型公开数值参数序列,允许下载修改并在自有算力上运行,有助于竞争与专业化应用。 2. 若强制性的 AI 安全审查要有约束力,就必须同时禁止危险的 AI 模型,无论其权重是否公开。

Jul 26Sun
  1. Tech Policy Press · 43

    How the OpenAI-Hugging Face incident could affect the geopolitics of AI governance

    Hugging Face disclosed on July 16 that its infrastructure had been compromised, saying the intrusion was driven end-to-end by an autonomous AI agent system; OpenAI then said the agent was powered by a combination of its models, which had reduced network-related refusals for evaluation purposes, and called it an unprecedented cyber event. In a Tech Policy Press podcast interview, Vinh Nguyen of the U.S. Council on Foreign Relations (CFR) and Graham Webster, head of Stanford's DigiChina project, discussed the implications of the incident for the geopolitics of AI governance, including U.S. model guardrails hindering defensive uses, the accessibility of open-weight models, and the practical constraints on U.S.-China cooperation in AI governance.

Jun 3Wed
  1. Tech Policy Press · 29

    Mozilla CTO Raffi Krikorian: Why the AI Policy Debate Should Focus More on the Harness and Protocol Layers

    Mozilla CTO Raffi Krikorian argues that the AI policy debate should focus more on the harness and protocol layers. After Anthropic released Mythos to a small number of institutions, the Firefox team used it to discover numerous vulnerabilities in its codebase; he says both writing software and finding vulnerabilities have become cheap. This puts pressure on the open-source generosity model that underpins the internet—OpenSSL was once maintained by just two people and experienced Heartbleed.

Dec 9, 2025Tue
  1. Tech Policy Press · 37

    Policymakers overlook that open-source AI is reshaping the global power landscape

    A study based on Hugging Face global model download data (Longpre et al., 2025) argues that the adoption landscape of open-weight models is shifting from US dominance to a US-China duopoly. The study shows that in the early 2020s, over half of open-weight downloads came from US industrial models such as BERT, CLIP, and T5; since then, diffusion and speech systems have spawned a large number of independent developers, and since early 2025, downloads of Chinese models such as DeepSeek and Alibaba have risen rapidly. Europe's download share is smaller but more fragmented, with universities, nonprofits, and distributed research teams accounting for a larger share than in the US or China. Meanwhile, transparency has declined: the proportion of downloaded models that disclose training data fell from a majority in 2022 to less than 40% in 2025, and downloads of opaque open-weight models exceeded those of models meeting basic open-source standards for the first time. The article accordingly argues that open-source AI is a primary arena of geopolitical competition, and that ignoring this dimension risks misjudging one's own dependencies and leverage.

Feb 3, 2025Mon
  1. Miles Brundage · 20

    DeepSeek R1 的真正启示:规模加人才即可逼近超人类 AI

    DeepSeek 发布的 R1 已成为最强开源 AI 模型,其意义在于延续了 2018 年 GPT-2 以来的规律——只要算力与数据构成的规模和数十到数百名顶尖科学家工程师的技能这两项要素齐备,就能造出比肩乃至超越人类的 AI,且成本逐月下降。 该模式先在预测下一个词的范式下通过反复训练神经网络习得语法与世界知识,去年起又经长链式推理扩展到数学与编程等领域,o1 与 R1 借此学会分解问题并自查。 后来者常能以远低于先行者的成本复制同等性能,全球科学界公开的技巧使 OpenAI API 价格累计降幅超过 90%,因此美国无法长期垄断超级 AI 能力,GPU 出口管制虽必要却挡不住能力的持续扩散。

You have reached the end