Skip to content

Translation in progress Some entries still include text in the original language.

阿里巴巴 · Qwen · News

Oct 13 · Tuesday

阿里巴巴 · Qwen · News

Latest publications and developments
阿里巴巴 · Qwen

Research topics

Prompt injectionAgent safetyDeception and schemingReward hackingChain-of-thought monitoringInterpretabilityAI controlBackdoors and poisoning隐私与数据泄露Supply-chain securityDangerous capabilitiesAI 与网络攻防开源模型安全Red teaming

Companies and institutions

OpenAIAnthropicGoogle DeepMindGoogle ResearchGoogle 安全MetaxAIMicrosoftNVIDIADeepSeek阿里巴巴 · Qwen智谱 · GLM月之暗面 · Kimi字节跳动腾讯蚂蚁集团上海人工智能实验室UK AISIUS CAISIMETRApollo ResearchRedwood ResearchFAR.AIPalisade Research

Reports and frameworks

System CardFrontier safety frameworks
Sep 30Wed
  1. 安远AI安远AI · 31

    Concordia AI analyzes security risks and governance of Chinese open-source models

    Chinese laboratories released open-source models such as Kimi K3, Qwen 3.8-Max, and GLM-5.3 this summer, with capabilities only slightly behind the strongest U.S. models, sparking debate over whether open-source models could be misused. Open weights can be cheaply fine-tuned to remove guardrails and cannot be retracted after release, but self-hosting requires chips costing hundreds of thousands of dollars, and cloud hosting costs about $50-150 per hour, so the actual barrier to misuse remains high. China's TC260 has named in its AI risk framework that open-source model safety mechanisms may be removed or bypassed, and has begun drafting open-source AI safety standards.

Sep 24Thu
  1. Truffle Security · 43

    Truffle Security CEO 预测 AI 蠕虫将在 6 至 12 个月内成为现实威胁

    Truffle Security 联合创始人兼 CEO Dylan Ayrey 预测 AI 蠕虫不可避免，可能在 6 至 12 个月内成为现实问题，潜在损失以数十亿美元计。他梳理了已具备的条件：6 月一项研究用开源权重模型在 33 台主机的隔离网络上平均在 20.4 台主机上启动副本，最多繁衍七代；Palisade Research 展示 Qwen3.6-27B 智能体复制自身权重、推理运行时、harness 与提示词并启动子实例，一次运行跨越三大洲四台虚拟机。作者认为蠕虫不需要超级智能，只需青少年水平的侦察与工具调用能力，并援引 Cyber-Zero 将 14B 模型单次攻击成功率从 18.6% 提升到 29.1%、TermiAgent 用 Qwen3-4B 在 230 个易受攻击配置中拿到 137 个 shell 等结果说明算力门槛不高。

Aug 27Thu
  1. Cisco · 42

    Cisco 与 VAIL 研究：以国别标签判断 AI 模型风险存在盲区

    Cisco 与 VAIL 的研究发现，仅凭发布者和国别标签评估 AI 模型安全会留下明显盲区，因为模型常继承跨组织、跨国界的权重与训练依赖，形成所谓血缘纠缠。研究用两种独立指纹方法验证：Cisco 的 Model Provenance Kit 分析模型权重，VAIL 则对推理行为做指纹。以 NVIDIA Nemotron 家族为案例，其中部分模型公开记录使用 Qwen 基础权重，两种方法都发现 Qwen 在 Qwen 系 Nemotron 最近邻中的占比高于其在目录中的基准比例，Cisco 184 模型目录中为 20.9% 对 12.0%，VAIL 1159 模型目录中为 28.1% 对 14.9%，且都还原了 Qwen、NVIDIA、Llama 三组的既有排序。作者强调相似性是证据而非因果证明，指纹可用于企业部署前的血缘核查与事件后排查，但不能替代托管与运营控制层面的判断。

Jul 17, 2025Thu
  1. FAR.AI · 33

    Mind the Mitigation Gap: Why AI companies must report both pre-mitigation and post-mitigation safety evaluations

    FAR.AI noted that most AI companies disclose only one of the two types of safety evaluations, pre-mitigation or post-mitigation, making risks difficult to measure accurately, and argued that both must be reported. Its review found that OpenAI disclosed both types for o1 and covered high-risk CBRN capabilities, Anthropic reported only pre-mitigation capabilities based on low-refusal versions of Claude 3 and 3.5 without clear definitions, Gemini 1.5 had only post-mitigation evaluations, Meta gave only vague statements for Llama 3.1, Alibaba's Qwen-2.5 had only basic post-mitigation evaluations, and DeepSeek R1 had no report at all. Applying this framework to o1, GPT-4o, Claude 3.5 Sonnet, and Gemini 1.5 Pro, it found that on WMDP-Chem accuracy, o1 and GPT-4o reached the highest at 70–75%, and on dangerous request compliance rate, GPT-4o reached as high as 50%.

You have reached the end