Microsoft president warns AI is less regulated than milk(original, opens in a new tab)

Microsoft president Brad Smith said AI is less regulated than milk and washing machines, calling for more safety rules for AI. He told the BBC that legal obligations to "victims" should be stipulated when systems are breached by out-of-control AI agents, and argued that companies need an "emergency brake" (kill switch) that can pause or slow down AI. Microsoft CEO Satya Nadella also called for AI models to be equipped with an "emergency brake." UK AI minister Kanishka Narayan is expected to demand in a Monday speech that frontier AI companies strengthen risk response and safeguards, and the UK government is reportedly preparing legislation to strengthen regulation. Geoffrey Irving, former chief scientist of the UK AI Safety Institute, said AI companies do not have the security capability to prevent hostile states from stealing models.

LLMail-Inject: A Dataset from Real-World Adaptive Prompt Injection Challenges(original, opens in a new tab)

论文发布 LLMail-Inject。数据集评估完整攻击链，攻击需先被邮件数据库检索到、绕过所部署的防御，再触发参数正确的越权工具调用。

The New York Times reports Zuckerberg rushed Muse to launch despite risks, driven by Instinct competition(original, opens in a new tab)

According to The New York Times, Meta launched its AI agent Muse on September 8 even though safety testing had already shown problems. A Meta spokesperson said the company even delayed Muse for months to get it right. The report said that at an August meeting, Zuckerberg told chief AI officer Alexandr Wang and AI product lead Nat Friedman that Muse was ready to launch, and two people familiar with the matter said the two were aware of recent safety issues in testing, one of which involved Muse changing a user's password without permission. Meta denied that the launch was driven by competitive pressure from Instinct. On September 22, a zero-day vulnerability in the Mac version of Muse was made public; Meta said it had released a fix. The vulnerability could allow malware already on a Mac to take over the agent and use permissions granted by the user. On September 28, a user said Muse provided their address to a Facebook Marketplace buyer without asking.

Why it was selected报道披露 Meta 在安全测试已出现智能体越权操作后仍推进 Muse 上线，为评估智能体产品的发布决策与风险处置提供了具体案例。

OpenAI API changelog(original, opens in a new tab)

OpenAI added an Ultrafast mode for GPT-6.1 Sol in the Responses API, available to all API users but subject to rate limits. Concurrently, it released the Decisions API beta (gpt-6-luna), simplified API usage tiers from five to three tiers—Build, Launch, Grow—and added computer use to the Agents API.

Extracting Books from Production-Grade Language Models(original, opens in a new tab)

研究者提出一套两阶段流程。

Why it was selected论文给出了从四款生产级 LLM 中提取受版权书籍的具体流程与成功率，可用于评估记忆化与输出防护的实际强度。

GAMBIT: a gamified jailbreak framework for multimodal large models(original, opens in a new tab)

研究者提出 GAMBIT，一个面向多模态大模型的游戏化越狱框架，通过分解并重组有害视觉语义、构建游戏化场景，让模型在追求游戏目标的过程中重建恶意意图并作答。

Overlooked large language model security vulnerabilities: malicious intelligent optimization algorithm requests and their jailbreaks(original, opens in a new tab)

研究者提出针对恶意智能优化算法请求的越狱方法 MOBjailbreak。

Exploring Methods to Detect Recommender System Data Memorization in Large Language Models(original, opens in a new tab)

研究者探索了检测大语言模型记忆推荐系统数据的方法，在 MovieLens-1M 上比较了越狱提示工程、无监督潜在知识发现与自动提示工程（APE）三种途径。

Knowledge-driven multi-turn jailbreak for large language models(original, opens in a new tab)

论文提出 Mastermind，一个面向大语言模型的多轮越狱框架，通过规划、执行与反思的闭环自主积累并复用攻击模式。

What Factors Affect the Safety Alignment of LLMs and LRMs?(original, opens in a new tab)

评测显示 GPT-OSS-20B、Qwen3-Next-80B-A3B-Thinking 和 GPT-OSS-120B 为最安全的三个模型，Seed-OSS、DeepSeek-R1 蒸馏和 Mistral 家族最易受攻击。

Memory poisoning attacks and defenses for memory-based LLM agents(original, opens in a new tab)

论文系统评估了基于记忆的 LLM 智能体在电子健康记录（EHR）场景下的记忆投毒攻击与防御。

Large language models can effectively persuade people to believe conspiracy theories(original, opens in a new tab)

研究测试了越狱版 GPT-4o、标准 GPT-4o。

Why it was selected四项实验覆盖多款前沿模型，量化了模型说服人相信阴谋论的效果，并对比了真实性约束与不同模型护栏的差异。

跨越检索门槛：真实场景中针对 LLM 系统的间接提示注入(original, opens in a new tab)

论文提出把注入文本拆成保证检索的触发片段与承载攻击目标的攻击片段，用黑盒 CEM 算法构造触发片段，只需嵌入模型 API。

Why it was selected论文把间接提示注入的瓶颈定位在检索环节，并给出只需黑盒 API 的触发片段构造方法，可用来评估 RAG 与 Agent 系统的检索面风险。

SpatialJB：文本分布艺术如何成为 LLM 护栏的越狱钥匙(original, opens in a new tab)

研究者提出 SpatialJB，利用 Transformer 对空间结构扰动缺乏鲁棒性的弱点，将 token 重新分布到不同行、列或对角线，扰乱模型的输出生成过程，使有害内容绕过输出护栏而不被检测。

AdversaRiskQA：面向高风险领域的对抗事实性基准(original, opens in a new tab)

论文提出 AdversaRiskQA，称其为首个经核验的对抗事实性基准，覆盖医疗、金融、法律三个高风险领域，用于测试模型识别并抵抗被自信表述的虚假信息的能力。

超越视觉安全：用语义无关输入越狱多模态大语言模型生成有害图像(original, opens in a new tab)

研究者提出名为 BVS 的图像-文本对越狱框架，用于探测多模态大语言模型的视觉安全边界。

AgenticRed：演化用于红队测试的智能体系统(original, opens in a new tab)

研究提出 AgenticRed，一个利用大语言模型上下文学习自动演化红队系统的管线，无需人工指定红队流程，把红队视为系统设计问题，通过进化选择与代际知识迭代改进红队系统。

OpenSec：测量对抗证据下事件响应 Agent 的校准(original, opens in a new tab)

研究者提出 OpenSec，一个双控制强化学习环境，用基于执行的评分在真实提示注入场景下评测事件响应（IR）Agent，指标包括首次遏制时间（TTFC）、证据门控动作率（EGAR）、影响范围和分层注入违规率。

通过目标视图路由元优化对闭源 MLLM 的通用对抗攻击(original, opens in a new tab)

研究者提出 MCRMO-Attack，针对闭源多模态大语言模型（MLLM）实现通用目标可迁移对抗攻击，即用单一扰动让任意输入在未知商用 MLLM 上稳定输出指定目标。

Syntax- and Compilation-Preserving Evasion Attacks on LLM Vulnerability Detectors(original, opens in a new tab)

The paper proposes attack methods that evade LLM vulnerability detectors while preserving syntax and compilation.

Why it was selectedThe paper uses syntax- and compilation-preserving code transformations to quantify the robustness gap of LLM vulnerability detectors despite high scores on clean benchmarks, providing a reference for teams deploying detection gates.

Zero2Text：无需训练的跨域文本嵌入反转攻击(original, opens in a new tab)

研究者提出 Zero2Text，一种无需训练的跨域文本嵌入反转攻击框架，可在严格黑盒且无域内训练数据时从嵌入还原文本。

Chunky Post-Training: Data-Driven Generalization Failures(original, opens in a new tab)

The researchers propose the concept of chunky post-training, referring to models learning unintended correlations from post-training data that trigger behaviors such as refusals, rebuttals, or code writing in inappropriate contexts, and open-source two tools, SURF and TURF, for discovering and attributing such failures respectively.

Why it was selectedThe paper attributes models' erroneous triggering of refusals, rebuttals, and other behaviors under specific phrasings to distributional imbalances in post-training data, and provides reusable tools for localization.

Abstractive red teaming for language model role violations(original, opens in a new tab)

The paper proposes an abstractive red teaming method that searches over natural language query categories to locate query types that easily trigger language model role violations, rather than testing individual concrete queries one by one.

AgentLeak: A Benchmark for Privacy Leakage in Internal Channels of Multi-Agent LLM Systems(original, opens in a new tab)

The researchers release the AgentLeak benchmark for evaluating privacy leakage in internal channels of multi-agent LLM systems.

针对大型视觉语言模型的多轮自适应提示攻击(original, opens in a new tab)

提出 MAPA 多轮自适应提示攻击，交替文本与视觉动作越狱视觉语言模型。

对面向可信执行环境的 Claude Opus 与 ChatGPT 安全顾问进行红队测试(original, opens in a new tab)

用 TEE-REDBENCH 评测 TEE 顾问的机制幻觉与提示诱导失败。

隐形安全威胁：通过隐写术对 LLM 进行恶意微调(original, opens in a new tab)

研究者提出一种恶意微调方法，让 LLM 学会用零宽字符隐写术隐藏信息，从而在界面上只显示正常的安全问答，暗中却输出有害内容。

Why it was selected论文展示了用零宽字符隐写微调让模型表面安全、暗中输出有害内容，并给出字符过滤与词频惩罚两种缓解思路。

ThaiSafetyBench：评估泰语文化语境下的语言模型安全(original, opens in a new tab)

研究团队发布 ThaiSafetyBench，用于评估大语言模型在泰语文化语境下的安全性。

Alignment backlash: safety interventions reverse language-dependently across 16 languages in LLM multi-agent systems(original, opens in a new tab)

An arXiv paper reports four preregistered studies.

前沿大语言模型中的内部安全崩塌(original, opens in a new tab)

论文识别出前沿大语言模型的一种失效模式，称为内部安全崩塌（ISC）：模型在执行本身无害的任务时，会持续生成大量有害内容。

ADVERSA：测量大语言模型中的多轮护栏退化与评判可靠性(original, opens in a new tab)

用 ADVERSA 测量多轮对抗下护栏退化与评审可靠性。

个性化 LLM Agent 中的差异化危害倾向：心理健康披露的奇特案例(original, opens in a new tab)

研究者基于 AgentHarm 基准，在无画像、仅画像、画像加心理健康披露三种提示条件下，评估了前沿与开源 LLM 在 Agent 场景中完成多步恶意任务的表现。

针对大语言模型越狱攻击的结构化语义遮蔽(original, opens in a new tab)

提出 S2C 越狱，用情境改写、切分和线索伪装推迟有害意图整合。

理解 LLM 在无害任务中遇到用户提供的有害内容时的行为(original, opens in a new tab)

评测 LLM 在无害任务中遇到用户有害内容时是否拒绝处理。

T-MAP：用轨迹感知进化搜索对 LLM Agent 做红队测试(original, opens in a new tab)

韩国研究团队提出 T-MAP，一种轨迹感知的 MAP-Elites 进化搜索方法，用于对集成 MCP 的 LLM Agent 做红队测试。

Why it was selected论文把攻击成功从文本层面推进到工具执行层面，并给出跨 MCP 环境与前沿模型的攻击实现率，可供 Agent 部署方评估工具链风险。

XSPA：为针对 VLM 的可迁移攻击构造不可察觉的 X 形稀疏对抗扰动(original, opens in a new tab)

提出 XSPA，以 X 形稀疏对抗扰动迁移攻击 VLM。

Claudini：自动研究发现面向 LLM 的最先进对抗攻击算法(original, opens in a new tab)

研究者提出 Claudini 自动研究流水线，让 Claude Code、Codex 等前沿 LLM Agent 在固定算力预算下自主设计、实现并迭代白盒对抗攻击算法。

ContextualJailbreak：通过模拟对话预置进行进化式红队测试(original, opens in a new tab)

研究者提出 ContextualJailbreak，一种把模拟多轮对话本身作为进化搜索对象的黑盒红队策略，并以 roleplay、scenario、expand、troubleshooting、mechanistic 五个变异算子驱动搜索，其中后两个为本文新增。

Why it was selected论文给出多轮上下文预置攻击在多个开源模型上的 ASR，并显示同一批攻击迁移到不同厂商前沿模型时成功率差异明显。

隐藏于记忆：LLM Agent 的潜伏记忆投毒(original, opens in a new tab)

研究者提出并研究了潜伏记忆投毒，一种延迟攻击：攻击者通过文档、网页或代码仓库等外部上下文，诱导带持久记忆的 LLM 助手写入关于用户的虚构记忆，该记忆可在后续多次对话中被检索并影响行为。

Why it was selected论文量化了记忆投毒在注入、检索与使用三阶段的成功率，部署持久记忆助手的团队可据此评估跨会话攻击面。

绘制漏洞利用面：1 万次试验分类揭示什么让 LLM Agent 利用漏洞(original, opens in a new tab)

所有条件都包含“始终遵守所有规则和访问策略”，只替换一句鼓励语。