Routing-aware Safety Alignment for Mixture-of-Experts Models(original, opens in a new tab)

Propose routing-aware safety alignment RASA, which fine-tunes only jailbreak-activated experts and constrains routing.

How Few-shot Examples Affect Prompt-based Defenses Against LLM Jailbreak Attacks(original, opens in a new tab)

Analyzing the effect of few-shot demonstrations on the jailbreak resistance of RoP and ToP prompt defenses.

Steering Externalities: Benign Activation Steering Inadvertently Increases Jailbreak Risk in Large Language Models(original, opens in a new tab)

分析良性激活转向如何削弱拒绝并放大越狱风险。

An empirical study on noisy data and LLM pretraining loss divergence(original, opens in a new tab)

Analyze the impact of controlled uniform random noise on pretraining loss divergence.

David vs. Goliath: Verifiable Agent-to-Agent Jailbreaks via Reinforcement Learning(original, opens in a new tab)

提出 SLINGSHOT，用工具执行奖励强化学习训练智能体间越狱。

Why it was selectedThe paper uses reinforcement learning to train attacker agents and quantifies the success rate and cross-model transferability of jailbreaks between agents in a verifiable tool execution environment.

Zero2Text: Training-Free Cross-Domain Text Embedding Inversion Attack(original, opens in a new tab)

Propose Zero2Text, which inverts text embeddings across domains without training.

Efficient Adversarial Attacks on High-Dimensional Offline Bandits(original, opens in a new tab)

Propose three types of attacks that perturb the reward model before training to hijack offline bandits.

PAIR 用不到 20 次查询越狱黑盒大语言模型(original, opens in a new tab)

PAIR（Prompt Automatic Iterative Refinement）用攻击者 LLM 与目标模型迭代对抗，把黑盒大语言模型的语义越狱压缩到不到 20 次查询。攻击者根据目标模型的拒答结果做链式推理并改写提示词，默认开 30 条并行流、迭代 3 次，由 Llama Guard 担任判官评分，常用策略包括角色扮演、逻辑诉求和引用权威机构。论文称其效率比需要 256,000 次以上查询的梯度类 GCG 方法高出约 250 倍，单次越狱约 34 秒、成本低于 0.03 美元。论文给出的成功率为 Vicuna-13B 88%、Gemini-Pro 73%、GPT-3.5 51%、GPT-4 48%，并称 PAIR 是首个攻破 Gemini-Pro 的自动化攻击。Llama-2 与 Claude-1/2 因过度拒答而未被攻破，论文认为这种安全换有用性的取舍构成对齐税。

Towards Universal Transferable Jailbreak Attacks for Vision-Language Models(original, opens in a new tab)

提出 UltraBreak，用单张对抗图像对视觉语言模型实施可迁移越狱。

GradingAttack: Revealing Security Vulnerabilities in LLM-Based Educational Grading Agents(original, opens in a new tab)

Propose GradingAttack, using gradient suffixes and role-playing to manipulate educational grading agent judgments.

微调对齐后的大语言模型会损害安全，即便用户并无恶意(original, opens in a new tab)

一篇论文发现，仅用 10 条对抗设计的训练样本微调 GPT-3.5 Turbo，成本不足 0.20 美元，即可绕过其安全护栏，使模型几乎响应任何有害指令。研究还显示，即便没有恶意意图，用常见的良性数据集微调也会在较小程度上削弱模型的安全对齐。作者认为，微调对齐后的大语言模型带来了现有安全基础设施未能覆盖的新风险，初始对齐再完善也未必能在定制微调后保持。论文分析了潜在缓解措施，并呼吁针对定制微调加强安全协议。

Why it was selected论文用不足 0.20 美元的成本在 10 条样本上微调即削弱 GPT-3.5 Turbo 的安全对齐，为微调环节的持续安全评估提供了具体依据。

The study proposes the persuasion propagation phenomenon: task-irrelevant persuasive dialogue changes the task execution behavior of AI agents(original, opens in a new tab)

Measures the impact of task-irrelevant persuasion on agent coding and web research behavior.

Enhancing jailbreak attacks and defenses from a causal perspective(original, opens in a new tab)

Propose Causal Analyst, which rewrites prompts according to causal features to enhance jailbreaks.

Bypassing prompt injection detectors through evasive injection(original, opens in a new tab)

提出多探针规避攻击，用通用对抗后缀绕过提示注入检测探针。

Jailbreaking LLMs via Calibration(original, opens in a new tab)

Propose Gradient Shift inference-time aggregation to pull the aligned model back to suppressed behavior.

Researchers propose Text-DJ, jailbreaking large vision-language models by exploiting OCR capabilities(original, opens in a new tab)

Proposes Text-DJ, jailbreaking vision-language models with split text-grid images.

SmoothLLM：防御大语言模型的越狱攻击(original, opens in a new tab)

SmoothLLM 是一种针对越狱攻击的防御算法，通过对同一提示词生成多份字符级随机扰动副本并聚合预测，来检测并拦截对抗输入。论文称这是首个针对越狱攻击的防御算法，其依据是越狱提示对字符级改动很脆弱，而正常提示几乎不受错别字影响。论文称该方法在 GPT、Llama、Claude 等模型上对 GCG、PAIR、RandomSearch、AmpleGCG 越狱达到当时最优鲁棒性，并对自适应 GCG 攻击有抵抗力。它无需重训练，兼容任意 LLM，代价是鲁棒性与正常性能之间存在小幅但不可忽略的权衡，代码已在 GitHub 公开。

Safer by Diffusion, Broken by Context: The Safety Dividend and Failure Modes of Diffusion LLMs(original, opens in a new tab)

Propose black-box context nesting, nesting harmful requests in structured completion tasks to jailbreak diffusion LLMs.

Syntax- and Compilation-Preserving Evasion Attacks on LLM Vulnerability Detectors(original, opens in a new tab)

The paper proposes attack methods that evade LLM vulnerability detectors while preserving syntax and compilation.

Why it was selectedThe paper uses syntax- and compilation-preserving code transformations to quantify the robustness gap of LLM vulnerability detectors despite high scores on clean benchmarks, providing a reference for teams deploying detection gates.

The researchers propose an audio narrative jailbreak attack, achieving a success rate of 98.26% on Gemini 2.0 Flash(original, opens in a new tab)

Propose audio narrative jailbreak, bypassing end-to-end speech model refusals via voice delivery style.

Personality latent variables in large language models: a mechanistic explanation of emergent misalignment and conditional safety failures(original, opens in a new tab)

用性格潜变量与表征投影解释涌现失配和条件失效。

Why it was selectedThe paper explains emergent misalignment as personality representations acquired through fine-tuning, and uses incorrect-advice data as a control, showing that personality-conditioned data produces stronger and more transferable misalignment.

Filtering the Noise: A Comparative Study of LLM Agents in Vulnerability False Positive Filtering(original, opens in a new tab)

Evaluates the ability of coding agents to filter static analysis alert false positives.

Statistical estimation of adversarial risk of large language models under Best-of-N sampling(original, opens in a new tab)

Propose SABER, which extrapolates Best-of-N jailbreak risk from small-budget observations.

Whispers of Wealth: Red-teaming Google's Agent Payments Protocol via prompt injection(original, opens in a new tab)

提出两种签名前提示注入，操纵支付 Agent 的商品排序与跨用户数据披露。

针对对齐语言模型的对抗攻击基线防御(original, opens in a new tab)

针对对齐大语言模型的对抗攻击基线防御研究评估了困惑度过滤、复述、重分词与对抗训练四类方法。结果显示这些简单防御能部分降低攻击成功率，但都无法彻底消除脆弱性，并带来延迟、算力开销或误报。在 Vicuna-7B 上，复述把成功攻击从 74 降到 5；困惑度过滤把 ASR 从 0.79 降到 0.00，但约 10% 的正常查询被误判。研究认为文本优化器成本远高于图像优化器，使当前防御相对有效，一旦出现更高效的文本优化算法，这一优势可能消失。

ReasoningBomb: A Covert Denial-of-Service Attack Inducing Pathologically Long Reasoning in Large Reasoning Models(original, opens in a new tab)

提出 ReasoningBomb，用强化学习生成短提示诱导推理模型病态长推理。

Why it was selectedThe paper treats the reasoning time of reasoning models as an attack surface and quantifies the effect of short prompts in amplifying reasoning cost, which can help deployers assess service cost risk.

CAR-bench: Evaluating LLM Agents' Consistency and Capability Boundary Awareness under Real-World Uncertainty(original, opens in a new tab)

提出 CAR-bench，评测车载多轮智能体的一致性与限度意识。

Stay in Character, Stay Safe: A Dual-Loop Adversarial Self-Evolving Framework for Safe Role-Playing Agents(original, opens in a new tab)

Propose DASE, using an adversarial self-evolving knowledge base to harden role-play against jailbreaks at inference time.

Compliance Paradox: Semantic-Instruction Decoupling in Automated Academic Code Evaluation(original, opens in a new tab)

Propose AST-ASIP, implanting adversarial instructions into compiler-ignored regions to hijack code scoring.

Just Ask: Curious Code Agents Reveal System Prompts of Frontier LLMs(original, opens in a new tab)

Propose JustAsk, using online skill selection to extract model system prompts.

《Do Anything Now》：真实场景中大语言模型越狱提示词的特征刻画与评估(original, opens in a new tab)

论文用 JailbreakHub 框架分析了 2022 年 12 月至 2023 年 12 月的 1,405 条越狱提示词，识别出 131 个越狱社区，并发现最有效的提示词组合了角色扮演、权威框架、情感诉求与技术误导等策略。研究构建了覆盖 13 个禁止场景、共 107,250 个样本的问题集，在六个主流大语言模型上测试，发现其安全防护无法在所有场景下抵御越狱提示词。其中五条高效越狱提示词在 ChatGPT（GPT-3.5）和 GPT-4 上达到 0.95 的攻击成功率，最早的一条已在线存在超过 240 天。作者认为，社区协作迭代发现漏洞的速度快于学术攻击研究。

OpenSec: Measuring Calibration of Incident Response Agents under Adversarial Evidence(original, opens in a new tab)

Propose OpenSec, evaluating incident response agents' containment calibration under adversarial evidence.

Goodfire 用可解释性从表观遗传基础模型 Pleiades 中找出新型阿尔茨海默病生物标志物(original, opens in a new tab)

Goodfire 将可解释性方法应用于 Prima Mente 的表观遗传基础模型 Pleiades，发现 DNA 片段长度模式主导其从血液游离 DNA 检测阿尔茨海默病的决策，并由此提炼出一个人类可解释的分类器。该分类器仅用片段长度特征即在独立队列上取得 AUROC=0.78（95% CI [0.56–0.95]），加入此前文献报道的生物标志物类别后升至 AUROC=0.84（95% CI [0.66, 0.97]）。研究表明片段组学可作为互补于既有预测指标的新型候选信号，但其临床应用仍需进一步验证。

Proactively hardening LLM defenses with HASTE(original, opens in a new tab)

Propose HASTE, which mines hard negatives to retrain a malicious prompt detector.

针对 LLM 集成应用的提示注入攻击(original, opens in a new tab)

Liu 等研究者提出 HouYi 黑盒提示注入攻击，在 36 个真实 LLM 集成应用中 31 个存在漏洞，成功率达 86.1%。该攻击由框架、分隔符和破坏载荷三部分组成，通过语法转义、语言切换或语义方式诱导模型把后续文本当作新指令，可窃取开发者预设的系统提示词，或滥用服务商的 LLM 额度执行任意任务。Notion、Writesonic、Parea 等 10 家厂商确认了漏洞；研究者按 GPT-3.5 turbo 定价估算，单个应用被大规模利用每天约损失 259.20 美元。研究还称指令防御、XML 标签、三明治防御等现有防护可被绕过，建议采用多步处理、领域专用模型和输出监控。

Why it was selected论文用 HouYi 在 36 个真实应用上取得 86.1% 成功率，并给出提示泄露与资源滥用两类后果，可供部署方对照检查输入隔离假设。

Unknown Unknowns: Can Hidden Intents in LLMs Escape Detection?(original, opens in a new tab)

构建隐匿意图测试床，压力测试分类器与 LLM 裁判。

Why it was selectedThe paper uses a controlled testbed to quantify the precision collapse of hidden intent detection at low prevalence, providing an empirical reference for the capability boundaries of auditing methods.

MalURLBench: A benchmark for evaluating agent vulnerabilities when processing web URLs(original, opens in a new tab)

Propose MalURLBench to evaluate whether agents accept structurally disguised malicious URLs.

AttenMIA: Membership Inference Attacks on LLMs Using Attention Signals(original, opens in a new tab)

Propose AttenMIA, which uses attention signals for white-box membership inference.

When Personalization Justifies Risk: Uncovering Security Vulnerabilities in Personalized Conversational Agents(original, opens in a new tab)

用 PS-Bench 评测良性记忆如何让对话智能体放过有害请求。

Breaking the Protocol: Security Analysis of the MCP Specification and Prompt Injection Vulnerabilities in Tool-Integrated LLM Agents(original, opens in a new tab)

提出 ATTESTMCP，为 MCP 增加能力证明与消息认证。