Physical prompt injection attacks against large vision-language models(original, opens in a new tab)

提出物理提示注入 PPIA，用印刷文字操纵视觉语言模型。

SycoEval-EM: Evaluating Large Language Model Sycophancy in Simulated Emergency Clinical Scenarios(original, opens in a new tab)

Build SycoEval-EM to evaluate sycophantic compliance in simulated emergency multi-round consultations.

Do VLMs have a moral backbone? A study on the moral fragility of vision-language models(original, opens in a new tab)

Evaluate the perturbation robustness of VLM moral judgment on Moralise.

Attributing and Exploiting Safety Vectors in Large Language Models via Global Optimization(original, opens in a new tab)

Propose GOSV, which globally optimizes to locate safety attention heads and re-attaches activations to jailbreak at inference time.

AdversaRiskQA: An Adversarial Factuality Benchmark for High-Risk Domains(original, opens in a new tab)

Propose AdversaRiskQA to evaluate whether models in high-risk domains can correct erroneous facts in prompts.

NeuroFilter: Activation-value guardrails for privacy-conscious LLM agents(original, opens in a new tab)

Propose NeuroFilter, using activation probes to filter multi-turn privacy attacks against agents.

Clever Side Effects: Safety Risks in Multi-Image Reasoning of Multimodal Large Models(original, opens in a new tab)

清华大学 CoAI 团队构建 MIR-SafetyBench，评测多图推理中的安全风险。

VirtualCrime: Evaluating LLMs' Criminal Potential and Agent Behavior through Sandbox Simulation(original, opens in a new tab)

Use the VirtualCrime sandbox to evaluate models' multi-step execution and excessive harm when assigned criminal roles.

Are LLMs' Stated Beliefs Fragile? Probing LLM Belief Resistance with Strategic Persuasive Dialogue Interventions(original, opens in a new tab)

Evaluate the resistance of LLMs' stated beliefs under multi-turn strategic persuasion.

When Wording Sways Evaluation: Framing Bias in LLM Judges(original, opens in a new tab)

Measure the inconsistency of LLM judges' decisions under predicate polarity flips.

AgenticRed: Evolving agentic systems for red teaming(original, opens in a new tab)

Proposes AgenticRed, evolutionary search of single-turn jailbreak agent workflows.

OI-Bench: An option injection benchmark for evaluating LLM susceptibility to instruction interference(original, opens in a new tab)

Construct OI-Bench to evaluate LLM susceptibility to option-injection instruction interference.

Insider knowledge: How much can RAG systems gain from evaluation secrets?(original, opens in a new tab)

Measure the extent to which RAG systems exploit evaluation secrets to inflate automated evaluation scores.

ChartAttack: Testing Large Language Models' Vulnerability to Malicious Prompts in Chart Generation(original, opens in a new tab)

Propose ChartAttack, using malicious prompts to drive code models to generate misleading charts.

CORVUS: Using LoRA to disguise internal signals and bypass single-forward hallucination detectors(original, opens in a new tab)

Researchers from Singapore Management University and other institutions propose CORVUS, using LoRA to disguise internal telemetry to bypass hallucination detectors.

Why it was selectedThe paper provides a complete method for white-box model-side attacks, transfer results across four open-source models, and item-by-item ablations, allowing assessment of the reliability boundaries of internal telemetry-based hallucination detectors.

VILTA: Incorporating VLM into closed-loop training to improve driving policy robustness(original, opens in a new tab)

Proposes VILTA, which uses a VLM to edit neighboring vehicle trajectories in closed-loop training to improve driving policy robustness.

Through internal state analysis and clustering visualization, evaluate LLM factual hallucination tendencies(original, opens in a new tab)

Propose the FalseCite benchmark to evaluate how fabricated citations change factual hallucinations.

TrojanPraise: Jailbreaking LLMs via Benign Fine-Tuning(original, opens in a new tab)

提出 TrojanPraise，用新词赞美的良性微调越狱 LLM。

Why it was selectedThe paper presents a benign fine-tuning jailbreak method that bypasses data review and explains its cause using the two representation dimensions of knowledge and attitude, which can help fine-tuning service providers assess filtering blind spots.

Containing multiple privilege escalations in LLM agent systems: a mandatory access control framework(original, opens in a new tab)

提出 SEAgent 强制访问控制框架，拦截 Agent 越权工具调用。

Faithfulness vs. Safety: Evaluating LLM Behavior under Counterfactual Medical Evidence(original, opens in a new tab)

Construct MEDCOUNTERFACT to evaluate whether the model still answers according to the evidence under counterfactual medical evidence.

Hidden-in-Plain-Text: A benchmark for indirect prompt injection in social network RAG(original, opens in a new tab)

Use OpenRAG-Soc to evaluate indirect prompt injection and retrieval poisoning in social web page RAG.

ToolSafe 提出步骤级护栏 TS-Guard 与 TS-Flow，降低 Agent 有害工具调用(original, opens in a new tab)

提出 TS-Guard 与 TS-Flow，在执行前检测并引导 Agent 修正有害工具调用。

Blue Team Testing Function-Calling Agents(original, opens in a new tab)

Propose Tool Obfuscation and evaluate the attack and defense of function-calling agents.

STaR: A sensitive trajectory regulation method for unlearning in large reasoning models(original, opens in a new tab)

Propose the inference-time framework STaR to suppress sensitive content in the reasoning traces of large reasoning models.

BenchOverflow: Measuring Output Overflow in Large Language Models with Plain-Text Prompts(original, opens in a new tab)

提出 BenchOverflow 基准测量纯文本提示诱发的输出膨胀。

Toolformer：语言模型能自学使用工具(original, opens in a new tab)

Schick 等人的 Toolformer 让 GPT-J 6.7B 自行为文本生成 API 调用标注，只保留能降低后续 token 困惑度的调用，从而学会使用计算器、Wikipedia 搜索等五种工具，并在与工具相关的任务上超过 GPT-J 与 GPT-3 175B。作者称，模型由此从被动文本生成转向自主调用工具，安全焦点也从内容生成转向行动安全。

False Friends in Shell: Revealing Emoji Semantic Confusion in Large Language Models(original, opens in a new tab)

Constructing a code-oriented benchmark to evaluate ASCII emoticons being misread as executable commands.

Constitutional AI：用 AI 反馈实现无害性(original, opens in a new tab)

Bai 等人提出 Constitutional AI：让语言模型按一套成文原则（宪法）批评并修订自身输出，再用这些自生成偏好标签做 RLHF。分两阶段，SL-CAI 用批评—修订对做监督学习，RL-CAI 由另一模型依宪法判断哪条回复更好，生成偏好标签。作者称，CAI 模型比仅用人工反馈的 RLHF 模型更有帮助且更少有害，但该法假设模型有能力准确应用宪法原则，对部分危害类型或更小模型未必成立。

How Safe Is Safe Code Generation? Adversarial Prompts Test LLM Defenses(original, opens in a new tab)

Uses adversarial prompts to audit the safety and functionality of secure code generation methods after natural language rewriting.

HELM：语言模型的整体评估(original, opens in a new tab)

HELM 在 30 个模型、42 个场景上开展多维度评测，发现没有任何模型在所有维度占优。评测最多采用准确率、校准、鲁棒性、公平性、偏见、毒性和效率 7 项指标，准确率领先的模型常在公平性、偏见或校准上排名靠后，鲁棒性与准确率独立变化，输入扰动下性能明显下降。指令微调与 RLHF 总体提升性能，却引入校准上的新失败模式，模型变得过度自信。

Adversarial Robustness of 3D Large Vision-Language Models(original, opens in a new tab)

Proposes Vision and Caption attacks, perturbing point clouds to manipulate 3D VLM outputs.

论文：Chung 等把指令微调扩展至 1,836 个任务，训出 Flan-PaLM 与 Flan-T5(original, opens in a new tab)

Chung 等人的论文把指令微调扩展到 1,836 个任务并混入思维链示例，训练出 Flan-PaLM 540B 与 Flan-T5。论文称 Flan-PaLM 540B 在 MMLU、BBH、TyDiQA、MGSM 等基准上取得 SOTA，Flan-T5-XL（3B）常能追平或超过 PaLM 62B。作者指出，模型越忠实地遵循指令，也就越容易被精心构造的对抗指令利用。

VIGIL: defending LLM agents against tool-flow injection with "verify before submit"(original, opens in a new tab)

Proposes VIGIL, which verifies before submitting, to defend against injection in agent tool flows.

Memory poisoning attacks and defenses for memory-based LLM agents(original, opens in a new tab)

提出输入输出审核与信任感知记忆消毒，防御智能体记忆投毒。

Knowledge-driven multi-turn jailbreak for large language models(original, opens in a new tab)

Proposes Mastermind, a knowledge-driven multi-turn jailbreak that attacks aligned models with a closed loop of strategy knowledge.

Anthropic 红队测试语言模型以减少危害：方法、规模行为与经验教训(original, opens in a new tab)

Anthropic 招募 324 名参与者对 2.8B 至 52B 参数模型发起 38,961 次红队攻击，攻击成功率从基础模型的约 25% 降至 RLHF 模型的 6-8%。红队成员之间的差异远大于模型规模差异，少数高手发现多数严重漏洞；显式内容与歧视性语言比危险活动、自残指令更易被诱导，总体成功率会掩盖分类别风险。

The Persona Paradox: Medical Personas as Behavioral Priors for Clinical Language Models(original, opens in a new tab)

Analyzes the non-monotonic effects of a medical persona in system prompts on clinical triage and safety advice.

Multi-turn jailbreak attacks on multimodal large language models and FragGuard defense(original, opens in a new tab)

Proposes multi-turn multimodal jailbreak and uses FragGuard to intercept harmful responses.

PC²: Generating Politically Controversial Content via Jailbreak Attacks on GPT-Based Text-to-Image Models(original, opens in a new tab)

Proposes the PC² jailbreak, which uses identity-preserving descriptions and remote multilingual translation to bypass text-to-image filters.

Large language models can effectively persuade people to believe conspiracy theories(original, opens in a new tab)

测量对话 LLM 被指示后提高阴谋论信念的能力。

Why it was selectedFour experiments covering multiple frontier models quantify the effect of models persuading people to believe conspiracy theories, and compare the differences between truthfulness constraints and various model guardrails.