Skip to content

STACK: Adversarial attacks against LLM guardrail pipelines

STACK: Adversarial Attacks on LLM Safeguard Pipelines

arXiv· arXiv:2506.24068· Ian R. McKenzie, Oskar J. Hollinsworth, Tom Tseng, Xander Davies, Stephen Casper, Aaron D. Tucker, Robert Kirk, Adam GleaveFAR.AI; UK AISI; OATML, University of OxfordIndexed by this site · 2026-10-15· Originally publishedSelected AttacksSelectedAttention42

Translation in progress — untranslated text is omitted.

Quick read

Attacks

Based on the paper PDF (generated by Gemini); refer to the original

AI summaryResearchers from FAR.AI, UK AISI, and other institutions propose a staged attack method called STACK for breaking through LLM multi-layer safety guardrail pipelines, achieving a 71% attack success rate in a black-box setting on the ClearHarm dataset, and 33% in a transfer setting. The study also built an open-source defense pipeline whose few-shot prompt classifier reduced attack success rate to 0% on ClearHarm, outperforming ShieldGemma across three attacks and two datasets. STACK sequentially finds jailbreaks for the input classifier, generation model, and output classifier, then combines them into a full jailbreak; the authors state it can bypass a strong pipeline where the PAP attack has a 0% success rate, and report component-level attack findings against Opus 4 and GPT-5.

Questions
Large language model deployment relies on defense-in-depth pipelines composed of input classifier, generation model, and output classifier to prevent catastrophic misuse, but systematic safety evaluation has been lacking, and non-adaptive attacks tend to overestimate the defensive robustness of the pipeline.
Methods
Proposes the staged attack framework STACK: decouples pipeline components, first uses PAP rewriting to induce the generation model, then uses the Confirm algorithm to sequentially optimize a universal adversarial prefix for the input classifier and a jailbreak string for the output classifier, and finally assembles them into a single prompt that breaks through multiple defense layers.
Experiments and results
On ClearHarm, black-box STACK achieves 71% ASR against the few-shot Gemma2 defense pipeline (PAP is 0%); in the zero-shot agent transfer setting, it achieves 33% ASR; in real-world testing, it also breaks Claude Opus 4 and GPT-5.
Limitations and future work
It relies on whole-sentence evaluation of the output classifier and does not apply to streaming classifiers; staged greedy optimization has inter-component interference; the universal adversarial prefix for the input classifier is difficult to transfer to the output side.
Why it was selectedThe staged attack method proposed in the paper dismantles each component of a guardrail pipeline layer by layer and can be transferred to test the robustness of one's own multi-component defenses under real adversarial conditions.

The staged attack method proposed in the paper dismantles each component of a guardrail pipeline layer by layer and can be transferred to test the robustness of one's own multi-component defenses under real adversarial conditions.

Related events and news

0 events · 2 news mentions

News that mentions this paper

Related papers

Show 3 more papers
Both use: StrongREJECT
Read originalarxiv.org