FAR.AI and UK AISI researchers built and attacked their own multi-layer defense pipeline, finding that conventional attacks had a 0% success rate against this defense, while their staged attack method STACK achieved a 71% attack success rate on the ClearHarm catastrophic risk dataset. Multi-layer defenses typically consist of three components: an input filter, model refusal training, and an output filter. The researchers built pipelines on open-source guard models such as Google ShieldGemma and Meta Llama Guard, with the best performance coming from Gemma 2 prompted with a few examples. STACK targets each layer in sequence: first finding a universal jailbreak text that makes harmful requests appear harmless to the input filter, then using existing techniques to induce the model to give a harmful answer, and finally finding text that makes the harmful answer appear harmless to the output filter. The 71% success rate depends on the attacker being able to observe which layer intercepted the request; in a fully black-box transfer attack scenario, the success rate drops to 33%.
