Skip to content
Original

Meta-SecAlign: Training Large Language Models to Resist Prompt Injection to Build Robust Agents

Meta-SecAlign: Training LLMs against Prompt Injection for Robust Agents

arXiv· arXiv:2507.02735· Sizhe Chen, Arman Zharmagambetov, David Wagner, Chuan GuoFAIR at Meta; UC BerkeleyIndexed by this site · 15 days ago· Originally publishedSelected DefensesSelectedAttention39

Translation in progress — untranslated text is omitted.

Quick read

Defenses

Based on the paper PDF (AI-generated); refer to the original

AI summaryResearchers from Meta FAIR and UC Berkeley propose a defense fine-tuning method, Meta-SecAlign, that trains large language models to resist prompt injection while preserving model utility. The paper's large-scale evaluation finds that the mainstream open-source defense SecAlign incurs notable utility loss, especially on agent tasks: after SecAlign defense, Llama-3.3-70B-Instruct's AgentDojo tool-calling utility drops from 59.8% to 6.2%. Meta-SecAlign avoids shortcut learning by randomizing the position of injections during training, and uses model self-generated responses as in-distribution training labels; on Llama-3.3-70B-Instruct, its maximum drop across 8 utility benchmarks is 2.7 percentage points, the WASP web navigation end-to-end attack success rate drops to 0%, the AgentDojo attack success rate drops from 14.7% to 1.9%, and it also outperforms SecAlign against GCG adaptive attacks.

Questions
Prompt injection is listed as the top threat to AI agents. The authors find that the existing open-source defense SecAlign substantially reduces benign utility on large models and agent tasks, contrary to the purpose of defense.
Methods
Two changes on top of SecAlign's DPO preference fine-tuning: randomize the injection position during training (about half placed at the beginning of the data) to avoid learning a position shortcut; use the initialized model to self-generate preference labels in place of outdated dataset annotations.
Experiments and results
Evaluated on 5 models and 6 prompt injection benchmarks. The authors state that Meta-SecAlign-70B has a maximum drop of 2.7 percentage points across the 8 utility benchmarks, AgentDojo ASR falls from 14.7% to 1.9%, WASP end-to-end ASR falls to 0%, but under GCG adaptive attacks ASR is still 47.1%.
Limitations and future work
The authors point out: malicious user attacks are not defended against; reliance on input trustworthiness being known; still vulnerable to strong adaptive attacks; cumulative attacks across multiple untrusted messages are not evaluated. Experiments cover only the text modality, and Qwen3.6-27B was tested only in non-thinking mode.
Why it was selectedThe paper finds that the mainstream open-source defense SecAlign incurs significant utility loss on agent tasks, and proposes two utility-preserving improvements to defense fine-tuning.

The paper finds that the mainstream open-source defense SecAlign incurs significant utility loss on agent tasks, and proposes two utility-preserving improvements to defense fine-tuning.

Related events and news

2 events · 3 news mentions

News that mentions this paper

LocalAlign：近目标对抗训练提升提示注入防御泛化

SRFT自我反思微调提升Agent抗提示注入能力

Related papers

Both use: TaskTracker
Both use: AlpacaEval2
Show 3 more papers
Both use: WASP
Read originalarxiv.org