DefensesarXiv 2606.18673
Understanding and mitigating prompt leakage attacks in real-world LLM applications
Propose soft prompt defense AREA, re-anchoring attention to mitigate system prompt leakage
- arXiv
- 2606.18673
- Published
- Layer
- Application layer
- Setting
- LLM applications
- Attacker
- Black-box
- Evaluated models
- Llama-2-7B-chat-hf, Llama-3.1-8B-Instruct, Mistral-7B-Instruct-v0.3 and others8
- DefensesInference-time intervention
- AttacksExtraction and theft
AbstractThe authors measure commercial prompt leakage, propose attention drift, and use AREA to strike a trade-off between usability and leakage resistance.
The authors study system prompt leakage in real LLM applications and propose AREA, a mitigation method that does not modify weights.