LeakDojo: Decoding the Leakage Threats of RAG Systems

1. 01What problem does this paper attempt to solve?

   Existing RAG leakage attack studies use varying setups and are hard to compare; the authors propose a unified framework to systematically evaluate leakage risk.

   Core problem: RAG systems expose high-value knowledge bases to LLMs, and existing leakage attack studies cannot systematically assess real leakage risk because attack strategies, target systems, and experimental settings are inconsistent.

   • Problem setting: RAG knowledge bases are costly to build and maintain and thus become high-value assets; attackers use prompt injection to induce the LLM to verbatim-recite retrieved chunks, and the authors state that such attacks have been shown effective on simple RAG systems.
   • Shortcomings of existing research: the authors point out that six representative works differ in attack strategy, target RAG structure, and interaction budget (Table 1), and that these differences hinder direct cross-study comparison; moreover, modern RAG introduces modules such as rewriter and reranker, and LLM instruction-following ability has strengthened, and existing studies do not cover the impact of these factors.
   • Core observation: the authors uniformly model leakage attacks as two independent components, a query generator and adversarial instructions, and assume the two contribute independently to leakage; Figure 1 shows that models with higher IFEval scores have higher leakage risk (Pearson r = 0.578, p = 0.039).
   • Paper proposes: a configurable evaluation framework, LeakDojo, which splits the RAG system, attacks, and defenses into independent configuration modules; it is used to benchmark 6 attacks, and two new adversarial instructions, RankerSet and CodeClaim, are proposed as a case study.
   • Threat model (copied from Section 2.1 of the paper):
     • Goal: maximize the number of unique chunks leaked from the knowledge base within N rounds of interaction.
     • Capability: can only send queries and receive final responses through a black-box RAG interface, with a limited number of rounds.
     • Knowledge: does not know the RAG structure or configuration; the high-level domain topic of the database is usually guessable.
     • Victim system: a RAG system containing a retriever, optional augmentation modules, and a backend LLM.

   Collapse this answer

2. 02What related research exists?

   Related work is divided into three groups: six RAG leakage attacks, prompt injection attacks, and other LLM attack benchmarks.

   RAG leakage attacks (the six benchmarked)

   • TGTB (Zeng et al., 2024) and PIDE (Qi et al., 2024): static strategies, where query generation does not depend on system feedback; PIDE has no public query set, so this paper substitutes an LLM-generated version, GEN-PIDE.
   • PoR (Di Maio et al., 2024): an interactive strategy that iteratively improves queries using keyword exploration.
   • RAG-Thief (Jiang et al., 2024): an interactive strategy that iteratively refines queries using context continuation.
   • DGEA (Cohen et al., 2024) and IKEA (Wang et al., 2025): interactive strategies; the authors note that IKEA explicitly considers the presence of a reranker in RAG, whereas other works do not.

   Prompt injection attacks

   • Direct prompt injection: Perez and Ribeiro (2022), Toyer et al. (2024), and Liu et al. (2024) study injected instructions overriding the model's expected behavior; the authors regard RAG leakage attacks as a kind of direct prompt injection (citing Debenedetti et al., 2025), but consider that the external retrieval pipeline brings additional complexity.
   • Related harms: injection can lead to unsafe actions being executed in agent scenarios (Debenedetti et al., 2024; Zhan et al., 2024) and to system prompt leakage (Zhang et al., 2024; Dong et al., 2025).

   LLM attack benchmarks

   • HarmBench (Mazeika et al., 2024): a standardized evaluation framework for jailbreak attacks.
   • Raccoon (Wang et al., 2024): a prompt extraction benchmark; Liu et al. (2024): a benchmark of prompt injection attacks and defenses.

   Baselines and benchmarks: the benchmarked attacks are the six above; the defense controls are Prompt-Guard-2-86M and Llama-Guard-3-8B (Appendix B.7); the evaluation datasets are SciFact, NFCorpus, Enron Email, and FiQA.

   The authors position the difference between this paper and benchmark-type work: RAG leakage attacks are multi-round and stateful, and RAG systems carry a knowledge base, retriever, and optional augmentation modules, requiring targeted evaluation design.

   Collapse this answer

3. 03How does the paper address this problem?

   LeakDojo splits RAG, attacks, and defenses into independent configuration modules, and unifies attacks as a query generator plus adversarial instructions.

   Overall approach: uniformly model the RAG system, leakage attacks, and defense mechanisms as independently configurable modules, and isolate the effect of each factor on leakage while holding the remaining components fixed. The framework is named LeakDojo (Figure 3).

   Configurable RAG system

   • Retrieval stage: the embedding model, retrieval strategy, similarity threshold, and number of returned chunks k are configurable; experiments use bge-large-en-v1.5 and the MMR strategy (top_k=10, fetch_k=40, score_threshold=0.75, top_n=5, Appendix A.1.1).
   • Generation stage: supports locally deployed (vLLM) and remote API LLMs.
   • Augmentation modules: the rewriter expands the query into multiple queries to broaden retrieval coverage; the reranker reorders candidate chunks; the summarizer compresses the context; the three can be toggled independently, and the backends of the rewriter and summarizer are gpt-4.1-mini.

   Configurable attack strategies

   • Two-component decomposition: attack = query generator + adversarial instruction. In round i, an anchor query Ai is generated, and the final adversarial query is Q^adv_i = A_i ⊕ I, where I is a fixed adversarial instruction used to induce the LLM to verbatim-recite chunks.
   • Two types of generators: static strategies (TGTB, PIDE) do not depend on feedback; interactive strategies (PoR, RAG-Thief, IKEA, DGEA) adaptively refine queries based on historical responses and can maintain state within N rounds.
   • Budget: default N = 200; Appendix A.2 uses Figure 7 and Table 6 to argue that this budget is sufficient to distinguish attacks (PoR has diminishing marginal returns after N=1000).

   Configurable defense mechanisms

   • Input-side intent detector: gpt-4.1-mini classifies the query as safe/unsafe/unknown; when unsafe, it strips the adversarial instruction and keeps the benign part; when unknown, it uses a rule-based fallback to strip instruction-override patterns (Prompt 2, Appendix A.1.3).
   • Output-side content detector: when the ROUGE-L F1 between the response and the retrieved context exceeds 0.5, it blocks and replaces the response with a warning message.

   Evaluation metrics (Appendix A.3)

   • CCL: the proportion of unique chunks leaked across all queries out of the ideal maximum k × N; the primary metric.
   • SLT: the proportion of queries that trigger leakage; a retrieval chunk counts as successful when its ROUGE-L recall with the response exceeds 0.5; the threshold follows Di Maio et al. (2024), and Appendix B.3 scans 0.3–0.9.
   • ARC: the proportion of unique retrieved chunks out of k × N, measuring the coverage of probing the knowledge base.
   • CRR: for successful queries, the verbatim recovery quality computed over contiguous matching token blocks of length at least α = 50.

   Case study: new adversarial instructions

   • RankerSet and CodeClaim: embed the chunk-extraction intent into a logical reasoning chain, disguising the leakage instruction as a benign task (Prompts 5, 6), used to test defense bypass.

   Collapse this answer

4. 04What experiments did the paper conduct?

   A benchmark of 6 main models × 4 datasets × 6 attacks, plus RAG module ablations, defense evaluation, and mechanism analysis.

   Experimental setup

   • Models tested: 6 in the main experiments—Gemini-3-flash, GPT-5.1, o4-mini, Qwen-3-8B, Qwen-3-235B, DeepSeek-V3; 8 in extended experiments (Table 14, 15)—Qwen-2.5-7B/14B/32B/72B, Kimi-K2, Gemma-3-27B, Gemini-2.5-flash, Doubao-1.6-flash, 14 in total.
   • Datasets: SciFact (5,183 documents), NFCorpus (3,633), FiQA (57,638), Enron Email (577,401), all using documents as atomic retrieval units (Table 5).
   • Attacks and auxiliary models: 6 attacks (TGTB, GEN-PIDE, DGEA, RAG-Thief, PoR, IKEA); the four LLM-assisted attacks (GEN-PIDE, PoR, IKEA, RAG-Thief) use gpt-4.1-mini by default; budget N = 200, greedy decoding.
   • Metrics and criteria: CCL, SLT, ARC, CRR (definitions in Q3); SLT threshold is ROUGE-L recall 0.5; faithfulness is measured with Ragas' LLM-as-a-Judge, and the paper does not specify the judge model.
   • How Table 2 values are obtained: each value is the average of three RAG variants: vanilla, with reranker, and with rewriter+reranker.

   Main results

   CCL (%, average of three RAG variants) on the Enron Email and FiQA datasets in Table 2:

   Swipe horizontally to read the table

   Attack	Gemini-3-flash (Enron)	DeepSeek-V3 (Enron)	Qwen-3-8B (FiQA)	Qwen-3-235B (FiQA)
   TGTB	72.3	0.1	45.7	13.6
   GEN-PIDE	69.4	38.8	45.2	28.0
   DGEA	11.2	8.1	4.1	4.2
   RAG-Thief	44.4	64.4	54.7	59.2
   PoR	88.3	6.8	53.9	54.7
   IKEA	23.2	1.0	30.6	21.1

   (According to Table 2; the GPT-5.1 and o4-mini columns and the SciFact and NFCorpus datasets are omitted; see the original table for complete values.)

   • Attack performance is unstable: the authors state that the relative ranking of attacks changes with LLM and dataset; for example, PoR is the highest in the table on Gemini-3-flash/Enron, while after switching to DeepSeek-V3, RAG-Thief leads.
   • The cross-model bottleneck is inducing leakage: on the same dataset, ARC is nearly unchanged while SLT differs greatly; for example, on Enron, TGTB's ARC is around 87% for all, Gemini-3-flash's SLT is 88.5%, while Qwen-3-235B rarely follows adversarial instructions (SLT 0.5%).
   • The cross-dataset bottleneck is coverage: with a fixed LLM, the authors state that attack differences are mainly driven by ARC; for example, on DeepSeek-V3, RAG-Thief retrieves more chunks on Enron than on other datasets.

   Instruction-following ability and leakage risk

   • What was measured: correlation between IFEval scores of 14 models (Table 4) and SLT (Figure 1).
   • Result: Pearson r = 0.578, p = 0.039.
   • Authors' interpretation: LLMs with stronger instruction-following ability may pose higher leakage risk.

   RAG module ablation and faithfulness trade-off

   • What was measured: leakage and faithfulness of 6 LLMs under four configuration levels T0 (vanilla) → T1 (+reranker) → T2 (+rewriter) → T3 (+summarizer) (Figure 4, Table 15).
   • Results: the authors state that rewriter improves average ARC and reduces its variance; summarizer reduces CCL but breaks context integrity, and faithfulness decreases accordingly; reranker has low and non-significant correlation with each metric (CCL: r = −0.186, p = 0.117). Faithfulness is positively correlated with CCL: FiQA r = 0.83 (p = 4.88e-7), SciFact r = 0.57 (p = 3.43e-3), NFCorpus r = 0.51 (p = 1.12e-2) (Figure 4b).
   • Authors' interpretation: modules that improve faithful generation may simultaneously increase leakage risk, implying a privacy-utility trade-off.

   Defense evaluation and bypass

   • Default instructions are blocked: under the FiQA/DeepSeek-V3/T2 configuration, enabling only the intent detector (Di) reduces CCL of most attacks to below 1%, e.g., TGTB from 7.3% to 0.6%, PoR from 48.7% to 0.2% (Table 3); the authors attribute this to default instructions containing explicit instruction words, which are easy to detect.
   • New instructions bypass: RankerSet/CodeClaim achieves CCL of 47.8%–59.6% under Di (Table 3, e.g., GEN-PIDE+CodeClaim 59.6%); it also maintains high CCL against Prompt-Guard-2-86M and Llama-Guard-3-8B (Table 12, e.g., PoR+RankerSet 58.5% under Llama-Guard, default instructions 48.7%).
   • Authors' interpretation: existing attacks lack stealthiness; disguising leakage intent as benign tasks can bypass detection.

   Other ablations and analyses

   • SLT×ARC decomposition: Figure 5 shows SLT×ARC is close to CCL; in Figure 8, the differences |SLT×ARC − CCL| are mostly concentrated near zero; Table 10 is a full-matrix validation of generator×instruction on SciFact/Qwen-3-8B.
   • Temperature: under NFCorpus/PoR, CCL at temperatures 0.0–1.0 fluctuates between 39.4% and 56.5%, and the authors state the effect is small (Table 9).
   • SLT threshold: results at four levels 0.3/0.5/0.7/0.9 are similar (Table 8).
   • Token cost: PoR output tokens are about 11.5k–15.9k, CCL 53.9%–70.2%; RAG-Thief output 579.7k–719.7k, CCL 42.9%–64.1% (Table 11).
   • Retrieval strategy: Figure 6 shows MMR has lower leakage than KNN, and the authors accordingly choose MMR.
   • Other utility metrics: correlations of Answer Relevancy and InfoDepth with CCL are not significant on most datasets (p > 0.05, Figure 9).

   Collapse this answer

5. 05What are some points that can be further explored?

   The authors list four limitations: attack cost, component coverage, budget, and multilingual; the experiments cover 14 models and 4 English datasets.

   Limitations and future directions noted by the authors

   • Attack efficiency not accounted for (Section 9): only attack effectiveness is considered; generation cost (token consumption, generation latency) is not explicitly considered; the authors state that relative comparisons under a fixed budget partly reflect differences in resource utilization.
   • Model and RAG design coverage (Section 9): cannot exhaust all RAG designs; only three representative plug-and-play components—rewriter, reranker, summarizer—are studied.
   • Fixed attack budget (Section 9): limited by experimental resources, all experiments fix N = 200; the authors use Figure 7 to argue this is sufficient to distinguish attack strategies.
   • Lack of multilingual evaluation (Section 9): experiments are limited to English corpora; the authors state that exploring how datasets in other languages affect RAG leakage risk is a future direction.

   Experimental coverage

   • Models tested: 6 LLMs in the main experiments (Section 4.1), 8 in extended experiments (Table 14, 15), 14 in total, including open-source and closed-source.
   • Datasets: 4 English datasets, covering scientific, medical, financial, and enterprise email domains (Table 5).
   • Attacks and defenses: 6 existing attacks benchmarked; defense evaluation includes a self-built intent detector + content detector (Table 3, 13) and two external defense models, Prompt-Guard-2-86M and Llama-Guard-3-8B (Table 12).
   • RAG configurations: four combinations of vanilla, +reranker, +rewriter, +summarizer (T0–T3, Section 4.3).
   • Not reported in the paper: the specific model used for the faithfulness review; the number of repetitions per cell in the main results (Table 2 values are the average of three RAG variants).

   Collapse this answer

6. 06Summarize the main content of the paper

   LeakDojo uniformly models RAG leakage attacks and benchmarks 6 attacks; the authors state that leakage is independently determined by retrieval coverage and induced leakage.

   A systematic evaluation paper on RAG leakage risk that proposes a configurable framework, LeakDojo, and uses it to benchmark existing attacks.

   • Problem: RAG knowledge bases are high-value assets; leakage attacks use prompt injection to induce the LLM to recite retrieved chunks; existing studies use differing settings, making comparison difficult, and do not cover the impact of modern RAG modules and stronger LLMs.
   • Method: LeakDojo splits the RAG system, attacks, and defenses into independent configuration modules; attacks are unified as the combination of a query generator and adversarial instructions; evaluated with four metrics CCL, SLT, ARC, and CRR on 14 LLMs and 4 datasets, with budget N = 200.
   • Attacks are unstable: PoR reaches CCL 88.3% on Gemini-3-flash/Enron Email, but only 6.8% on DeepSeek-V3; RAG-Thief leads at 64.4% (Table 2).
   • Mechanism: the authors state that CCL is approximately SLT×ARC, with the two components contributing independently (Figure 5, 8); SLT correlates with IFEval score (r = 0.578, p = 0.039, Figure 1).
   • Trade-off: faithfulness is positively correlated with CCL (FiQA r = 0.83, Figure 4b); the authors state that modules that improve faithfulness may increase leakage risk.
   • Defenses and bypass: the intent detector reduces the CCL of default-instruction attacks to below 1%, but the authors' proposed RankerSet/CodeClaim disguises leakage as a benign task, reaching CCL up to 59.6% under defense (Table 3).
   • Authors' conclusion: the results provide a basis for understanding attack effectiveness and designing stronger attacks and defenses; the framework code has been released with the paper.

   Collapse this answer

Related papers