Cited but not used: parametric memory sources in retrieval-augmented generation
Cited but Unused: Parametric Provenance in Retrieval-Augmented Generation
The authors replace measured RAG citation sources with counterfactual values, reporting that 1.2%–15.7% of citable claims come from parametric memory, which NLI verification cannot identify.
- RAG systems attach citations to generated claims, but the citations often mislabel the source: the claim actually comes from the model's parametric memory rather than the retrieved passage. The authors state that standard verification based on NLI support structurally cannot detect this post-rationalisation.
- Propose counterfactual value substitution: replace the numeric values shared by the answer and the retrieved passages with fake values in the passages (including titles) and ask again; if the model reports the fake value, it is judged as coming from context, and if it reports the original value, it is judged as coming from memory; the labels are produced by construction. Then compare ten attribution methods in detecting these labels.
- The authors report a parametric source rate of 1.2%–15.7% on ASQA, rising with scale within a family; NLI verification AUROC is no different from random; token-level context contrastive detection is best (average AUROC 0.788), but flagging is not worthwhile at low base rates; after correcting the artefact of not substituting the title, the rate roughly halves.
- Limitations listed by the authors include: only 0.5B–14B open-source models tested, 19%–24% of records undeterminable, the protocol covering only claims containing a replaceable numeric value, blind-label verification covering only 120 items from Qwen2.5-14B, and the detection signal measuring what the model knows rather than what it used.
- Evaluated models
- Qwen2.5-0.5B-InstructQwen2.5-1.5B-InstructQwen2.5-3B-InstructQwen2.5-7B-InstructQwen2.5-14B-InstructLlama-3.1-8B-Instruct
- Benchmarks
- ASQAELI5QAMPARI
- Metrics
- parametric provenance rateAUROCCohen's κprecision
The paper measures the source authenticity of citations in retrieval-augmented generation (RAG), using a counterfactual value substitution method to replace facts in retrieved passages with false variants and then re-asking the question; it finds that 1.2% to 15.7% of citable claims actually come from parametric memory rather than retrieved content, covering five sizes of one model family and one comparison family. At the most conservative substitution distance, this rate was 4.6% and 8.5% on the two tested models, and within the same family the rate rises with model scale. Natural language inference (NLI) verification results, as a standard method, were no different from random, even when restricted to cited passages. The authors attribute this to structural factors: a model answering from memory will select citations consistent with its output; after comparing ten attribution methods on the same claims, the difference in the model's output distribution with and without context best distinguished the source, but the authors note that a detector built on this is still not accurate enough, measuring what the model knows rather than what it actually used, and distinguishing the two remains an open problem.
Related events and news
1 events · 1 news mentions
News that mentions this paper
研究:RAG 引用常源自参数记忆而非检索
- CASRAIcites the paper's id or link研究发现 RAG 引用常反映模型参数记忆而非检索来源
Abbas, T., Zia, T., and Windridge, D. (2026). “Cited but Unused: Parametric Provenance in Retrieval-Augmented Generation.” arXiv:2611.17161v1. https://arxiv.org/abs/2611.17161 (Checked 2026-11-15).
Paper analysis
What problem does this paper attempt to solve?
RAG citations often mislabel the source (the claim comes from parametric memory rather than retrieval), and the authors state that standard verification structurally cannot detect this.
Core problem: the citations that RAG systems attach to claims often mislabel provenance—the claim actually comes from the model's parametric memory rather than the retrieved passage, and standard citation verification cannot detect this.
- Setting and importance: RAG presents generated claims together with citations to retrieved passages, and readers check the evidence via the citations. The authors point out that if a passage merely agrees with the claim rather than producing it, the reader is directed to evidence with no confirming force.
- Shortcomings of existing methods: the authors state that standard verification asks "does the cited passage support the claim" (NLI entailment), but support is necessary, not sufficient; a model answering from memory will pick a citation consistent with its own output (which Wallat et al. 2025 call post-rationalisation), so the support check necessarily passes.
- Core observation: measuring provenance requires ground truth that does not rely on inferring internal states. The authors propose counterfactual value substitution: replace a value shared by the answer and the passage with a fake value and ask again; reporting the fake value indicates the context was used, reporting the original value indicates memory was used, and the label is produced by construction rather than inferred.
- Paper outputs: a provenance measurement protocol covering five sizes of one model family and a second comparison family; a comparison of ten attribution methods on the same set of claims; a token-level detection signal and an end-to-end usability evaluation; and a measurement artefact in the entity-substitution protocol that roughly doubles the measured rate, along with its correction.
What related research exists?
The paper divides related work into five lines: RAG attribution evaluation, citation improvement, faithfulness discrimination, context attribution, and parametric/contextual knowledge.
RAG and attribution evaluation
- Lewis et al. (2020) proposed retrieval-augmented generation; Rashkin et al. (2023) formalized attribution as attributable-to-identified-sources; AutoAIS by Bohnet et al. (2022) uses NLI as a proxy; the ALCE citation protocol of Gao et al. (2023b) has become a de facto standard.
- Liu et al. (2023) found that about half of the sentences from generative search engines are fully supported; Yue et al. (2023), Min et al. (2023), and Niu et al. (2024) improved automatic scoring and span-level annotation. The paper points out that these works all measure the degree to which passages support claims.
Citation quality improvement
- RARR (Gao et al., 2023a) revises sentences to match evidence; CiteFix (Maheshwari et al., 2025) and Li et al. (2024) reassign citations; Self-RAG (Asai et al., 2024) and others train models to be more supported. The paper points out that the criteria they optimize provide no information on the source question (argued in Section 6.1).
Correctness vs faithfulness
- Jacovi and Goldberg (2020) distinguish faithfulness from plausibility; Turpin et al. (2023) provide evidence of models' post-hoc rationalization; Wallat et al. (2025) bring this problem to the citation setting and name it post-rationalisation; van Dort and Heuss (2026) provide a mechanistic explanation on an 8B model; Worledge et al. (2024) provide the terms corroborative/contributive. The authors position this paper's contribution as quantitative and methodological: multiple models, labels produced by construction.
Context attribution methods
- ContextCite (Cohen-Wang et al., 2024), MIRAGE (Qi et al., 2024), PECoRe (Sarti et al., 2024), ARC-JSD (Li et al., 2026), SelfCite (Chuang et al., 2025), and AttriBoT (Liu et al., 2025) are all deletion-based attribution; the paper argues in Section 6.3 that deletion-based marginals can only weakly separate sources.
Parametric vs contextual knowledge
- Longpre et al. (2021) introduced entity substitution in question answering, which is the basis of this paper's protocol and the artifact analysis in Section 8; Mallen et al. (2023), Xie et al. (2024), Neeman et al. (2023), and others study the arbitration between memory and counter-evidence; ConSens (Vankov et al., 2025) and TrustMargin (Xu et al., 2026) compare perplexity or likelihood margins with/without context, and are the closest methods to this paper's detection comparison.
Baselines and benchmarks: the support baseline is a DeBERTa NLI cross-encoder (taking the maximum entailment over all retrieved passages); the detection comparison includes ten attribution methods; the data are ASQA (main), ELI5, and QAMPARI.
How does the paper address this problem?
Construct source labels using counterfactual value substitution, then compare ten attribution methods and one token-level detection signal on the same set of claims.
Overall approach: first use counterfactual value substitution to define and construct source labels at the behavioral level, then use this set of labels to evaluate whether support verification and various attribution signals can identify the source without regeneration.
Problem setting and definitions
- Let q be the question, C = (d1, ..., dk) be the retrieved passages, c be a generated claim with a citation to dj, and Z ∈ {contextual, parametric} be the latent source of c.
- Following Wallat et al. (2025): if dj entails c, the citation is correct; if c is causally influenced by dj, it is faithful; a post-rationalised citation is correct but not faithful, which is exactly the case that support verification cannot detect.
- Z is not directly observable; the authors operationalize it with an intervention: if the passage's value is changed to another value and the model still reports the original value, the claim is parametric.
Counterfactual value substitution protocol
- Locate the value v that appears in both the answer and the retrieved passages, replace it with a fake value v' in all passages (including titles), and regenerate. Reporting v' is judged contextual, reporting v is judged parametric, and reporting both or neither is discarded as undetermined.
- Samples where v appears in the question are excluded (the question cannot be edited, so the source cannot be determined). Both design choices come from the artifact analysis in Section 8.
- The default substitution distance is 18–40 years (dates); near is 2–5 years, far is 120–200 years (Table 2).
Experimental setup
- Models: five scales of Qwen2.5-Instruct (0.5B/1.5B/3B/7B/14B) and Llama-3.1-8B-Instruct, bfloat16, greedy decoding.
- Data: ASQA + GTR top-5 retrieval as the main reference set, 800 questions per condition; ELI5 (BM25) and QAMPARI use Qwen2.5-7B to verify whether the phenomenon is limited to factual questions and dense retrieval.
- Prompts are ALCE default, two demonstrations, completion style (under the chat template, models mostly ignore the citation format).
- Statistics: rates use Wilson intervals; AUROC and paired comparisons use 2000 claim-level bootstraps; annotation agreement uses Cohen's κ.
Detection signals and evaluation method
- Support baseline: DeBERTa NLI cross-encoder, taking the maximum entailment of each claim over all retrieved passages (the strongest form, not penalizing it for citation errors).
- Ten attribution methods are compared by AUROC on the same set of claims and labels, including Localised (ours), JSD no context, ConSens, Sufficiency, LOO max/sum, Necessity, NLI entailment, and TrustMargin.
- The authors' signal: at disputed tokens, the change in distribution with/without context (change in log p and entropy), i.e., the model distribution before the context arrives.
- End-to-end usability: provenance honesty is defined as the proportion of claims whose source statement is true; flagging is only worthwhile when precision > 1/(1+c) (c is the cost ratio of false negatives to false positives).
What experiments did the paper conduct?
On ASQA the parametric source rate is 1.2%–15.7%, NLI verification is near random, token-level contrastive detection is best but flagging is not worthwhile.
Experimental setup
- Models tested: Qwen2.5-0.5B/1.5B/3B/7B/14B and Llama-3.1-8B, six in total, all bfloat16 with greedy decoding.
- Data: ASQA (GTR top-5, 800 questions per condition) as the main dataset; of 800 answers each for ELI5 (BM25) and QAMPARI, only 132 and 174 contain an editable value, using Qwen2.5-7B.
- Baseline: DeBERTa NLI cross-encoder (taking the maximum entailment); Appendix C additionally tests DeBERTa-v3-large, RoBERTa-base, and DeBERTa MNLI-F.
- Metrics: parametric source rate (Wilson interval), AUROC (2000 bootstraps), Cohen's κ, precision.
- Human verification: three non-author annotators each labeled 120 items (all from Qwen2.5-14B, balanced across the two classes), with agreement rates with the automatic labels of 81.7%/78.0%/76.9%, and 82.8% for the majority vote (κ = 0.658).
- Repetitions: the paper explicitly states that runs are not repeated under different seeds; intervals reflect claim sampling rather than counterfactual value selection.
Main results
According to Table 1 (ASQA, default substitution distance):
Swipe horizontally to read the table
Model Parametric source rate 95% CI par./dec. Qwen2.5-0.5B 1.2% [0.6, 2.6] 6/498 Qwen2.5-1.5B 6.5% [4.5, 9.3] 27/417 Qwen2.5-3B 8.3% [6.1, 11.2] 38/456 Qwen2.5-7B 8.2% [6.0, 11.2] 35/426 Qwen2.5-14B 13.3% [10.4, 16.9] 56/420 Llama-3.1-8B 15.7% [12.6, 19.4] 70/446 - The authors state that within the Qwen family the rate rises near-monotonically with scale (Kendall's τ = 0.80), with the evidence driven mainly by the endpoints: five of the ten pairwise comparisons have non-overlapping intervals.
- The authors state that the family effect is at least as large as the scale effect: Llama-3.1-8B is higher than Qwen2.5-14B, which has nearly twice the parameters.
- Table 4: in the four models of 1.5B and above, 89%–100% of parametric claims still carry citations, differing from each model's contextual citation rate by only a few percentage points; the authors state that models answering from memory still give citations.
Support verification and attribution method comparison
- NLI verification is near random: in Table 5, the bootstrap intervals for all six models in the nli_entailment row contain 0.5; in Appendix C, the mean over 24 measurements across four entailment models is 0.532. Restricted to cited passages (Qwen2.5-7B, 408 claims with citations), AUROC is 0.532 [0.415, 0.646]; the average entailment for parametric claims is in fact higher (0.329 vs. 0.214). The authors state the reason is structural: a model answering from memory picks a citation consistent with its own output.
- Ten attribution methods grouped (Table 5, Figure 2): token-level context contrast is best (Localised (ours) average 0.788, JSD no context 0.789), sentence-level contrast next (ConSens 0.699, sentence-level likelihood 0.669), passage-level deletion next (LOO max 0.613, Necessity 0.594), and support/preference methods worst (TrustMargin 0.462). The authors state that methods cluster by the quantity they measure rather than by lineage, and that resolution matters more than the method itself.
- Deletion-based attribution is weak: leave-one-out six-model AUROC is 0.541–0.667; the original ContextCite system on Qwen2.5-7B is 0.507 [0.413, 0.605]. The authors argue that deletion answers "how much does the passage contribute to the claim's likelihood", which differs from "whether the claim originates from it".
Localisation, usability, and artefact
- Localisation improves detection (Table 6): measuring at the disputed token raises AUROC for every model, with significant gains for three models, Qwen2.5-1.5B/3B/7B; the signal computed with context has intervals containing 0.5 on five of six models, and the authors state that provenance information exists in the distribution before the context arrives. Without an oracle location, scanning all numeric spans drops AUROC on Qwen2.5-7B from 0.760 to 0.668 (p = 0.002).
- The signal measures knowledge, not use (Section 6.6): on Qwen2.5-7B, among the 220 claims where the model can give the value without context, 13.2% are parametric, versus only 2.9% of the 205 where it cannot; restricted to claims the model already knows, AUROC drops to 0.651.
- End-to-end flagging is not worthwhile (Table 7, Figure 3): under symmetric costs, none of the six models shows an honesty gain at any budget; peak precision is 0.45 (Qwen2.5-14B), below the 0.5 break-even line; the only positive gain on ELI5, +0.022, is based on 3 correct and 1 incorrect flag, which the authors regard as noise.
- Measurement artefact (Section 8, Table 8): the default substitution changed the passage body but not the title, so the true value could still be read from the title; after correction the measured rate roughly halves (e.g., Qwen2.5-7B from 18.7% to 8.2%, Llama-3.1-8B from 22.3% to 15.7%).
Other ablations and analyses
- Substitution distance (Table 2): under near/default/far, Qwen2.5-7B is 4.6%/8.2%/16.3% and Qwen2.5-14B is 8.5%/13.3%/17.6%.
- Retrieval depth (Appendix E): Qwen2.5-7B under 1/3/5/10 passages is 9.1%/9.3%/8.2%/7.7%, with no effect (p = 0.47).
- Number of demonstrations (Appendix E): with 0/2/4, the rate is 5.4%/8.2%/12.6% (p = 0.0003).
- Conflict flagging (Table 3): Qwen2.5-14B has 140 cases (25.7%) that report the retrieved value while also asserting the value it holds and questioning the source, and models below 7B have almost none; the authors state this is a step-like threshold rather than a gradual change.
- Other datasets: ELI5 is 4.5% (4/89 determined), QAMPARI is 2.4% (3/124); too few positive examples for significance inference.
What are some points that can be further explored?
The authors list seven limitations, including model scale, undetermined samples, and coverage, and state that distinguishing "knowing" from "using" is a core open problem.
Limitations and future directions noted by the authors
- Model scale (Limitations): only 0.5B–14B open-source models are evaluated; the method requires token-level log probabilities with and without context, which closed-source frontier models do not expose; trends within the Qwen family suggest larger models are more affected, but this was not extrapolated or tested.
- Undetermined samples (Limitations): 19%–24% of records were discarded because the answer asserted neither the true value nor the substituted value; if these samples differ systematically from determined samples, the rates are biased, and this cannot be tested within the protocol.
- Coverage (Limitations): the protocol requires the answer and passage to share a numeric value, covering about 70% of ASQA answers, but only 16.5% and 21.8% for ELI5 and QAMPARI; whether claims without such values behave the same is unknown.
- Operational definition (Limitations): source is defined by behavioral response to counterfactual edits; a model may have used the passage yet resisted the edit and would be judged parametric (the authors note that annotator disagreement runs in the opposite direction, and by the annotators' reading this is an underestimate).
- Verification scope (Limitations): blind labeling covers only 120 items from Qwen2.5-14B; whether labels are equally reliable on the other five models was not tested; the majority-vote κ = 0.658 is substantial rather than strong.
- Detection ceiling (Limitations): AUROC drops to 0.651 on claims the model already knows; the authors state they did not test whether a signal distinguishing knowledge from use exists under black-box access, and regard this as the core open question left by this paper.
Experimental coverage
- The models tested are five Qwen2.5 sizes and Llama-3.1-8B, six in total, all open-source instruction models; closed-source models were not tested (the method requires log probabilities).
- Detection claims are mainly based on ASQA; ELI5 and QAMPARI have only 4 and 3 parametric positive examples respectively, and the authors report them descriptively only.
- In the attribution comparison, apart from ContextCite run as-is on Qwen2.5-7B, the other methods are the authors' reimplementations of the quantities each method computes; methods that read attention or hidden states (MIRAGE, Wu et al. 2025) are cited but not run.
- The paper does not repeat runs under different random seeds; intervals reflect claim sampling only.
- The paper does not report agreement other than between the judge model and human annotators (e.g., inter-annotator 74.8%–81.4%, κ 0.486–0.604 are reported).
Summarize the main content of the paper
Counterfactual value substitution measures a RAG citation source error rate of 1.2%–15.7%, support verification fails structurally, and the detection signal is not yet deployable.
A measurement and detection analysis of RAG citation provenance: citations may not only fail to support the claim, but may also point to a consistent passage even when the claim comes from parametric memory.
- Problem: the authors state that standard verification only asks whether the passage supports the claim, and post-rationalised citations pass this check by nature, so provenance errors are invisible.
- Method: counterfactual value substitution—replace a value shared by the answer and the passage in the passage (including the title) with a fake value and ask again; reporting the fake value is contextual, reporting the original value is parametric, and the label is produced by construction; measured on ASQA with five Qwen2.5 sizes and Llama-3.1-8B.
- Main results: parametric source rate from 1.2% (0.5B) to 15.7% (Llama-3.1-8B), rising with scale within a family and differing about twofold between families (Table 1); 89%–100% of parametric claims still carry citations (Table 4); NLI support verification is no different from random on all models (Table 5).
- Detection: token-level with/without-context contrast averages AUROC 0.788, better than sentence-level and deletion-based methods; but the signal measures what the model knows rather than what it used, dropping to 0.651 on known claims; under symmetric costs flagging yields no honesty gain on any of the six models, with peak precision 0.45 below the 0.5 break-even line (Table 7).
- Authors' conclusions: detection quality and deployability are two different things; distinguishing what a model "knows" from what it "uses" remains an open question; if the entity-substitution protocol does not edit all surfaces carrying the value (such as the title), it overestimates the measured rate by about twofold.