Study finds RAG citations often reflect model parametric memory rather than retrieval sources
RAG Citations Often Reflect Parametric Memory — CASRAI
A study published on November 12, 2026 shows that RAG systems often attach citations to retrieved documents that did not participate in generating the corresponding statements (arXiv:2611.17161). Researchers at COMSATS University Islamabad and Middlesex University London used a counterfactual value substitution method to measure causal provenance, replacing numbers in retrieved passages and titles with fake values; if the model outputs the fake value, it indicates reliance on retrieval, while restating the original value indicates the statement came from parametric memory. On the ASQA benchmark with GTR retrieval, they evaluated Qwen2.5-Instruct (0.5B–14B) and Llama-3.1-8B-Instruct; 1.2%–15.7% of citable statements actually originated from parametric memory, with the Qwen series rising with parameter scale from 1.2% at 0.5B to 13.3% at 14B, and Llama-3.1-8B reaching 15.7%, and the models still attached citations to 89%–100% of parametric statements.
For teams relying on RAG for literature synthesis or fact-checking, a citation passing a semantic entailment check does not mean the statement actually came from the cited retrieved content.