Skip to content
CASRAI·Indexed by this site · Originally published· Evaluation →· Its place in the scan record →SelectedAttention40

Study finds RAG citations often reflect model parametric memory rather than retrieval sources

RAG Citations Often Reflect Parametric Memory — CASRAI

AI summary

A study published on November 12, 2026 shows that RAG systems often attach citations to retrieved documents that did not participate in generating the corresponding statements (arXiv:2611.17161). Researchers at COMSATS University Islamabad and Middlesex University London used a counterfactual value substitution method to measure causal provenance, replacing numbers in retrieved passages and titles with fake values; if the model outputs the fake value, it indicates reliance on retrieval, while restating the original value indicates the statement came from parametric memory. On the ASQA benchmark with GTR retrieval, they evaluated Qwen2.5-Instruct (0.5B–14B) and Llama-3.1-8B-Instruct; 1.2%–15.7% of citable statements actually originated from parametric memory, with the Qwen series rising with parameter scale from 1.2% at 0.5B to 13.3% at 14B, and Llama-3.1-8B reaching 15.7%, and the models still attached citations to 89%–100% of parametric statements.

Why it was selected

For teams relying on RAG for literature synthesis or fact-checking, a citation passing a semantic entailment check does not mean the statement actually came from the cited retrieved content.

Related papers
Read originalcasrai.org