CleanBase: Detecting malicious documents in RAG knowledge bases

The paper proposes the CleanBase method, which defends against prompt injection attacks by detecting documents in RAG knowledge bases that have been implanted with malicious prompts; such attacks mislead RAG into generating attacker-specified answers when a user asks a targeted question. Its core observation is that, to increase attack success rates, attackers make malicious documents targeting the same target question highly semantically similar. Based on this, CleanBase builds a similarity graph over the knowledge base, with documents as nodes, connecting an edge when the semantic similarity of two documents computed by an embedding model exceeds a statistically determined threshold; malicious documents tend to form clusters in the graph, and detecting these clusters flags malicious documents. The authors derive theoretical upper bounds on false positive and false negative rates, and experimentally verify detection effectiveness on multiple datasets and multiple prompt injection attacks; source code at https://github.com/WeifeiJin/CleanBase。