Skip to content
Original

Poisoning the Neighborhood: Topology-Aware Topic-Level Poisoning Attacks Against Large-Scale HNSW-RAG Systems

Poisoning the Neighborhood: A Topology-Aware Topic-Level Attack on Large-Scale HNSW-based RAG Systems

arXiv: Adversarial Attacks, Backdoors & Poisoning· arXiv:2610.21464· Yufei Chen, Yao Wang, Haibin ZhangXidian UniversityIndexed by this site · 28 days ago· Originally publishedAttacks

Quick read

Attacks

Based on the paper PDF (AI-generated); refer to the original

Topic-level poisoning against HNSW-RAG achieves an ASR of 31.29% on Wikipedia 2M, Llama 3.1-8B, m=60.

Threat modelblack-box: The attacker only knows that the target RAG uses HNSW, and does not know unseen queries, model parameters, index topology, or HNSW configuration. They can only inject a small number of natural language documents through public or third-party sources, and cannot modify prompts, retrieval, reranking, or the index. The goal is to get poisoned documents into the top-k and generalize the target entity and claim on unseen decision-class topic queries.

Questions
RAG continuously ingests from public and third-party sources, creating a poisoning surface. Existing attacks are mostly evaluated under exact retrieval; the authors state that the search budget, graph reachability, and local competition of large-scale HNSW make topic-level poisoning fail on unseen queries.
Methods
Under black-box settings, partition the proxy query space and craft poisoned documents as Hub–Satellite: Satellites provide multiple entry points, Hubs strengthen local cohesion. On the generation side, use evidence anchoring and entity–claim coupling to promote the same entity, then two-stage filter by retrieval and generation utility.
Experiments and results
Wikipedia, 2M, Llama 3.1-8B, m=60: ASR 31.29%, RSR 74.00%. In the same table, the highest baseline RSR is 6.00%. The authors state that tested defenses cannot mitigate this attack, and Hub–Satellite forms connected structures in HNSW.
Limitations and future work
The paper does not specifically discuss limitations. Experiments cover Llama 3.1-8B and Qwen3-30B, three retrievers, four corpora up to 2M, and hubness, RAGDefender, perplexity, rewriting, reranking, chunking, and insertion order. The name of the LLM generating queries is not reported.
Experimental setupLlama 3.1-8B、Qwen3-30B et al. · Wikipedia、PubMed et al. · RSR、ASR et al.
Evaluated models
Llama 3.1-8BQwen3-30BBGE-base-en-v1.5all-mpnet-base-v2GTE-base
Benchmarks
WikipediaPubMedSEC-EDGAREPA-HEROMS MARCONatural QuestionsHotpotQABEIR
Metrics
RSRASRCond.ASRPoison LCC
AI summaryResearchers propose a topology-aware topic-level poisoning attack against large-scale HNSW-RAG systems, organizing poisoned documents into a Hub-Satellite structure, where satellite documents provide competitive entry points across multiple semantic regions and the Hub enhances local cohesion and induces cascading retrieval. In the generation stage, an entity-assertion boosting attack is used, combining evidence anchoring with entity-assertion coupling, so that target entities are boosted across different query regions via target assertions. On Wikipedia with 2 million documents, the attack achieves a retrieval success rate above 72% and an attack success rate of 31.29%, clearly outperforming baselines; structural analysis shows that the Hub-Satellite topology does indeed form in HNSW, and existing poisoning defenses cannot mitigate the attack.

Researchers propose a topology-aware topic-level poisoning attack against large-scale HNSW-RAG systems, organizing poisoned documents into a Hub-Satellite structure, where satellite documents provide competitive entry points across multiple semantic regions and the Hub enhances local cohesion and induces cascading retrieval. In the generation stage, an entity-assertion boosting attack is used, combining evidence anchoring with entity-assertion coupling, so that target entities are boosted across different query regions via target assertions. On Wikipedia with 2 million documents, the attack achieves a retrieval success rate above 72% and an attack success rate of 31.29%, clearly outperforming baselines; structural analysis shows that the Hub-Satellite topology does indeed form in HNSW, and existing poisoning defenses cannot mitigate the attack.

Related events and news

2 events · 1 news mentions

News that mentions this paper

RL/RLHF 训练条件可诱发模型智能体提权逃逸

Paper analysis

6 questions; open one to read the full answer

  1. What problem does this paper attempt to solve?

    On large-scale HNSW, poisoning unseen topic queries is constrained by graph search; the authors propose the Hub–Satellite attack.

    On large-scale HNSW-RAG, can limited poisoned documents cover unseen topic queries and alter generation.

    • Scenario: The authors state that RAG continuously ingests from public web pages and third-party sources; attackers can inject documents during collection or index updates; once retrieved, documents can steer outputs toward specified content.
    • Existing Gaps: The authors state that most poisoning is designed under exact retrieval, assuming queries compare against the full corpus and global ranking. Topic-FlipRAG and DiscourseFlip still construct documents using topic queries at attack time, relying on known query sets.
    • Preliminary Observations: At 500K, exact retrieval takes 70.5 ms/query, HNSW 0.7 ms (Figure 2a). Unseen-query RSR: PoisonedRAG drops from 90.8% at 10K to 9.6% at 100K; Topic-FlipRAG (m=40) drops from 93.7% at 10K to 9.2% at 500K.
    • Proposed Attack: On the retrieval side, Hub–Satellite improves graph search reachability; on the generation side, entity–claim promotion makes unseen decision-class queries recommend the same entity. The authors state they do not directly modify the graph but induce neighborhoods via document semantic relationships.
    • Threat Model:
      • Goal: The poisoned set P should enter the top-k of unseen topic query set QT as much as possible; on the generation side, make the model treat claim ct as a supported conclusion and recommend entity et accordingly.
      • Knowledge: black-box. Only know the system uses HNSW; do not know QT, retriever or generator parameters, index topology, or HNSW configuration.
      • Capabilities: Can use public language models and proxy retrievers to construct and filter documents; can only place a small number of documents in public or third-party sources that may be indexed later. Cannot modify system prompts, retrieval pipeline, rerankers, generation process, or the target index.
      • Victim System: RAG with a large-scale HNSW index. HNSW only computes similarity for documents reached via graph traversal. Main experiments use top-5 as generation context.

  2. What related research exists?

    Related work is divided into embedding-space, query-specific, and topic-level poisoning; the authors state the difference lies in large-scale HNSW and unseen queries.

    The authors group by query coverage and retrieval assumptions, and point out three types of limitations: attack permissions, retrieval backend, and query coverage (Section 2.2).

    Embedding-space retrieval attacks

    • GASLITE: Ben-Tov and Sharif (2025) optimize adversarial passages for unseen concept-related queries, requiring white-box access to retriever parameters.
    • Continuous-space poisoning: Li et al. (2025) perform unsupervised poisoning in embedding space, mainly disrupting retrieval ranking rather than downstream answers.
    • Black-Hole: Li et al. (2026) use embedding-space centrality to attract unknown queries, but require directly injecting malicious vectors. The authors classify such methods as beyond the permission of black-box natural-language poisoning.

    Query-specific attacks

    • Single query or fixed query set: PoisonedRAG (Zou et al., 2025) induces specific wrong answers; FlippedRAG (Chen et al., 2025) constructs misleading evidence in a black-box setting; Jamming (Shafran et al., 2025) uses blocker documents to make the system refuse specific queries.
    • Extensions on predetermined sets: GRAG-POISON (Liang et al., 2026) exploits shared relations in multi-hop queries; UniC-RAG (Geng et al., 2025) combines clustering with instruction hijacking; Chang et al. (2026) optimize retrieval triggers for known queries for indirect prompt injection; Pathak and Sharma (2026) coordinate poisoning on predetermined queries to evade ingestion-time defenses. The authors state that these methods all require predefined target queries.

    Topic-level attacks

    • Topic-FlipRAG: Gong et al. (2025) edit documents using semantics and keywords shared by topic queries, manipulating known topic queries.
    • DiscourseFlip: Gong et al. (2026) organize queries into a semantic network and use graph-guided budget allocation for discourse-level opinion manipulation. The authors state that both construct documents from pre-given topic queries and are mainly evaluated under exact retrieval.

    Baselines and positioning

    • Baseline methods: PoisonedRAG and Topic-FlipRAG have public implementations. The authors keep their construction and optimization, changing the target to the same entity–claim. Benchmarks are Wikipedia, PubMed, SEC-EDGAR, EPA-HERO, and real queries from MS MARCO, Natural Questions, HotpotQA, BEIR.
    • Positioning: The authors state that, to their knowledge, this is the first RAG poisoning attack targeting large-scale HNSW and generalizing to unseen topic queries. The difference is not assuming poisoned documents participate in global ranking, but using semantic coordination to indirectly induce HNSW neighborhoods.

  3. How does the paper address this problem?

    Construct Hub–Satellite after partitioning agent queries, and use entity–claim to generalize to unseen topic queries.

    Instead of modifying the HNSW graph, coordinate the semantics of injected documents to induce multi-entry, locally cohesive neighborhoods. The retrieval side is Hub–Satellite, the generation side is entity–claim promotion, and a two-stage filter selects the set (Figure 3).

    Query Space

    • Proxy Queries: At attack time, unseen queries QT are not visible. A public LLM generates n queries Qs covering different subdomains, intents, and phrasings; encoder E_eq embeds them and K-means partitions into K regions. The paper does not give K; Figure 3 labels 10 subdomains and 12 intents.
    • Experimental Correspondence: Appendix A uses 1,500 training queries per topic to construct documents, with 5,000 decision-class test queries not used in construction. Decision classes include recommendations, option comparisons, and management strategies.
    • Partition Use: Each region Ck's query subset Qk becomes a local target, preventing documents from crowding into only a few semantic directions.

    Hub–Satellite

    • Division of Labor: Hubs aggregate cross-region shared semantics as common anchors; Satellites align to local subclusters as competitive entry points. Regions with more queries are split into more subclusters, not equal numbers per region.
    • Hub Reward: Extract keyword anchors Ak from each region's queries; reward is coverage of the union of all anchors, AnchorCov. The authors state this prevents Hubs from specializing in a single region.
    • Satellite Reward: Rretsat(p;k,j)=λ1cos(zp,μk,j)+λ2 aH(p)+λ3AnchorCov(p,Ak)+λ4QueryCov(p,Qk,j). The four terms are subcluster center similarity, maximum cosine with built Hubs, region anchor coverage, and subcluster query coverage.
    • Weights: In Table 7, Satellite's λ1–λ4 are 0.30, 0.05, 0.05, 0.15; Hub's cross-region anchor weight is 0.15. The authors state they are chosen based on role priors and local sensitivity, prioritizing whether adjacent weights are stable.

    Entity–Claim Promotion

    • Evidence Anchoring: Each document consists of background, mechanism, applicability conditions, outcome evidence, and the same claim ct. Hubs emphasize cross-region concepts; Satellites adapt arguments by subcluster intent but retain ct.
    • Entity Coupling: Attribute mechanisms and outcomes supporting ct to entity et, making the claim incomplete without et. Generation reward is Rgen(x)=λclaimSclaim(x,ct)+λevidSevid(x,ct)+λlinkSlink(x,et,ct).
    • Generation Weights: Hub's three terms are 0.30, 0.15, 0.40; Satellite's are 0.20, 0.05, 0.20. Appendix F gives construction prompts; sentence and length constraints are not detailed here.

    Filtering and Evaluation

    • Document-Level: Each role generates multiple candidates; joint score is a weighted sum of retrieval and generation rewards. Hub's α, η are 0.15, 0.85; Satellite's are 0.55, 0.45. High-scoring Hubs form set H.
    • Set-Level: When Satellite count equals msat, greedily maximize the union of covered proxy queries, plus association with H, balanced by ρ. The paper does not give ρ. At m=40, Hub has 2 documents, Satellite 38; at m=60, 4 and 56.
    • Success Criteria: RSR is the proportion of queries with at least one poisoned document in top-k; ASR is the proportion of answers adopting ct and promoting et; Cond.ASR computes ASR only on queries where retrieval succeeded. The authors write ASR = RSR × Cond.ASR. Automatic evaluation Syn-ASR: no negation cues, and either restates the claim or has both entity or alias, positive phrasing, and topic outcome words.

  4. What experiments did the paper conduct?

    Wikipedia, 2M, Llama 3.1-8B, m=60 ASR is 31.29%, RSR is 74.00%.

    Experimental Setup

    • Knowledge bases and preliminary study: Main experiments are on Wikipedia, PubMed, SEC-EDGAR, EPA-HERO, at scales 500K, 1M, 2M. Section 3 compares exact retrieval with HNSW on Wikipedia; the preliminary study in Appendix B uses cosine, main experiments use inner product.
    • Retrieval: Default BGE-base-en-v1.5 (also written as BGE-base-en in the main text). Transfer targets are all-mpnet-base-v2, GTE-base. HNSW defaults are MHNSW=16, ef_construction=200, ef_search=128, taking top-5. Embedding dimension is 768.
    • Generation and judging: Llama 3.1-8B, Qwen3-30B. Main ASR uses Syn-ASR, not an LLM judge. Appendix A: accuracy relative to GPT-4o on 4,000 samples is 93.34%, absolute ASR difference 2.24 percentage points; Krippendorff's α on 400 human-annotated samples is 0.8912, average Cohen's κ with Syn-ASR is 0.8637.
    • Queries and budgets: Four decision domains, eight topics. Each topic has 1,500 training queries and 5,000 synthetic test queries. m is 40 or 60. Real queries come from MS MARCO, Natural Questions, HotpotQA, BEIR.
    • Baselines: PoisonedRAG, Topic-FlipRAG, keeping original construction, with target changed to the same entity–claim (Appendix B).
    • Repeats: Each HNSW configuration builds 3 indexes with seeds 42, 43, 44 and averages. Temperature is 0.0. The poisoned document generation model is MiMo-v2.5.

    Main Results

    Units are %. Wikipedia, 2M, according to Table 1.

    Swipe horizontally to read the table

    MethodGeneration modelm=40 RSRm=40 ASRm=60 RSRm=60 ASR
    PoisonedRAGLlama 3.1-8B0.400.020.600.05
    Topic-FlipRAGLlama 3.1-8B1.800.086.000.18
    Our MethodLlama 3.1-8B72.4929.4974.0031.29
    PoisonedRAGQwen3-30B1.900.120.900.08
    Topic-FlipRAGQwen3-30B5.600.665.901.71
    Our MethodQwen3-30B72.3525.8774.0527.62
    • Authors' interpretation: The authors state that baselines mainly optimize semantic alignment, do not handle benign document competition and path-dependent traversal; high similarity does not guarantee being visited and returned by HNSW.
    • Scale: The authors state that when the corpus grows from 500K to 2M, baseline RSR drops from a maximum of 17.0% to 0.4%–6.0%; our method at 2M still has RSR above 72% under both generation models and both budgets.
    • Another scale: At 500K, Llama 3.1-8B, m=60, our RSR is 89.55%, ASR is 42.71% (Table 1). The authors state that Cond.ASR and ASR at 2M are still above 35.75% and 25.87%.

    Domain corpora and real queries

    • What was tested: PubMed, SEC-EDGAR, EPA-HERO, generation model reported only for Llama 3.1-8B, m=40/60, 500K–2M (Table 2).
    • Domain results: At 2M, average RSR is 63.46%–65.68%, Cond.ASR is 39.95%–41.19%, ASR is 25.35%–27.05%. The authors state that from 500K to 2M, average RSR drops by 12.14–12.91 percentage points. EPA-HERO, m=60, 2M ASR is 41.95%; SEC-EDGAR, m=40, 2M is 15.68%.
    • Real queries: Still on the Wikipedia corpus. At 2M, m=40 RSR/ASR is 59.23%/24.08%, m=60 is 61.45%/26.04%. Table 3: synthetic queries Effective Clusters, PairMean, PairP99 are 11.089, 0.372, 0.537; real queries average n=77.25, corresponding to 10.261, 0.367, 0.521. The authors state that in Figure 4 synthetic queries form multiple internally compact and mutually separated regions; the positions of points are not visible in plain text.

    Hub–Satellite topology

    • Whether it forms: At 2M, Poison LCC is 100% for both m=40 and m=60. Satellites directly connected to at least one Hub are 44.7% and 44.6%. The authors state that Hub Mediation Rate increases with ef_search, from 29.2% to 78.2% (m=40), and from 41.8% to 93.8% (m=60) (Figure 5).
    • Cascade: At ef_search=16, after first encountering a poisoned document, an average of 25.9 and 34.3 more documents are visited; at ef_search=128, 37.4 and 56.1. At ef_search=16, removing Hubs gives RSR 56.30% and 59.60%, while the full structure gives 65.90% and 68.50%; the authors state this is an increase of 9.60 and 8.90 percentage points. At 500K, m=40, 35.4% of queries have all top-5 as poisoned documents (Figure 6).
    • Separating from exact retrieval: The authors state that under exact retrieval, Full is only 0.69 and 0.35 percentage points higher than Satellite-only; after subtracting, the HNSW-specific gain at ef_search=16 is 8.91 and 8.56 percentage points, dropping to 0.71 and 0.96 at 256 (Figure 7). After randomly removing 50%, m=40 remaining 20 documents at 2M still have RSR 60.8%, and Poison LCC is still 100% (Figure 8).

    Existing defenses

    • hubness: On 2M Wikipedia, neither Hub nor Satellite exceeds the threshold, poisoned document detection rate is 0%. At target FPR 1%, k=20 k-occurrence actual FPR is 0.706%, HNSW in-degree is 0.951%–0.952% (Figure 10).
    • Clustering and perplexity: When only 1 of top-5 is poisoned, RAGDefender removal rate is 21.29% (m=40) and 23.88% (m=60), benign false removal is 63.27% and 60.63%; with no poisoning, benign false removal is 79.34% and 79.53% (Figure 11). The authors state that filtering out 50% (20 documents) of poisoned documents also removes 25.6% of benign documents (the authors state about 512,000 documents).
    • Rewriting and reranking: After rewriting at m=40, RSR at 500K/1M/2M is 89.16%, 74.02%, 72.88%, ASR is 39.96%, 34.04%, 29.60%. The authors state that average RSR drops by 1.08 percentage points, ASR from 34.54% to 34.53% (Figure 13). After DPR-XM reranking top-50 and keeping top-5, ASR is still 37.58%, 28.08%, 26.69% (Figure 14). The authors state that the tested defenses cannot mitigate this attack.

    Other ablations and analyses

    • Retrievers: At 2M, m=40, all-mpnet-base-v2 RSR/ASR is 59.64%/22.80%, GTE-base is 66.30%/26.39%; at m=60 it is 62.16%/24.48% and 69.18%/28.79% (Table 4). The authors state that Cond.ASR remains above 38% in all settings.
    • top-k=3: At 2M, m=40 ASR is 27.40%, m=60 is 28.72%; the authors state this is 2.09 and 2.57 percentage points lower than top-5, and Cond.ASR is not lower.
    • Removing joint reward: RSR drops by 7.26–13.70 percentage points; the authors state that in five of six groups, overall ASR drops.
    • Removing entity–claim coupling: The authors report that average RSR increases by 11.38 percentage points, Cond.ASR and ASR drop by 12.90 and 6.88 percentage points.
    • Greedy and high-score direct selection: At 2M, m=40, Score Top RSR/ASR is 67.60%/23.05%, Global Greedy is 70.90%/26.94%; at m=60 ASR goes from 26.53% to 29.48% (Table 4).
    • Chunking and insertion order: At 2M, m=40, after 512-token chunking RSR goes from 72.49% to 73.34% (Table 9). The maximum difference in RSR across three insertion orders is 0.15 (m=40) and 0.12 percentage points (m=60) (Figure 15). Without retrieval context, target entity mention rate for Llama 3.1-8B is 0.657%, for Qwen3-30B is 0.519% (Table 11).

  5. What are some points that can be further explored?

    The paper does not state limitations or future directions in the limitations or future work sections.

    Limitations and future directions noted by the authors

    The paper does not specifically discuss limitations. The Conclusion restates the attack effectiveness, topological cooperation, and defense results, without framing any item as a shortcoming, open problem, or future work.

    Experimental coverage

    • Models: The generative models are Llama 3.1-8B and Qwen3-30B. Table 1 reports Wikipedia results for both; Table 2's domain corpora and real queries report only Llama 3.1-8B. The retrievers are BGE-base-en-v1.5, all-mpnet-base-v2, GTE-base (Section 6.2, Table 4).
    • Data: The corpora are Wikipedia, PubMed, SEC-EDGAR, EPA-HERO, with main experiments at scales of 500K, 1M, 2M; preliminary studies additionally cover 10K–1M (Sections 3, 6.2). Eight topics, each with 1,500 training queries and 5,000 synthetic test queries.
    • Real queries and judgments: Real queries come from MS MARCO, Natural Questions, HotpotQA, BEIR, with Table 3 averaging n=77.25. The main metric is judged by Syn-ASR; GPT-4o checks 4,000 items, and human checks 400 (Appendix A).
    • Defenses and index parameters: Evaluated hubness detection, RAGDefender, perplexity filtering, query rewriting, DPR-XM reranking, as well as chunking and insertion order (Section 7.4, Appendices C, D). Figure 9 scans MHNSW at 8, 16, 32 and ef_search at 16, 32, 64, 128, 256. Each configuration uses 3 random indexes.
    • Names not reported in the paper: The poisoned document generation model is stated as MiMo-v2.5. Section 6.2 only says an LLM generates the training and test queries; the paper does not report that model's name.

  6. Summarize the main content of the paper

    Using Hub–Satellite and entity–claim to generalize, attack unseen topic queries in large-scale HNSW-RAG.

    This is a black-box topic-level poisoning attack against large-scale HNSW-RAG: using limited documents to cover unseen decision-class queries and making responses recommend the same entity.

    • Problem: The authors state that existing poisoning assumes exact retrieval with global ranking. HNSW only visits a subset of nodes in the graph; as the corpus grows, benign documents crowd out the search budget and candidate slots. In preliminary studies, PoisonedRAG's unseen-query RSR drops from 90.8% at 10K to 9.6% at 100K.
    • Approach: Use proxy queries and K-means to approximate the unknown topic distribution. Satellites align to different local regions and maintain entry competitiveness; Hubs aggregate cross-region anchors, inducing local connections via semantic proximity. On the generation side, the same claim is placed in the evidence chain, and mechanisms and results are tied to the target entity. Then select the final set based on joint scores and marginal coverage.
    • Main Results: Wikipedia, 2M, m=60, Llama 3.1-8B: RSR 74.00%, ASR 31.29%; Qwen3-30B: ASR 27.62% (Table 1). In the same table, the highest RSR among all 2M baselines is 6.00%.
    • Other Settings: Llama 3.1-8B, three domain corpora, 2M: average ASR 25.35% (m=40) and 27.05% (m=60) (Table 2). Real queries, Wikipedia, 2M, m=60: ASR 26.04%. Poison LCC at 2M is 100% (Figure 5a).
    • Authors' Conclusions: The authors claim Hub–Satellite forms connected structures in the target index and improves poisoned document retrieval under limited budgets; tested defenses either remove many benign documents or still leave residual ASR. The authors claim this is, to their knowledge, the first RAG poisoning attack targeting large-scale HNSW and generalizing to unseen topic queries.

Related papers

Both use: BEIR
Show 3 more papers
Both use: BEIR
Both use: MS MARCO
Read originalarxiv.org