EmoRAG research reveals the vulnerability of RAG retrieval to emoji perturbations

Is retrieval truly determined by semantic relevance, or can it be dominated by symbol perturbations like emojis.

• Scenario: The authors believe RAG is used for general question answering and code tasks, and retrieval quality is typically assumed to depend on the semantic relevance between the query and library text. If this assumption does not hold, the generator will receive context prepared by the adversary.
• Existing attacks: The authors claim existing RAG attacks require designing retrievable text for each target query, and malicious text may still be hit by non-target normal queries, affecting normal functionality.
• Observation: The authors claim semantic relevance and retrieval results can be decoupled. Figure 1 illustrates: an emoji in the query causes retrieval to return semantically irrelevant text with the same emoji.
• Method: The authors propose EmoRAG, varying the number, position, and type of emojis on question-answering and code data, analyze the mechanism, and compare standard defenses with targeted defenses. In the contribution list, the authors claim this is the first empirical study across three datasets, multiple retrievers, advanced RAG, and multiple LLMs.
• Threat model:
  • Goal: Manipulate RAG output. One is to wait for a benign query with an emoji to trigger library text; the other is for the attacker to directly submit a query with an emoji. The authors discuss question answering, code evaluation, and comment scoring.
  • Knowledge: No access to the parameters of retriever R and generator G. Appendix D.3 states that baseline comparisons use a black-box setting.
  • Capability: Cannot change the training of R or G; can submit queries q and inject text into the knowledge base D, with injection amount less than 0.01‰ of D.
  • System: A RAG consisting of a knowledge base, retriever, and generator. The attack does not update the parameters of these two models.

Composition and applications of RAG

• System structure: §2.1 describes RAG as a knowledge base, retriever, and generator. The retrieval stage retrieves documents by relevance, and the generation stage has a pretrained language model answer by combining the query with the documents. The text mentions dense passage retrieval, used to align queries and documents in a shared vector space.
• Deployed systems: General-domain examples include WikiChat, Shopify's Sidekick, Amazon recommendations, and RedNote; code-domain examples include Vertex AI Codey, Qodo, Codeforces, LeetCode, GitHub Copilot, and Cursor.

Existing attacks

• Training-time poisoning and backdoors: The paper jointly discusses Biggio et al. (2012), Fang et al. (2020), Jia et al. BadEncoder (2022), Liu et al. Trojaning (2018), and Shafahi et al. Poison Frogs (2018). The authors state that these attacks require directly manipulating training and are often infeasible when extended to RAG.
• Retrieval corpus poisoning: Zhong et al. (2023) inject retrievable meaningless passages; Zou et al. PoisonedRAG (2024) inject semantically misleading information; Zhang et al. HijackRAG (2024) formulate the construction of malicious text as an optimization problem. The authors state that these works need to design text for target queries, and specifically call out PoisonedRAG: its text can still be retrieved by non-target normal queries.

Baselines and data

• Baseline methods: Appendix D.3 adapts Prompt Injection (Greshake et al., 2023; Liu et al., 2023), Corpus Poisoning (Zhong et al., 2023; Zou et al., 2024), and what the paper calls the GCG Attack (reference [70]). Corpus Poisoning in the paper's black-box setting splits malicious text into target queries and malicious content. The retriever in Table 6 is SPECTER.
• Benchmarks: Main experiments use Natural Questions, MS-MARCO, and Github-code-clean. §4.1 additionally uses BEIR's Covid, NFCorpus, DBPedia, and Touche, with nDCG@10 as the metric.

The authors separate EmoRAG from the above retrieval poisoning: training-free, injecting only a small number of emojis. The authors state that a single emoji can push the F1-Score above 0.9 (Figure 4), and that normal retrieval is unchanged when emojis are absent.

The authors call this phenomenon EmoRAG: when the query and a small amount of library text are placed in the same emoji, retrieval returns documents with that emoji, and the generator then answers based on those documents. The parameters of the retriever and generator are not updated.

Retrieval and Generation

• Given a query q, the retriever R ranks documents in the knowledge base K by relevance. The paper writes this step as D = R(q, 𝒦), and generation as r̂ = G(q, D), where D is additional text.
• The attacker does not change the parameters of R or G, nor their training. Perturbations are only added to the query and the injected library text.
• Default values in Appendix A.5: the same emoji is injected four times at the beginning and four times at the end of the query and its corresponding perturbed text; the number of injected entries N=5; the top k=5 most similar results are returned. The generator temperature is 0.0 for all. The authors cite others' results that, under conservative analysis, 6.5% of Wikipedia documents can be maliciously edited; this paper's injection amount is set to less than 0.01‰ of the library content.

Text and Emoji

• The authors claim the trigger comes from the emoji itself; the perturbed text need not align with the original query topic, so it can be reused across domains. Each perturbed text uses the same emoji as its corresponding query. Appendix A.4 gives examples in the question-answering domain; the original sentences are not repeated here.
• Variable factors include the number, position, and type of emojis, and whether the query and library text use the same emoji; similarity computation compares dot product and cosine.
• When selecting emojis, the embedding offset is computed, i.e., the dissimilarity between the original query embedding and the injected embedding; the one with the largest offset is considered the most disruptive. Equation (1) scores using total token count and unique token count. The authors state this is only a preliminary metric.
• Appendix E compares Noticeability and Alertness of emojis, emoticons, and garbled text with 32 volunteers. The authors claim emojis are more natural on both dimensions, so they are used as the primary symbol.

How Success Is Determined

• The F1 of the perturbed query is composed of Precision and Recall, checking whether the pre-injected text is retrieved. ASR is the proportion of answers manipulated under perturbed queries.
• Answers shorter than three words use substring matching; answers longer than three words use GPT-4o mini review. Appendices D.1 and D.2 give the full prompts for the generator and reviewer. The authors then verify both algorithms with human judgment, with results in Table 8.
• Clean queries also report F1. The authors interpret lower F1 as not retrieving the perturbed text.

Mechanism

• Rare tokens: Emojis may be split into subwords, or become unk when out of vocabulary. The authors claim they are in the long tail, with embeddings far from high-frequency tokens but close to each other, so they disproportionately affect sentence vectors. Figure 5 corresponds to MS-MARCO, Figure 7 to NQ. The paper plots clean queries as blue circles and perturbed queries as red squares; the authors claim the former are scattered and the latter form dense clusters.
• Position shift: The paper writes the final representation as the sum of token embeddings and position embeddings. Inserting at the beginning shifts the positions of all subsequent tokens; inserting at the end does not change the relative positions of preceding tokens. The authors use this to explain why the beginning is effective and the end is not.
• High dimensionality: The authors claim retrieval models with more parameters have higher embedding dimensions, giving small changes more propagation paths. §4.1 compares clean and perturbed queries using nDCG@10 on four BEIR datasets. The main text does not give the numbers for Figure 10.

Defenses

• Dilution: Increase k so that more unperturbed text appears in the returned results. Appendix B.1 still uses N=5.
• Query sanitization: Generate five paraphrases for each query, retrieve k texts for each and answer, then aggregate. §6 says the model generating paraphrases is GPT-4o, Appendix B.2 says GPT-4. The authors claim paraphrasing removes emojis.
• Detection: First compute perplexity using tiktoken's c1100k_base. Then build a dataset and train bert-base-uncased. The authors additionally suggest S1 using special tokens for pretraining, S2 expanding the vocabulary, and S3 adding character and subword embeddings.

实验设置

• 检索器：Contriever与SPECTER为110M，Qwen2-7B为7.6B，E5-Mistral-7B、SFR-Embedding-2R、BGE-EN-ICL为7.2B，CodeBERT为124M（附录A.2）。Table 1表头写作e5-7B-mistral、SFR-Embedding、BGE-en-icl。
• 生成器：GPT-4o、LLaMA-3-8B、Qwen2.5-1.5B，temperature为0.0。§3.3.1将中间一者写作LLAMA-3.1-8B，Table 3写作LLaMA3，Table 8写作LLaMA3-8B。论文未解释这些写法。长答案评审为GPT-4o mini。
• 数据：NQ为2,681,468条文本、6,289条查询；MS-MARCO为8,841,823条文本、9,129条查询；CODE（Github-code-clean子集）为3,343,303条文本、7,450条查询（Table 7）。论文未写明Table 1的ASR分母是否等于全部查询。
• 默认扰动：表情在查询与扰动文本的开头和结尾各四次；N=5；k=5（附录A.5）。这不是单表情设置。
• 指标：扰动查询看ASR、Precision、Recall、F1；干净查询看F1。短答案用子串匹配，长答案用GPT-4o mini。BEIR四数据集用nDCG@10。
• 未写明项：Table 1、Table 2、Table 6的生成器，以及Table 3、Table 5、Table 9的检索器，论文都未说明。论文未报告重复次数。

主结果

下表是Table 1的NQ块。论文未说明生成器。

| 检索器 | 扰动ASR | 扰动F1 | 干净F1 | | SPECTER | 100.00% | 0.96 | 0.00 | | Contriever | 100.00% | 0.97 | 0.00 | | Qwen2-7B | 100.00% | 1.00 | 0.00 | | e5-7B-mistral | 100.00% | 1.00 | 0.00 | | SFR-Embedding | 99.98% | 1.00 | 0.00 | | BGE-en-icl | 100.00% | 1.00 | 0.00 |

• Table 1其余两块：MS-MARCO扰动ASR为99.97%–100.00%、F1为0.97–1.00；CODE扰动ASR为99.91%–99.98%、F1为0.96–1.00。两块干净F1均为0.00。CodeBERT在CODE上扰动F1为0.96、ASR为99.96%，干净F1为0.00（Table 2）。
• 作者称参数量超过7B的模型在所有数据集上F1为1.0，且更大模型更容易被扰动。Table 1中四个7B级检索器的扰动F1均为1.00；110M的SPECTER与Contriever为0.96–0.99，但扰动ASR仍在99.91%–100.00%。
• 作者称干净查询下没有扰动文本被检索到，系统仍按预期工作。Table 1与Table 2的干净F1均为0.00。

生成器、高级RAG与基线

• 生成器（Table 3，检索器未说明）：NQ上GPT-4o、LLaMA3、Qwen2.5的ASR为100.00%、94.85%、95.04%；MS-MARCO为99.97%、98.57%、99.98%；CODE为99.93%、94.36%、96.96%。正文写几乎所有情形超过95%；表中LLaMA3的NQ与CODE低于95%。
• 高级RAG（Table 5，N=5，k=10，检索器未说明）：NQ上Robust-RAG的F1/ASR为0.97/75.51%，Self-RAG为0.97/76.77%；MS-MARCO为0.98/79.79%与0.98/85.86%；CODE为0.99/83.16%与0.99/91.28%。作者称高级框架仍然会被EmoRAG影响。Robust-RAG只用了关键词聚合。
• 基线（Table 6，SPECTER）：三数据集上Corpus Poisoning的ASR为96.39%–97.11%，Prompt Injection为71.21%–75.39%，GCG为1.13%–3.01%，EmoRAG为99.91%–100.00%。作者称EmoRAG高于这些基线。同为NQ、SPECTER时，Table 1的EmoRAG F1是0.96，Table 6是0.97。

表情因素与不能触发的情形

• Figure 4（Contriever）：单个表情放在查询开头时，各数据集F1超过0.92；增至两个时，作者称几乎所有数据集F1为1.00。摘要称单个表情使检索到无关文本的可能性接近100%；该句的口径不是Figure 4的F1。
• Figure 6（Contriever）：开头可以形成干扰，开头加结尾略好，随机位置较弱，只放末尾无效。正文未给出该图各点F1。
• 作者称约83%的表情类型F1接近1.0，17%较低；结构更复杂者通常更高（Figure 2，共96个表情）。21组交叉实验里，只有同一表情的F1为1.0，不同表情时F1基本为0.0（Figure 9）。
• 六种emoji在开头和结尾各注入四次，库中五条恶意文本。作者称emoji远不如表情有效（Figure 8）；正文未给出数值。

防御与表示

• Table 9：释义后NQ的F1/ASR从0.96/100.00%变为0.00/0.00%，MS-MARCO从0.97/99.97%变为0.00/0.00%。检索器未说明，表中无CODE。作者称有效，但耗资源。
• 作者称增大k的稀释未能把ASR降下来（附录B.1、Figure 13）；正文未给出图中数值。Figure 12的perplexity在TPR高时FPR也高，作者称不足以分类；正文未给出FPR。
• bert-base-uncased的识别准确率为99.22%（附录B.3）；摘要写99%。作者称Figure 10中表情会破坏语义对齐；该图与Figure 5、Figure 7的PCA都没有正文数值。

其他消融与分析

• Table 4点积/余弦：NQ的F1为0.98/0.97、ASR为99.97%/100.00%；MS-MARCO为1.00/0.98、100.00%/99.98%；CODE的F1均为0.99、ASR均为99.96%。检索器未说明。正文把该实验称作Table 5，表题是Table 4。
• Figure 3（NQ）：k≤N（默认N=5）时ASR保持高位，k>N后Recall接近1；N≤k（默认k=5）时ASR随N上升，N>k后F1先升后降。正文未给出各点数值。
• Table 8（NQ、MS-MARCO）：子串匹配、GPT-4o与人工评估算出的ASR在0.99到1.0之间。论文未说明查询条数。这些数字与Table 3不是同一组结果，论文未解释差别。
• 用户研究（附录E）：32人、30条。表情两维分数低于1.75；emoji的Noticeability为4.66；乱码两维均高于4.4；每人每条平均13.7秒。
• 检测数据：摘要写来自NQ。附录B.3既写由NQ与MS-MARCO构造1,542,788条、表情池约1,500个，又写从NQ各取约760,000条扰动与干净样本；测试集约7,000条。训练为bert-base-uncased、学习率1e-5、AdamW、batch 64、3个epoch。

Limitations and future directions noted by the authors

• Appendix E: The experiments mainly focus on emoticons and do not analyze emoji or garbled text in depth; the authors plan to do so later.
• Appendix E: There is still no theoretical framework explaining how emoticons change the text representation in the retriever; the authors plan follow-up research.
• Appendix E's Future Work: query sanitization that filters adversarial characters, embedding regularization, anomaly detection for identifying perturbed text, as well as S1 special token pretraining, S2 expanding the vocabulary, and S3 character and subword embeddings.
• Appendix B.2: Paraphrasing can remove emoticons, but it is time-consuming and resource-intensive, requiring more efficient query sanitization.
• Appendix B.3: The current detection only targets one type of special character, emoticons; training separately for each type is costly, and methods that cover more perturbation patterns are needed.
• The conclusion section does not list limitations item by item. In that section, the authors state they will advance representation learning and improve the system's safety when handling complex, unpredictable inputs.

Experimental coverage

• The retrievers are the seven in Appendix A.2, the generator slots are GPT-4o, LLaMA-3-8B, and Qwen2.5-1.5B; the data are NQ, MS-MARCO, and CODE (Table 1–3, Table 7). The notation for the same generator is inconsistent across §3.3.1, Appendix A.2, and Table 3.
• The advanced framework experiments include Robust-RAG's keyword aggregation and Self-RAG, with settings of N=5 and k=10 (§3.4, Table 5).
• The comparison attacks are Corpus Poisoning, Prompt Injection, and GCG Attack, with SPECTER as the retriever (Appendix D.3, Table 6).
• The defense experiments include dilution, five paraphrases, perplexity, and bert-base-uncased; there are also six emoji comparisons (Figure 8) and nDCG@10 on BEIR (Figure 10).
• The paper does not report the ASR denominator for Table 1, the generators for Table 1, Table 2, and Table 6, the retrievers for Table 3, Table 5, and Table 9, the number of human evaluation items, the number of repetitions, and the specific values for Figure 8, Figure 10, Figure 12, and Figure 13.

EmoRAG measures RAG's response to symbol perturbations: when the same emoji appears in both the query and the library text, retrieval deviates from semantic matching.

The authors examine whether retrieval is truly determined by semantic relevance. In implementation, they do not change the retriever or generator parameters, only inject the same emoji into the query and text less than 0.01‰ of the library content. The default is four times at the beginning and four times at the end, N=5, k=5.

In Table 1, on NQ, SPECTER's perturbed query ASR is 100.00% and F1 is 0.96; the clean query F1 for all six retrievers in that table is 0.00. In Table 1, the perturbed ASR for MS-MARCO and CODE falls between 99.91% and 100.00%. On Contriever, with a single emoji placed at the beginning of the query, F1 exceeds 0.92 across datasets (Figure 4). The abstract describes the retrieval consequence of a single emoji as nearly 100%, which differs in metric from the F1 in Figure 4.

Among the nine ASRs in Table 3, the lowest is LLaMA3's 94.36% on CODE; the paper does not specify the retriever for that table. In Table 5, on NQ, Robust-RAG's ASR is 75.51% and F1 is 0.97. In Table 9, after paraphrase defense, NQ's ASR is 0.00%. In Appendix B.3, bert-base-uncased's recognition accuracy is 99.22%.

The authors conclude that semantic relevance and retrieval success can be decoupled, and explain this with rare tokens, position shift from beginning insertion, and high-dimensional embeddings. The authors believe methods like perplexity are insufficient for classification; paraphrasing and specialized detection can handle emojis. The abstract states that defense-related components have been open-sourced, while Appendix B.3 says the models and datasets are planned for release.