DECEIVE-AFC: Adversarial Claim Attacks against Retrieval-Augmented LLM Fact-Checking Systems

Can merely rewriting the input claim cause search-enhanced LLM fact-checking systems to give incorrect verdicts.

• Scenario: The authors state that misinformation affects public opinion and undermines trust in institutions, and manual verification struggles to keep up with the volume. Automated fact-checking (AFC) retrieves evidence and reasons, outputting verdicts and rationales. Search-enhanced systems use LLMs to interpret claims, generate queries, gather evidence from open web pages, and summarize.
• Existing shortcomings: The authors state that existing adversarial claim attacks are mostly designed for traditional DNNs, with evidence from static corpora or directly fed golden evidence; character-level and word-level modifications struggle against stronger understanding, dynamic retrieval, and reasoning. Evidence attacks and claim–evidence pair attacks usually require modifying the evidence source or benchmark pairing, which the authors consider often unrealistic in open web retrieval.
• Observation: The claim is the only interface that starts the entire pipeline. The authors believe that, while preserving the original factual intent and without writing explicit falsehoods, modifying wording or structure may still disrupt query, retrieval, and subsequent reasoning.
• Proposed method: Propose an agentic framework DECEIVE-AFC that searches attack trajectories under black-box conditions and constrains whether claim-level attacks remain effective.
• Threat model:
  • Goal: Modify claim c to c′, making the verdict incorrect while preserving factual content and keeping the rationale relevant to c′ to avoid being spotted at first read.
  • Constraint formulation: Section 3 writes Sim(c, c′)≤τ, where Sim is defined as semantic distance (lower means more similar); and Corr(c′, jc′)≤γ. The values of τ and γ are not given in the paper.
  • Knowledge: Only the target system's inputs and outputs are visible; internals are unknown, and the evidence source cannot be controlled. The abstract calls it input-only, the main text calls it black-box, and states this matches common deployment of commercial systems.
  • Capability: The surrogate system can be queried arbitrarily to generate c′, then transferability is used to attack the target. The victim is a search-enhanced LLM fact-checking system that includes retrieval, verdict, and rationale generation.

According to Section 2, related work is divided into automatic fact-checking and adversarial attacks on fact-checking systems; experimental baselines come from FACTEVAL.

Automatic Fact-Checking

• Standard pipeline: Given a claim, evidence is retrieved, followed by verdict and rationale generation (Akhtar et al., 2023; Guo et al., 2022; Thorne & Vlachos, 2018). The mechanisms and models used at each stage vary greatly.
• DNN systems: Evidence is gathered from pre-built document collections via lexical matching, embedding similarity, or learned retrieval models (Pan et al., 2023a; Nakov et al., 2021a; Wu et al., 2024), and neural models jointly model claims and evidence (Thorne et al., 2018; Kotonya & Toni, 2020; Nakov et al., 2021b). The authors state that these rely on supervised training on fixed datasets, with static corpora limiting coverage and timeliness, citing Guo et al. (2022) and Hu et al. (2022).
• LLMs and search augmentation: Zero-shot, few-shot, and chain-of-thought are used for fact-checking (Pan et al., 2023b; Hu et al., 2024; Caramancion, 2023; Tahmasebi et al., 2024). The authors state that relying solely on pretrained knowledge is still insufficient; subsequent work integrates external search, where LLMs interpret claims, generate queries, and summarize web content (Qi et al., 2024; Wang et al., 2024; 2025; Zhang & Gao, 2023; Xuan et al., 2024; Braun et al., 2024).

Adversarial Attacks on Fact-Checking

• Adversarial claim attacks: Claims are rewritten while preserving factual intent, e.g., via paraphrasing, entity substitution, or temporal and numerical changes, to induce incorrect verdicts (Hidey et al., 2020; Kim & Allan, 2019; Thorne et al., 2019; Mamta & Cocarascu, 2025). Only the claim is modified, not the evidence source directly.
• Adversarial evidence attacks: Misleading evidence is injected, modified, or re-ranked during the retrieval stage (Boucher et al., 2022). The authors state that such attacks typically assume the ability to tamper with evidence databases, knowledge bases, or retrieval indexes.
• Claim–evidence pair attacks: Both the claim and its evidence are manipulated simultaneously, often constructing difficult pairs in existing benchmarks (Schuster et al., 2019). The authors state that these rely on strong control over evidence sources or benchmark construction. Liu et al. (2025) is cited as a survey categorizing attacks along the two attack surfaces of input claims and retrieved evidence.

Baselines and Benchmarks

• Baseline methods: FACTEVAL (Mamta & Cocarascu, 2025) includes LEET, Homoglyph, Character Swap, and Phonetic, covering visual confusion, character noise, and phonetic variants. The authors cite their attack success rates on LLM fact verification as the reason for selection; this is not the result in Table 2 of this paper.
• Benchmark: MOCHEG (Yao et al., 2023). Experiments manipulate existing claims on the test set and remove Not Enough Information (NEI).

The authors position this work within claim-level attacks: the assumptions of evidence-based attacks are difficult to satisfy in open web retrieval; the introduction states that existing claim attacks mainly target traditional DNNs or static retrieval and are unlikely to cause system-level failures in search-augmented LLM fact-checking systems.

DECEIVE-AFC writes the attack as a closed loop: rewrite the claim according to a strategy, read the verdict and reasoning on a surrogate, and then a planner decides to stop, continue, or switch strategies. The target system's internals and evidence sources are not visible.

Problem Setting and Attack Interface

• The target system is written as f(c, E(c)) = (y_c, j): c is the claim, E(c) is the retrieved evidence, yc is the verdict (0 means the claim is true, 1 means otherwise), and j is the reasoning.
• The attack only replaces c. A valid attack must simultaneously satisfy incorrect verdict, unchanged factual intent, and reasoning related to the adversarial claim. The τ and γ in Section 3 are not implemented numerically.
• The refinement stage uses a surrogate to avoid repeatedly querying a rate-limited target system. Each query is independent, and no historical claims are retained to avoid contamination from previous interactions.

Three Types of Claim-Level Strategies

Strategies are grouped by pipeline stage, with explanations in Appendix A. Below, only the categories and targets are kept.

• Search Engine Misguidance: Modify wording and structure to affect query generation and ranking. Sub-strategies are low-frequency synonyms, non-standard entity references, redundant background, and keyword dispersion. The authors claim factual intent is preserved, but irrelevant or low-quality evidence is more likely to be retrieved.
• LLM Reasoning Disruption: Target the organization of retrieved results into evidence, as well as the final verdict and reasoning. Sub-strategies are inserting factually correct but irrelevant statements, increasing syntactic complexity, conditional or speculative tone, and double negation. The authors claim this does not change the factual correctness of the claim itself but increases reasoning ambiguity.
• Structural Complexity Escalation: Turn single-hop facts into multi-hop questions. Sub-strategies are splitting explicit entities into indirect references and rewriting atomic facts into compound relations. The authors claim that an error in any intermediate hop could break the verification chain.

Effectiveness Evaluation

• Refinement stage: The verdict must be incorrect; semantic similarity is used to prevent obvious off-topic (citing Meng et al., 2013 in the main text); natural language inference (NLI) is used to check that the original fact is not changed into a contradiction (Wang & Jiang, 2016); then an LLM judges whether the reasoning is related to the adversarial claim.
• Final evaluation: Verdict flip remains a necessary condition. The similarity threshold is lowered, and NLI is relaxed from bidirectional entailment to no explicit contradiction. Samples that fall below the strict refinement threshold but meet the relaxed conditions undergo manual confirmation to check whether the factual intent has changed and whether the text remains readable.
• Implementation (Appendix C): Similarity uses Sentence-BERT (all-mpnet-base-v2), with a refinement threshold of 0.85 and a final relaxation to 0.7; NLI uses DeBERTa-large-MNLI. Section 3 defines Sim as semantic distance and writes it as ≤τ, while the appendix implements the above similarity thresholds. The paper does not report the number of people involved in manual confirmation.

Agent Refinement

• Adversarial Claim Generator: GPT-4o. Generates or rewrites candidates according to the selected strategy or previous feedback, retaining previous variants and feedback within the same conversation.
• Victim Module: The surrogate is GPT-4o-search-preview with web retrieval, returning verdict and reasoning. No history is shared between queries.
• Attack Evaluator: Also GPT-4o. Compares whether the verdict flips between benign and adversarial claims, the degree of reasoning shift, etc. Each evaluation is independent and does not see previous evaluations.
• Semantic Guard and Attack Planner: The guard checks semantic drift according to Section 4.2. The planner decides to terminate based on evaluation and guard, or to continue within the current strategy, switch to a variant of the same family, or move to another high-level strategy, then returns guidance to the generator.
• Budget: At most 10 rounds per claim. The strategy space is the combination of strategies in Section 4.1. Only adversarial claims that meet the semantic preservation criteria replace the original claim in the evaluation set.

Experimental Setup

• Models: Targets are HiSS (hierarchical step-by-step prompting, decomposing claims into sub-claims), LEMMA (generating multiple queries and feeding external knowledge into reasoning), DEFAME (zero-shot, can dynamically select tools and search depth). The surrogate used for refinement is GPT-4o-search-preview. Both the generator and evaluator are GPT-4o.
• Data: MOCHEG test set after removing NEI has 1642 items, with 817 positive and 825 negative. The approach manipulates existing claims, not generating from scratch.
• Baselines and settings: FACTEVAL's LEET, Homoglyph, Character Swap, Phonetic. All are black-box. Attacks are refined on the surrogate and then transferred to the three targets.
• Metrics: ASR is the proportion of incorrect verdicts under the validity constraints in Section 4.2. Also report Acc, Prec, Rec, F1. Rationale comparison uses ROUGE-1/2/L, MAUVE, SummaC; the authors state that higher SummaC indicates stronger semantic consistency, and negative values indicate weaker consistency.
• Validity threshold: During refinement, similarity threshold is 0.85 with bidirectional NLI entailment; final evaluation threshold is 0.7, with NLI relaxed to no explicit contradiction. Only those meeting the relaxed condition undergo manual confirmation. The similarity model is Sentence-BERT (all-mpnet-base-v2), and the NLI model is DeBERTa-large-MNLI.
• Budget: At most 10 rounds per item. The cost of the round ablation is calculated per 100 items per tier. The paper does not report the number of repetitions for ASR, nor the sample size and consistency of manual confirmation.

Main Results

According to Table 2. The baseline column is the maximum of the reported ASR for that system and the corresponding method, not a summary statistic separately reported by the paper.

Swipe horizontally to read the table

Target	Benign Acc	DECEIVE Acc	DECEIVE ASR	Max reported baseline ASR
HiSS	77.5%	61.9%	19.5%	8.5% (Homoglyph)
LEMMA	66.1%	51.1%	18.8%	12.1% (LEET)
DEFAME	78.7%	53.7%	31.4%	8.9% (LEET)

• The authors state that DECEIVE-AFC achieves the highest ASR across all three systems, and adversarial claims can transfer across systems. The 78.7% to 53.7% in the abstract matches the DEFAME row. In Table 1, when the target is the surrogate model GPT-4o-search-preview, accuracy drops from 90.5% to 57.4%, with an ASR of 36.6%.
• On LEMMA, accuracy drops from 66.1% to 51.1%, recall rises from 64.9% to 79.6%, and F1 goes from 62.3% on benign to 61.9% on adversarial (the percent sign is not printed in that cell of Table 2). The paper does not separately explain this row.
• LEET is not reported on HiSS. The authors state that heavy spelling distortion leads to refusal predictions and is hard for humans to read. The reported baseline ASR in Table 2 ranges from 4.3% to 12.1%. The authors state that these traditional perturbations cause limited performance degradation.

Rationale Similarity

• What was measured: The rationales of successful and failed adversarial claims are compared with their benign rationales (Table 3). The paper does not specify which target system this table corresponds to.
• Results: Successful: ROUGE-1 0.470, ROUGE-2 0.257, ROUGE-L 0.324, MAUVE 0.385, SummaC -0.389; failed: 0.617, 0.423, 0.465, 0.945, -0.193.
• Authors' interpretation: Rationales of failed cases are closer to benign rationales; successful attacks introduce shifts in evidence use and reasoning structure. Section 6 states that incorrect verdicts are accompanied by seemingly relevant and coherent rationales, making them harder to detect by rationale alone.

Result Types of the Refinement Process

• What was measured: Figure 3 categorizes optimization result types for successful (left) and failed (right) cases. The legend includes Shift justification, Degraded justification, Resistant, Resistant but degraded, Consistent justification, Flip but invalid, Others.
• Results: On the successful side, 6.4%, 29.8%, 63.8% are visible; on the failed side, 7.2%, 0.5%, 29.5%, 62.7% are visible. The plain text does not specify which percentage corresponds to which category.
• Authors' interpretation: Success mainly comes from justification shift or degradation; failure mainly comes from model resistance, or reasoning degradation without verdict flip; a small portion is invalidated by semantic constraints.

Semantic Guard

• What was measured: Table 4 compares the full framework with the semantic guard removed. The table title does not specify the model. The full framework's ASR of 36.6% matches the DECEIVE-AFC row in Table 1. The final evaluation still applies semantic constraints.
• Results: The full framework has a label flip rate of 42.1% and ASR of 36.6%; without the guard, 43.3% and 23.1%.
• Authors' interpretation: Removing the guard slightly increases flips, but many flips fail the semantic constraints in final evaluation, so ASR decreases. The authors state that the guard guides optimization, not just post-hoc filtering.

Other Ablations and Analyses

• Refinement rounds from 1 to 10 (Figure 4): the x-axis is Refinement Rounds, the left y-axis is Attack Success Rate, the right y-axis is API Cost (USD). The authors state that ASR increases with rounds, gains diminish beyond a certain number of rounds, cost rises, and excessive refinement may over-optimize. The main text does not give per-round readings. Cost is calculated per 100 items per tier.
• The authors state that GPT-4o-search-preview has high accuracy on benign inputs because it often retrieves authoritative fact-checking articles that directly contain the original claim and verdict, so they do not treat it as an independent verification system; because it is easy to deploy, fast in reasoning, and moderate in cost, it is used as a surrogate for large-scale black-box evaluation.
• Table 1 (target is that surrogate model): benign Acc 90.5%, Prec 95.1%, Rec 85.4%, F1 90.0%; under DECEIVE-AFC, Acc 57.4%, Prec 61.1%, Rec 40.1%, F1 48.4%, ASR 36.6%.

Limitations and future directions noted by the authors

The paper has no dedicated Limitations or Future Work section, nor does it frame the experimental scope as method limitations.

• Fine-tuning the fact-checking model (Section 6): The authors state that most systems use general-purpose base LLMs lacking safety alignment against adversarial claims; they propose fine-tuning with paired benign claims and adversarial variants to learn common perturbation patterns and explain their semantics.
• Interpretation and retrieval validation (Section 6): The authors propose adding adversarial-aware correction or validation in the intermediate stages of interpreting claims, generating queries, and checking against evidence, to identify erroneous queries and low-confidence evidence.
• Restatement in the conclusion (Section 7): The authors state that discussing these mitigations is to improve existing practices and facilitate designing more robust fact-checking frameworks, without listing limitations separately.

Experimental coverage

• The target systems tested are HiSS, LEMMA, DEFAME, 3 in total; the surrogate used in Table 1 and the refinement stage is GPT-4o-search-preview.
• Data is the MOCHEG test set with NEI removed, 1642 items (positive 817, negative 825), set as manipulating existing claims.
• Baselines are LEET, Homoglyph, Character Swap, Phonetic from FACTEVAL; set to unified black-box, with generation and evaluation agents as GPT-4o, at most 10 rounds per item.
• Validity calculation uses Sentence-BERT (all-mpnet-base-v2) and DeBERTa-large-MNLI; samples meeting only the relaxed threshold are additionally human-confirmed.
• Ablations include semantic guards (Table 4) and refinement rounds 1–10 (Figure 4, cost per 100 items per tier). The paper does not report the sample size and agreement for human confirmation, per-substrategy ASR, number of ASR repetitions, or per-round readings for Figure 4. Tables 3 and 4 do not specify the target system.

DECEIVE-AFC studies whether search-enhanced LLM fact-checking gives incorrect verdicts under black-box, claim-only modification conditions.

• Problem: Open web retrieval makes attacks that tamper with evidence sources difficult to implement. The authors believe that claim attacks targeting static corpora or character noise cannot defeat the query generation, evidence retrieval, and LLM reasoning of such systems.
• Method: Three strategy categories target retrieval, reasoning, and multi-hop structure respectively. Similarity, NLI, and rationale relevance are used to preserve factual intent. GPT-4o handles generation and evaluation, GPT-4o-search-preview serves as surrogate, and a planner decides whether to switch strategies within at most 10 rounds.
• Transfer results: On the MOCHEG test set with NEI removed (1642 items), DEFAME accuracy drops from 78.7% to 53.7%, with ASR of 31.4% (Table 2). HiSS and LEMMA have ASR of 19.5% and 18.8%, higher than the character-level baselines reported in the same table. LEET is not reported on HiSS.
• Surrogate model and exceptions: When the target is GPT-4o-search-preview itself, accuracy drops from 90.5% to 57.4%, with ASR of 36.6% (Table 1). On LEMMA, F1 goes from 62.3% to 61.9%, recall rises from 64.9% to 79.6%, and the paper does not explain this separately.
• Authors' conclusion: The authors state that these failures come from disrupted retrieval and downstream reasoning, not surface perturbations; removing semantic guards drops ASR from 36.6% to 23.1% (Table 4). Section 6 proposes fine-tuning the fact-checking model and adversarial-aware validation in query and retrieval stages. The paper does not specifically discuss method limitations.