Security of LLM-as-a-Judge: A Comprehensive SoK

The core question is: after LaaJ enters benchmarks, reinforcement learning, and safety filtering, judges can be both adversarially manipulated and used as attack or defense tools, yet the security literature still lacks a framework organized by role.

• Scenario: The authors state that LaaJ uses language models to evaluate correctness, quality, safety, and alignment, and has entered benchmarks, reinforcement learning pipelines, automated evaluation, and AI-assisted review. The authors believe that once it enters critical pipelines, whether the judge is trustworthy can affect evaluation results.
• Existing gaps: The authors state that existing research has pointed out reliability issues such as position bias, but the work is scattered across communities such as NLP, software security, and AI security. The authors believe there is no framework yet that organizes these security implications.
• Authors' categorization: In the literature, LaaJ can be an attack target, an attack tool, a defense or security analysis tool, or an evaluation strategy in security tasks. This paper is a survey, not a new attack or defense algorithm.
• What this paper does: The authors claim this is the first SoK focused on LaaJ security. They analyzed 863 papers, selected 45 from 2020–2026, provided a taxonomy, and compared limitations, threats, and open questions.
• Threat setting (as summarized by the survey, not a single attacker):
  • Goal: Manipulate the judge's scores, selections, preference labels, or safety judgments; or use the judge as an optimizer to attack other systems.
  • Knowledge: The cited work distinguishes web poisoning, malicious annotation, weight poisoning, and black-box settings that do not access parameters, training data, or evaluation inputs and only modify natural-language rubrics.
  • Capability: At inference time, can modify the evaluated content, candidate answers, or evaluation prompts; at training time, can poison data, annotations, or weights. In JudgeDeceiver-type settings, the attacker controls only one candidate answer and does not know competing answers.
  • Victim systems: LaaJ evaluators, as well as rankings, RLAIF/RLVR, safety filtering, RAG reranking, or embodied agents that depend on them; the specific systems depend on each cited work.

The authors position this paper using Section 3 and Table 2: comparing year, time range, number of included papers, whether definitions are given, whether evaluation is treated as system measurement, whether safety is discussed, and domain. Below only works explicitly discussed in the main text are retained.

General LaaJ Surveys

• Gu et al. (2024): Provides formal definitions and taxonomy, discusses consistency and bias mitigation, and proposes a reliability-oriented benchmark. Table 2 records about 220 papers, 2023–2025. The main text places it on the functionality and reliability side, not adversarial robustness.
• Li et al. (2024) [26]: Discusses functionality, methods, applications, meta-evaluation, and limitations; the limitations section has a short paragraph on adversarial attacks. Table 2 records about 300 papers, 2023–2024. The authors consider dedicated security research still scarce.
• Li et al. [23]: Three-dimensional taxonomy by what is evaluated, how it is evaluated, and how benchmarks are made. The main text states safety is only briefly treated as one evaluation output, alongside usefulness, reliability, and relevance. Table 2 records year 2024, about 270 papers; the reference is a 2025 conference paper.

Domain Surveys

• He et al. (2025): Software engineering oriented, discusses using LLMs to evaluate code and software artifacts, and provides a roadmap to 2030. The main text states adversarial threats appear only in limitations and are domain-specific. Table 2 records 42 papers.
• Medical and recommendation: Kocaman et al. (2025) and Genovese et al. (2026) review LaaJ in medical settings; Nawara and Kashef (2025) review recommendation systems and treat LaaJ as an evaluation practice. The main text treats the first two as narrative reviews and offers no separate security critique for the latter.
• Cao et al. (2025): Moves evaluation from human to automated pipelines, with LaaJ as one component. The authors state it does not specifically analyze LaaJ security risks or adversarial vulnerabilities. Table 2 records about 370 papers.

Agent-as-a-Judge

• Yu (2025) and You et al. (2026): Connect methodological evolution to Agent-as-a-Judge and provide agent evaluation frameworks and taxonomies. Table 2 records about 60 and about 40 papers, respectively. The main text does not describe them as security surveys.
• Entries listed in Table 2 but not expanded in Section 3: Xu et al. (2024) focuses on benchmark data contamination; Li et al. (2025) [27] is software engineering, with the safety column written as briefly mentioned limitations. Checkmarks cannot be discerned in plain text, so no judgment is made on which columns are checked.

The authors state: LaaJ has been widely studied, but security surveys covering adversarial robustness, attack vectors, and defense strategies are largely missing; this paper fills that gap with 45 cross-domain papers. Table 2 records this paper as 2026, timeline 2020–2026, 45 papers, Cross-domain.

This paper does not propose a new algorithm, but provides a LaaJ safety taxonomy, using a unified background definition to support the subsequent organization of attack, defense, and evaluation literature.

Literature Screening

• Sources: Google Scholar, IEEE Xplore, ACM Digital Library, Springer, ScienceDirect. Keywords are “LLM-as-a-Judge” plus security, privacy, toxicity, reliability.
• Inclusion: Priority to Q1 journals and A*/A conferences; exclude duplicates, review articles, and non-English. Due to the field's novelty, a few arXiv preprints are included, conditional on at least one author having over 5,000 citations.
• Scale: A total of 863 papers were analyzed, selecting 45 from 2020–2026. The authors did not report the number of screeners or agreement.

Definitions and Process

The authors write LaaJ as producing a result for a sample in an evaluation context: R=P_θ(X_n,C). Pθ is an LLM evaluation function with parameters θ, Xn is the sample to be evaluated, C can include instructions, criteria, examples, or history, and R can be a score, label, ranking, preference, or natural language rationale.

n=1 is point-wise, n=2 is pair-wise, n>2 is list-wise. Protocols are divided into scoring, ranking, and selection. The process in Figure 1 is in-context learning (ICL, without updating parameters during inference), model selection, prompt design, output post-processing, and meta-evaluation against human judgment. Under the scoring protocol, the authors write the final selection as ŷ=argmax_{yj∈ Y}S_θ(y_j∣ x,C), i.e., the candidate evaluation with the highest score. Application targets are divided into four categories: evaluating model outputs, evaluating data, evaluating agents, and evaluating reasoning processes.

Safety Taxonomy

• Five perspectives: Section 4 is divided into (i) attacking LaaJ, (ii) launching attacks through LaaJ, (iii) using LaaJ for defense, (iv) evaluating LaaJ itself, and (v) using LaaJ as an evaluation strategy in safety-related tasks.
• Figure 2: Groups (iv) and (v) under Evaluation. Attack targets are Raina, Shi, Wei, Li, Maloyan, Tong, Zhao, Ding, totaling 8 papers; attacks launched through LaaJ are Liu et al., 1 paper; defenses are Mao, Li (J-Detector), Shen (GPTracker), totaling 3 papers; evaluation targets are Ye et al. and Lai et al.
• Figure 3: As attack targets, further divided into training-time (backdoor and poisoning; rubric/protocol manipulation) and inference-time (prompt injection; token/surface perturbation; robustness evaluation).

Reviewed Mechanisms and Organization Approach

• Training-time: BadJudge is divided into web poisoning, malicious annotation, and weight poisoning. RIPD only modifies the rubric through natural language, causing target-domain preferences to deviate from human references, while benchmark consistency remains around 0.85; the main text calls the search strategy Biased Rubric Search, and the bias can enter downstream methods like DPO.
• Inference-time: Raina et al. search for short universal phrases on a small surrogate and then transfer. Maloyan et al. distinguish content-author attacks from system-prompt attacks. JudgeDeceiver formulates injection as an optimization problem, using shadow candidates to simulate unknown competing responses, with the loss covering target judgment, position robustness, and perplexity. The injected sentences are not paraphrased.
• Surface forms and attacker usage: Emoji Attack exploits tokenization segmentation and uses context to have the evaluated text insert emojis. Zhao et al. study reward judges giving false positives to shallow “master key” inputs; the mitigation is training a dedicated Master-RM on truncated reasoning openings. Liu et al. use the judge as an optimization engine to determine whether poisoned demonstrations look human-written.
• Tables and dimensions: Attacks use Table 4, defenses use Table 5, and evaluations use Table 6. Figure 4 further divides “evaluation tools” into attack detection, code review/software engineering, intelligent environments, and other domains. Table 7 describes practices using seven independent binary dimensions: black-box/white-box, absolute/relative scoring, cloud/local, first-order/higher-order, rubric-integrated/single, internal/external knowledge, and post-hoc/ante-hoc.

This paper contains no new model experiments. The numbers below are all results that the authors extracted from the cited studies into tables or the main text.

Experimental Setup

• Literature scope: 863 papers analyzed, 45 included (2020–2026). Rules for databases, keywords, Q1 and A*/A priority, exclusion of reviews and non-English works, and preprints with more than 5,000 citations are given in Section 1.
• Classification: Five perspectives in Section 4; Table 3 records year, scope, classification, and application context. Figure 2 gives corresponding citations.
• Threat table: Table 4 summarizes 9 items, recording judge role, attack category, judge model, metric, main result, and benchmark. The authors did not rerun these models.
• Defenses and evaluation table: Table 5 covers three mitigation strategies; Table 6 distinguishes whether the judge is the evaluation target (T) or an evaluation tool (I), and records framework, benchmark, metric, and main result.
• Metrics: The cited works commonly use ASR, score inflation, FPR, detection rate changes, Accuracy, F1, AUC, Precision, etc. This paper does not uniformly recompute them and has no own decision thresholds.
• Sample size and replication: The paper does not report the number of reviewers, agreement, or duplicate coding for literature screening. Replication counts for each cited experiment are not added where not listed in the tables.

Main Results

Cited study (Table 4)	Condition	Table 4 main result
Maloyan et al. 2025	Gemma-3-4B-Instruct; Gemma-3-27B-Instruct	ASA highest ASR 73.8%; CM 67.7%
Wei et al. (listed as 2024)	ShieldLM et al.; ten judges	Detection drops up to 12%; ShieldLM from 71.9% to 3.5%
Tong et al. 2025	1% poisoning; web poisoning	Highest inflation 3× (1.4→4.6/5); web poisoning inflates 20%; toxic prompts misclassified 89%
Liu et al. 2025	Davinci-002; ProgPrompt et al.	ASR ≥82.5%, 83.3%, VisProg >80%
Zhao et al. 2025	10 keys, 5 benchmarks	FPR up to 80%
Li et al. 2025	GPT-4o; prompt templates	PAIR and Combined ASR ≈70%; robustness differs by up to 40%
Shi et al. 2024	Open-source LLMs (e.g., Llama-3-8B); closed-source models written as implicit	Defenses miss ≥70% of attacks; ASR only written as Very high

• The above table omits Raina et al. and Ding et al. Table 4 writes Raina et al. ASR up to ≈70%, perplexity detection modestly effective; Ding et al. reduces usefulness accuracy by 9.5%, harmlessness by 27.9%, win-rate drops to ≈40%. Table 3 lists Wei et al. as 2025, Table 4 as 2024, reference [55] as 2025.
• The main text and Table 4 are not fully identical. BadJudge main text writes weight poisoning scores from 1.5/5 to 4.9/5, 1% poisoning ASR over 76%, guardrails judging toxic content as safe 83.9%, RAG re-ranking placing poisoned documents first 96.9%. JudgeDeceiver main text writes four open-source LLMs, average ASR 89%–99% on MT-Bench and LLMBar; injections optimized on Llama-3-8B are 88% against Claude-3 Sonnet and 79% against GPT-4; false negatives for three detections are 40%–100%.
• The authors state that absolute scoring is more susceptible to short generic phrases than pairwise comparison. Maloyan et al. main text states system prompt attacks are on average 15 percentage points higher than content author attacks; open-source models average success rates 50%–68%, GPT-4 and Claude-3-Opus 27%–44%. Zhao et al. main text writes FPR up to 80%–90%, not fully identical to Table 4's up to 80%.

Defense-Oriented Mitigations

• Audit scoring: J-Detector in Table 5 has accuracy ≈85%–95%, F1 and AUC >0.9. Data written as multiple datasets, names not listed. The authors state LLM scoring and human scoring can be separated and bias can be observed.
• Ecosystem monitoring: GPTracker collected over 750,000 GPT instances over eight months, flagged 2,051 GPTs violating terms, precision ≈73% (Table 5). The authors state evasion methods include external APIs, hiding instructions in prompts, and redirecting to external domains. Payloads not written.
• Interpretable detection: LLMVulExp in Table 5 has precision, recall, F1 ≈90%; dataset only written as standard vulnerability datasets. Method is LoRA instruction tuning, Chain-of-Thought, and key code extraction. The authors state explanations include vulnerability type, location, and possible fixes.

Evaluation Targets and Evaluation Tools

• Evaluation targets: CALM lists 12 bias categories, metrics are robustness rate and consistency rate; main text states models prefer longer answers or specific positions, no percentages given. BiasScope error rate over 50% on JudgeBench-Pro (Table 6 and main text consistent).
• Attack detection: CyberRAG classification accuracy 94% (Table 6). CrossInject ASR increases up to 71.7%, PNA always 100%. SocketAI precision 16% higher and F1 9% higher than CodeQL; GPT-4 F1 97%; hybrid filtering reduces analysis volume by 78% (Table 6).
• Code and adoption: HyVD-VP in Table 6 is 95% F1; main text writes accuracy 95.2%, F1 95.1%, F1 18.3% higher than Fortify, and found 16 previously unknown vulnerabilities in six enterprise projects. RevMate direct adoption rate 8.1% (Mozilla) and 7.2% (Ubisoft) (Table 6). SE-Jury correlation improvement over baseline 29.6%–140.8% (Table 6). Table 6 states jailbreaks suppressing refusals are effective in low-resource languages, fine-tuning may increase bias; the authors give no percentages in that cell.

Other Ablations and Analyses

• Liu et al.: Removing judge-driven optimization drops ASR from over 90% to about 20%–25%; real-vehicle collision rate under attack conditions 80% (Section 5.2).
• Maloyan et al.: Single defenses bypassed 32%–67%; a committee of seven architecturally different models reduces ASR to 10%–19% (Section 5.1.2).
• Raina et al.: Perplexity detection F1 0.70–0.82; the authors state adaptive attacks optimizing both high and low perplexity may bypass. Main text surrogate is FlanT5-3B, Table 4 writes FlanT5-xl.
• Li et al.: Main text states Combined Attack ASR 100% on Openchat-3.5, Qwen-2.5-7B, Mistral-7B; PAIR 100% on Qwen-2.5-7B and 90% on Llama-3.1-8B; not the same condition as Table 4's GPT-4o ≈70%. PAI-Judge sub-scores rise from about 1.5 to over 6.0. The authors state Re-tokenization and LLM-based detectors are most effective, the former harms normal performance, the latter has high computational cost.
• Shi et al. [46]: Removing manual rubric refinement drops κ from 0.57 to 0.38; free-form rubric 0.29. On consistent patches κ=0.75, NPV=0.94; on disputed patches κ=0.57.
• Baek et al.: Removing explanation supervision drops explanation score from 4.90 to 2.43/5, detection accuracy drops only 0.5 percentage points. CodeGemma-FT detection accuracy 99.3%, human teacher 86.5%.

Limitations and future directions noted by the authors

• Adversarial prompt manipulation (Section 8.1): The authors state that malicious users can craft responses according to the model's reasoning patterns, thereby influencing the judge. The open problems listed are: building LLM evaluation systems that resist adversarial attacks, designing methods to detect prompt injection and evaluation manipulation, and designing defenses for the evaluation pipeline.
• Position bias (Section 8.2): The authors state that the order of candidates can affect results, and attackers may exploit the ordering. Open problems are position-invariant evaluation frameworks, more robust ranking mechanisms, and quantifying and reducing position bias.
• Length and style bias (Section 8.3): The authors state that longer, more fluent, or better-formatted responses may receive higher scores even if factual quality is low. Open problems are metrics that prioritize factual correctness, normalization to reduce length bias, and separating style quality from content quality.
• Three limitations of current practices (Section 9): The authors state these include judges being susceptible to adversarial manipulation, a lack of standardized evaluation methods, and a lack of robust benchmarks for assessing the safety and reliability of LaaJ.
• Future directions (Section 9): The authors list standardized benchmarks for LaaJ robustness, defenses against adversarial manipulation, and hybrid evaluation combining automatic judges with human oversight.
• Source of problems (beginning of Section 8): The authors state that many problems arise from LLMs' sensitivity to prompt manipulation, bias, and output variability, and that obtaining a safe evaluation framework requires addressing these characteristics.

Experimental coverage

• The search in Section 1 covers Google Scholar, IEEE Xplore, ACM Digital Library, Springer, and ScienceDirect; 863 papers were analyzed, and 45 were included (2020–2026), excluding duplicates, surveys, and non-English works.
• Figure 2 covers attack targets, attacks implemented via LaaJ, defenses, evaluation objects, and evaluation tools; Figure 3 further divides attack targets into training-time and inference-time.
• Table 4 summarizes 9 attack studies; Table 5 covers J-Detector, GPTracker, and LLMVulExp; Table 6 summarizes works that use LaaJ as an evaluation object or evaluation tool.
• Figure 7 marks 2020–2023 as 0, 2024 as 3, 2025 as 33, and March 2026 as 12; Figure 8 shows conferences 19, journals 15, arXiv 11. The paper does not explain how the Figure 7 annotations correspond to the 45 papers.
• The paper does not report the number of reviewers or the consistency of literature screening, nor does it report independent replication of the cited experimental figures.

The authors collected literature from 2020–2026 into a LaaJ security SoK, to illustrate what role judges play in security. The authors claim this is the first SoK in this direction.

• Problem: Automated evaluation has entered benchmarks, alignment training, and safety filtering, but judges can be manipulated, and can also be used to carry out attacks or support defenses; related work is scattered, and the authors believe a structured framework is missing.
• Approach: Analyzed 863 papers, included 45, categorized by attack target, attack tool, defense tool, evaluation target, and evaluation tool. In the background, they define a judge as giving a score, ranking, or selection for samples under contextual conditions, and distinguish point-wise, pair-wise, and list-wise.
• Publication: Figure 7 marks 3 papers in 2024, 33 in 2025, 12 as of March 2026, and 0 for 2020–2023. Figure 8 shows 19 conferences, 15 journals, 11 arXiv. The authors state that publications increased from 3 to 33 between 2024 and 2025.
• Attack summary: In Table 4, the highest ASR of ASA on Gemma-3-4B-Instruct is 73.8%; ShieldLM's unsafe detection rate goes from 71.9% to 3.5%; the score inflation from 1% poisoning is up to 3× (1.4→4.6/5). The text also states that JudgeDeceiver achieves an average ASR of 89%–99% on four open-source LLMs. These are cited results, not rerun by the authors.
• Defenses and evaluation: In Table 5, J-Detector accuracy ≈85%–95%, GPTracker flags 2,051 violating GPTs out of more than 750,000 instances, with precision ≈73%. In Table 6, BiasScope has an error rate above 50% on JudgeBench-Pro.
• Authors' conclusion: Existing judges are vulnerable to adversarial manipulation, and standardized evaluation methods and robust benchmarks are lacking. The follow-up directions the authors list are robustness benchmarks, defenses against manipulation, and combining automatic judges with human oversight.