REPRO: Reflective Evolutionary Prompt Optimizer for Automated LLM Red Teaming

1. 01What problem does this paper attempt to solve?

   Under pure black-box single-turn, automatically search for jailbreak prompts using algorithmic structure rather than a strong auxiliary model.

   How to automatically search for adversarial prompts that bypass LLM safety guardrails under pure black-box access, no per-model handcrafted templates, and a single consumer GPU.

   • Deployment scenario: the authors state that LLMs have entered software pipelines such as customer service, diagnosis, code generation, and education; despite RLHF and constitutional AI, adversarial prompts can still elicit harmful content. The authors believe manual red-teaming cannot keep up with changing threats, and automated, reproducible safety testing is needed before deployment.
   • Existing shortcomings: the authors state that template matching cannot stop context-aware defenses, and gradient methods cannot be used on proprietary APIs. Andriushchenko et al. require logprobs and per-model templates; the introduction writes roughly 10,000 target queries per behavior, and later text writes up to 10,000. PAIR and TAP have no population diversity; other methods either require privileged access or rely on stronger auxiliary models.
   • Design idea: optimization intelligence should come from algorithmic structure rather than auxiliary model capability, so the pipeline needs only a quantized open-source model plus a single consumer GPU.
   • This paper proposes: REPRO (Reflective Evolutionary Prompt Optimization). The authors state that semantic crossover, and Thompson Sampling informed by structured failure analysis, are absent from prior work; and they formulate a bi-level multi-objective problem over attack success, cross-environment transfer (XFER), detectability, and length. The experiments in this paper exercise only attack success.
   • Threat model:
     • Goal: bypass safety guardrails and elicit harmful content.
     • Knowledge: pure black-box, only text outputs, no gradients, no logprobs.
     • Capability: single-turn; the attacker submits an attack scaffold, which the reflection model mutates and recombines. The authors place multi-turn attacks such as Crescendo and indirect prompt injection outside this scope.
     • Victim system and judgment: an aligned target LLM (local or production API). The judge labels the concatenated output as harmful or refusal; search stops when the judge accepts, so ASR indicates whether a search was found within the iteration budget, not a fixed-budget success rate.

   Collapse this answer

2. 02What related research exists?

   Related work is divided into software testing, jailbreaks, prompt optimization, and framework comparisons; baseline numbers are all transcribed from the original papers.

   The paper discusses related work in four groups; the baseline numbers in the experiments are all transcribed from the original sources, not rerun by the authors.

   LLM safety in software testing

   • Testing surveys: Wang et al. (2024) list adversarial robustness evaluation as an open problem; Chen et al. (2025) survey attacks and defenses for code language models. Jiang et al. (2024) and Sun et al. (2026) discuss LLM-assisted fuzzing.
   • Defenses and the authors' prior work: JailGuard by Zhang et al. (2025) uses TAP as a baseline attack. The authors state that REPRO provides offensive test generation, and the attacks found can serve as regression corpora for defenses. The authors view their prior faithfulness red-teaming work as reproducible test generation and build a general framework on that basis.

   Jailbreak attacks

   • White-box and gray-box: GCG (Zou et al., 2023) achieves ASR 80%–95% on open-weight targets and requires gradients. AutoDAN (Liu et al., 2024) reports 72%–98% and requires gradients. Adaptive random search (Andriushchenko et al., 2025) is called by the authors the current state of the art in raw effectiveness, with ASR 100%, but requires logprobs, per-model templates, and up to 10,000 queries per behavior.
   • Black-box iterative: PAIR (Chao et al., 2023) and TAP (Mehrotra et al., 2023). The authors state that both are effective on Vicuna and GPT-3.5, TAP reports over 80% on GPT-4/GPT-4-Turbo, but performance drops on Llama-2-Chat and Claude (see Table 1); both optimize only ASR and have no population diversity.
   • Population and technique libraries: GPTFUZZER 75%–88%. AutoDAN-Turbo reaches up to 97.6% on individual HarmBench targets; the authors state its strategy library is closest to Bayesian strategy selection, but it lacks multi-parent crossover and structured failure analysis. KDA reaches up to 100% on Vicuna and Mistral in HarmBench; DAGR up to 99.33%. AmpleGCG reports transfer to closed-source models after white-box training, which the authors say motivated setting XFER as a first-class objective. MasterKey focuses on cross-chatbot transfer but has no multi-objective optimization. Wei et al.'s six failure modes are used as the organizing principle for strategy families; several published techniques are folded into the mutation library rather than being separate baselines. Crescendo and indirect prompt injection are placed outside the single-turn scope.

   Prompt optimization and orchestration frameworks

   • Usefulness optimization: DSPy/MIPROv2 (Khattab et al., 2024) uses Bayesian optimization to select instructions; GEPA (Agrawal et al., 2025) uses natural-language reflection and Pareto selection. The authors state that both target usefulness objectives.
   • OpenRT: Wang et al. (2026) integrates 37 attacks and evaluates 20 frontier models; EvoSynth achieves ASR 100% on 18/20 models. The authors state that OpenRT orchestrates existing methods, REPRO discovers single-turn attacks on its own, and the model sets on the two sides differ, so direct comparison is not feasible.

   Baselines and positioning

   • Baseline methods: PAIR, TAP, PAP, Adaptive RS; the JailbreakBench and HarmBench tables also include GCG, AutoDAN, AutoDAN-Turbo, KDA, DAGR, Human, and JB-Chat from the original papers. Benchmarks: AdvBench (50), JBB-Behaviors (100), HarmBench (200 standard behaviors).
   • Authors' positioning: In Table 1 the authors state that REPRO is the only entry in that comparison that simultaneously has pure black-box access, population search with crossover, Bayesian strategy selection, and multi-objective criteria. The authors state that they inherit population search, Pareto, and reflection, switch to adversarial objectives, and add semantic crossover, Thompson Sampling, and refusal-classification feedback; in the comparison they compiled, no other single-turn black-box method reports consistent 100% on a comparably broad model set.

   Collapse this answer

3. 03How does the paper address this problem?

   REPRO performs pure black-box single-turn search with population evolution, semantic crossover, and Thompson Sampling.

   REPRO frames red-teaming as closed-loop test generation: target responses are judged by a separate judge acting as a test oracle, and failure information then drives evolutionary prompt search. The default reflection model is Mistral-7B (EXL3, 4 bpw); GPT-4.1, GPT-5-nano, Qwen3-235B, and Gemini-2.5-Pro switch to Mistral-3.1-Small (EXL3, 4 bpw).

   Problem setting and formalization

   Given a target M, a set of harmful behaviors T, and a set of deployment environments E. Attack success is indicated as σ(x,t,e)=I[J(M(x ⊕ t ⊕ e))=harmful], i.e., the judge J labels the output of the sequentially concatenated system prompt, attack scaffold, and task as harmful.

   • Four objectives: ASR takes the max over environments and then averages over tasks; XFER is the success proportion over all task–environment pairs; detectability is the count of adversarial marker words; length is the number of tokens. The first two are maximized, the last two minimized. The Pareto set is the set of prompts not dominated by other prompts.
   • Three environments: empty system prompt, minimal assistant instruction, strict safety guideline. Experiments use only the provider-default environment, in which case XFER degenerates to ASR. The authors state multi-environment evaluation is implemented but left for future work.

   Evolutionary loop

   Starting from a single seed, the reflection model bootstraps an initial population of size N at temperature 0.9; per-behavior default N=24, multi-behavior coverage uses N=18. Each generation has seven steps: full evaluation, updating strategy and context-example statistics, epsilon-Pareto parent selection, strategy-based mutation, semantic crossover, self-refinement, and population selection with elitism. Stagnation management can restart or early-stop. The authors state the algorithm provides structured feedback and the reflection model only performs text transformations.

   Failure analysis and strategy selection

   On failure, refusals are classified into six categories: Ethical, Safety, Capability, Deflection, Hard refusal, Confusion. Each category maps to an ordered set of countermeasures, and the tried strategies and their success/failure ratios for that behavior are recorded; Table 2 gives the mapping. A stubborn behavior is defined as having a failure rate above 0.7 after at least 3 attempts, and then switches to an unexhausted strategy family. The authors state in advance in the methods section that the 100-generation-budget ablation did not isolate a measurable ASR benefit for this component, so they label it a design contribution with unquantified empirical contribution.

   • Strategy library: Grows from 37 handcrafted strategies in the literature to 58 strategies across 14 semantic families, of which 21 are synthesized by the reflection model based on failure analysis and retained across runs. Success is defined as ASR or XFER above 0.5, used to update a Beta posterior.
   • Mixed sampling: Weights are w_s=αp̂_s(1+c̄_s)+β r_s+γ√(log N_total/N_s), i.e., historical success rate, keyword Jaccard with the current failure analysis, and a UCB exploration term. Default weights are (0.4, 0.4, 0.2). Mutation prompts are placed only into the top K=8 by mixed score, and those with relevance below 0.3 are excluded.

   Generation, crossover, and exploration control

   • Parents and crossover: Parent selection uses ε=0.05, frontier at least 3 and at most 6; at termination ε=0.03. When the frontier is insufficient, it is filled lexicographically by ASR, XFER, negative detectability, and negative length. Semantic crossover performs fitness-weighted sampling over 2–3 parents, with the reflection model merging complementary techniques; with only one parent, self-crossover is used.
   • Pre-evaluation filtering: Self-critique (temperature 0.3) returns keep, refine, or reject, followed by branch expansion, so each generation sends exactly N candidates to the target. Elite count E=3, which the authors state keeps the best ASR so far non-decreasing across generations. Near-failures additionally receive a continuous soft score with a fitness bonus capped at 0.1 (Appendix B).
   • Stagnation and caching: Diversity uses average pairwise cosine distance of sentence vectors to adjust mutation rate and number of offspring; there are also 15 structural seeds, and on restart 25% of new seeds come from these templates. Stagnation windows are 20/15/10 by ASR, with at most 3 restarts. Successful samples enter an amortized cache; new runs may use at most 8 of them as initial candidates. Default values in Table 3 come from engineering debugging, and the authors state no grid search was performed.

   Two-level formulation and query cost

   The upper level maximizes the hypervolume of the population's Pareto frontier, and the lower level selects strategies outside the task taboo list. The authors write elitist monotonicity, Thompson Sampling's sublinear regret, and Pareto approximation under operator coverage as design properties, and state that the classical assumptions of the latter two do not strictly hold under non-stationarity and growing action spaces. Worst-case target evaluations are generations × N × tasks × environments. In per-behavior mode, the initial population costs N target queries, then N−E per generation; the reflection model's mutation and refinement are not counted as target queries.

   Collapse this answer

4. 04What experiments did the paper conduct?

   The authors state that under the protocol, single-turn ASR is 100% across 36 targets, with a median of 24 queries; baselines were not rerun.

   Under a pure black-box until-success protocol, the authors report ASR of 100% across three benchmarks and 36 targets, while also reporting queries, ablations, and judge validation.

   Experimental setup

   • Target models: 36 models across 11 families (Tables 4, 11): Llama-2-7B/70B (Table 11 writes Llama-2-7B-Chat, Llama-2-70B-Chat), Llama-3.1-8B/70B/405B, Llama-3.3-70B; Qwen-2.5-7B, Qwen3-8B, Qwen3-235B; GPT-3.5-Turbo, GPT-4o-mini, GPT-4o, GPT-4.1, GPT-5-nano; Claude-3-Haiku, Claude-3.5-Sonnet; Gemini-2.0-Flash, Gemini-2.5-Flash, Gemini-2.5-Pro; Grok-2-1212, Grok-3-Mini, Grok-4.1; Gemma-2-9B, Gemma-3-4B; Phi-3-Mini, Phi-4; Mistral-7B-v0.3, Mixtral-8x7B, Mistral-3.1-Small; DeepSeek-V3, DeepSeek-R1-Distill-8B (Table 10 writes DS-R1-8B, Table 11 writes DeepSeek-R1-8B); and Vicuna-13B, OpenChat-3.5, Starling-7B, Kimi-K2, GLM-4.5-Air. Local models are EXL3 4-bit, APIs are full precision.
   • Reflection model and judge: default Mistral-7B (EXL3, 4 bpw); the four largest frontier targets use Mistral-3.1-Small. Both optimization and reported ASR use Qwen3-8B, stopping on acceptance. Cross-validation additionally uses GPT-4 and Mistral-Small-3.1-24B-Instruct.
   • Benchmarks: AdvBench 50 behaviors; JailbreakBench 100 behaviors in 10 categories; HarmBench 200 standard behaviors. Ablations use stratified JBB-50 with seed 42.
   • Baselines: PAIR, TAP, PAP, Adaptive RS, plus GCG, AutoDAN, AutoDAN-Turbo, KDA, DAGR, Human, and JB-Chat from the respective original tables. All are transcribed; the authors did not rerun them. PAIR is written by the authors as roughly 20 queries per behavior, Adaptive RS up to 10,000; REPRO runs until success or the iteration cap.
   • Protocol: ASR is the proportion of behaviors that elicit a harmful reply, with Wilson 95% intervals. Experiments use the providers' default environments. Per-behavior N=24, multi-behavior coverage N=18. The strategy library and amortized cache accumulate across runs, so the 36 targets are not independent repetitions; ablations disable the cache and reset strategy weights. The median behavior ends at generation 0, with the longest observed at 266 generations; ablations fix 100 generations.
   • Re-judging scope: For JBB and HarmBench, attacks already accepted by Qwen3 are re-judged with each benchmark's evaluation prompts, yielding 100%. The authors state that they did not use the Llama-3-70B specified by JailbreakBench, nor the HarmBench classifier, and cannot give a search-independent false discovery rate on that basis.

   Main results

   Swipe horizontally to read the table

   Target	Benchmark	REPRO	PAIR	Other transcribed	Adaptive RS
   Llama-2-7B	AdvBench	100%	4%	TAP 0%; DAGR 90.96%	100%
   Vicuna-13B	AdvBench	100%	72%	TAP 100%	100%
   GPT-3.5	AdvBench	100%	44%	PAP 86%	100%
   GPT-4o (baseline is GPT-4)	AdvBench	100%	39%	PAP 88%	100%
   Llama-2-7B	JBB	100%	0%	GCG 3%	90%
   GPT-4o (baseline GPT-4-0125-Preview)	JBB	100%	34%	GCG transfer 4%	78%
   Llama-2-7B	HarmBench	100%	7.5%	DAGR 91.83%	not reported
   GPT-4o (baseline GPT-4 0613)	HarmBench	100%	38.5%	AutoDAN-Turbo 87.8%	not reported

   Per Tables 8, 9, 10. REPRO is 50/50 on AdvBench, 100/100 on JBB, and 200/200 on HarmBench.

   • The authors state that 100% means every behavior within budget was accepted by the in-loop judge, not a success rate under a fixed query budget, and not a ranking against transcribed baselines. The Wilson intervals are interpreted by the authors as sampling uncertainty over the behavior set: AdvBench [92.9, 100.0], JBB [96.3, 100.0], HarmBench [98.1, 100.0].
   • The authors state that the baselines differ in judge, budget, and model version. Table 10 also notes: the Vicuna baseline row for HarmBench is Vicuna-7B, while REPRO uses Vicuna-13B; KDA's GPT-3.5/GPT-4 use Turbo 1106, while the HarmBench baseline uses 0613.
   • No main-table entry in the paper has REPRO ASR below 100%, so the table above does not select a separate "worst REPRO." Omitted rows are also 100% (50/50) on AdvBench-50 in Table 11. On HarmBench, KDA is 100% for Mistral-7B, DAGR 99.33%, and REPRO 100%; the authors state that the consistent 100% refers to all evaluated targets, not that REPRO is the only method reaching 100% on each individual target.

   Target query cost

   • Table 12 covers 211 behaviors with retained logs across 5 targets, not all runs behind the ASR tables. Overall mean 161.1, median 24, min 16, max 5589, longest 266 generations. The authors state that some early runs had population 16, so the minimum query count can be 16; with the default initial population, success is exactly 24.
   • gpt-4o-mini on the AdvBench-50 subset (N=27) has median 24, mean 231.1, max 5589; on JBB-50 (N=50) median 66, mean 410.8, max 2103. grok-3-mini (N=20) is 24 queries each, at most 1 generation. grok-4-1-fast (N=24) has median 24, max 465.
   • The authors state that over half of behaviors are solved by the initial population, and relative to gray-box up to 10,000 queries, the median is roughly 417 times fewer and the worst case roughly 1.8 times fewer. The authors also state that most runs are warm-started by the cache, and cold-start is not reported.

   Component ablations

   • JBB-50, up to 100 generations, cache off, seed 42 once; targets GPT-4o-mini and Llama-3.3-70B-Instruct; reflection model Mistral-7B, judge Qwen3-8B (Table 14).
   • Full REPRO: GPT-4o-mini 100% (50/50, Wilson [92.9, 100.0], median 3 generations), Llama-3.3-70B 98% (49/50, [89.5, 99.6], median 3 generations). The authors state that 98% comes from the 100-generation cap, and that target is 100% in the main experiment without that cap. Removing semantic crossover gives 90% on both sides; removing Bayesian strategy selection gives 88% and 90%. The authors' bootstrap ΔASR intervals exclude 0 in these four groups. Removing failure analysis: GPT still 100%, Llama 94%, and both intervals contain 0. Removing elitism: GPT 90% (interval excludes 0), Llama 94% (contains 0).
   • After Bonferroni correction (α/4=0.0125), no individual McNemar test reaches the authors' stated threshold. The authors state that with n=50 the test has power 0, and detecting the observed difference would require roughly 93–140 behaviors. Cochran's Q: GPT-4o-mini 16.57, p=0.002; Llama 4.87, p=0.30. The authors state that failure analysis has no measurable ASR benefit under that budget and makes convergence later on GPT-4o-mini (log-rank χ²=4.53, p=0.033, corrected p=0.133).

   Multi-behavior coverage and judge validation

   • One shared attack is optimized and evaluated on the same 50 AdvBench behaviors (Table 13). Llama-3.1-8B, Qwen-3-8B, and GPT-4o-mini are all 100% (50/50, Wilson [92.9, 100.0]), with 193, 26, and 89 iterations. The authors state the 7-fold difference and attribute it to differences in safety training across families; they also state this is not generalization to held-out behaviors and avoid calling it universal.
   • 365 stratified replies (AdvBench query pool 165, JBB ablation 200, seed 42, 70/30 within strata): three-way raw agreement 80%, Fleiss' κ=0.72; Cohen's κ for Qwen3-8B against GPT-4 and Mistral-24B is 0.67 and 0.81 (Table 5). On disagreements, GPT-4 is more lenient in 54/60 and Mistral-24B in 23/32 (Table 6). The authors state that switching to these two judges would raise rather than lower the reported ASR.
   • grok-4-1-fast (n=33) has three-way agreement 61%, κ=0.37–0.46, which the authors call the noisiest slice in the validation. The gpt-4o-mini AdvBench slice (n=33) has κ=0.93 against both alternative judges; on its JBB slice (n=200), Qwen3-8B against GPT-4 has κ=0.58 (Table 7).

   Other ablations and analyses

   • Reflection model: Mistral-7B and Mistral-3.1-Small (24B) are both 100% on the full AdvBench-50; cross-family reflection models were not done.
   • Figure 3: the authors state that all configurations have similar initial slopes, with differences mainly in the asymptotic ceiling; ablations stop at 88%–94%, the full configuration reaches 98%–100%. The text does not retain per-point coordinates.
   • With failure counted as generation 101, the authors state that all comparisons have Cliff's δ absolute value no greater than 0.16.
   • Table 15 ranks strategies across 11 families by number of successful offspring; the authors state this is a descriptive observation, not a causal claim about the training process, and raw counts that would allow computing absolute success rates are not retained.
   • The authors state that cross-version transfer within a family is only a qualitative observation and was not systematically measured. Table 16 and Figure 4 give short attacks and one DeepSeek-R1-Distill-Llama-8B response pair; the payload is not repeated here.

   Collapse this answer

5. 05What are some points that can be further explored?

   The authors note that the judge is inside the search loop, baseline budgets are not aligned, and multi-objective and cold-start settings have not been evaluated.

   The authors list the in-loop judge, unaligned transcribed baselines, and unevaluated multi-objective settings as the main limitations.

   Limitations and future directions noted by the authors

   • In-loop judge: ASR is produced by the same Qwen3-8B that guides the search, stopping on acceptance, so false acceptances are counted in the results and not rechecked. The authors call a search-independent gold-standard post-hoc false discovery rate, and human annotation on the same sample, the highest-value extensions (§5.2, §5.4).
   • Baselines not rerun: the numbers for PAIR, TAP, PAP, Adaptive RS, AutoDAN-Turbo, KDA, and DAGR come from the original papers, with different judges, model versions, and query budgets. The authors list reruns with the same judge and aligned budgets, and ASR–budget curves, as future work (§4.1.2, §5.2).
   • Unevaluated objectives: only ASR is exercised; the paper has no Pareto frontier, hypervolume, detectability, or length results. The authors call the three-environment protocol in §3.1 the next experiment (§1.2, §5.2).
   • Cross-run state and logs: the strategy library and cache accumulate across runs, and cold-start is not reported. The query log covers only 211 behaviors across 5 targets, and the median of 24 is computed only on that subset (§4.1.5, §5.2).
   • Scope and cost: the current implementation optimizes single-turn and does not explore multi-turn attacks like Crescendo. Input classifiers beyond standard APIs and detectors like JailGuard are not tested; the authors state that the reported ASR should be read as an upper bound relative to multi-layer systems. In the Limitations, the authors list the worst-case 5,589 queries and the continuous inference of the reflection model as computational costs (§5.2).
   • Other explicit follow-ups: multilingual beyond English, multimodal, and domain deployments are untested; ASR does not distinguish harm severity; the ablation is a single run, and the authors recommend more seeds with n≥93; hyperparameter sensitivity, reproduction of the benchmarks' original scorers, and human annotation are left for later. §6.1 states that multi-turn extensions are in progress, evaluation targeting defense deployment has begun, and using the found attacks for adversarial fine-tuning and extending to image-text jailbreaks are listed as directions (§4.4, §5.4, §6.1).

   Experimental coverage

   • The tested targets are the 36 models across 11 families in Tables 4 and 11; benchmarks are AdvBench 50, JailbreakBench 100, HarmBench 200; the protocol is single-turn, and experiments use the providers' default environments.
   • Reflection-model experiments use Mistral-7B, plus Mistral-3.1-Small on the four largest frontier targets. The main-result judge is Qwen3-8B; cross-validation uses 365 replies from 5 targets.
   • The ablation is JBB-50, GPT-4o-mini and Llama-3.3-70B-Instruct, a 100-generation cap, seed 42 once, cache off; the removed components are failure analysis, semantic crossover, Bayesian strategy selection, and elitism.
   • The query-cost table covers 211 behaviors across 5 targets. Multi-behavior coverage uses 3 targets on the same 50 AdvBench behaviors, with the optimization set identical to the evaluation set.
   • The authors state that multi-turn attacks and indirect prompt injection are outside the single-turn threat model and were not directly compared. XFER, detectability, and length are implemented; the paper does not report measurements for these three.

   Collapse this answer

6. 06Summarize the main content of the paper

   REPRO performs pure black-box evolutionary red-teaming with a 7B reflection model; in-protocol ASR on evaluated targets is 100%, which the authors treat as a coverage catalog.

   REPRO is a pure black-box, single-turn automated red-teaming optimizer for LLM software systems, with the default reflection model being 4-bit Mistral-7B.

   • Problem: Aligned models can still be bypassed by adversarial prompts. Gradient methods, random search requiring logprobs, and per-behavior iterative methods either cannot be used on production APIs or require handcrafted templates and high query volumes.
   • Method: Population evolution with multi-parent semantic crossover, failure analysis routed by six categories, and Thompson Sampling for strategy selection. The formal objectives include ASR, XFER, detectability, and length; experiments run only ASR, and stop when the in-loop Qwen3-8B accepts.
   • Coverage: The authors report 100% in-protocol ASR across 36 targets on AdvBench (50/50), JailbreakBench (100/100), and HarmBench (200/200). Wilson interval lower bounds are 92.9%, 96.3%, and 98.1%, respectively. The AdvBench entry in Table 11 is also 100%.
   • Cost and ablations: Median target queries are 24 across 211 behaviors and 5 targets, with a maximum of 5589 (Table 12). On JBB-50, removing semantic crossover or Bayesian strategy selection gives ASR of 88% or 90% (Table 14). Failure analysis shows no separately measurable ASR benefit under a 100-generation budget. Fleiss' κ = 0.72 across three judges on 365 responses, and the authors state Qwen3-8B is not the more lenient party.
   • Authors' conclusions: The authors claim algorithmic structure can push search to the ceiling of this protocol without logits or per-model templates, and treat the results as a coverage catalog rather than a ranking against transcription baselines. The authors also state: the judge is in the loop, baseline budgets are not aligned, and multi-objective and cold-start are not evaluated; these numbers should be read as judge-referenced results under the protocol until success.

   Collapse this answer

Related papers