DAPRO: Dynamic budget allocation for the number of rounds needed for jailbreaks in multi-round LLM evaluation

1. 01What problem does this paper attempt to solve?

   Key events in multi-turn interaction often occur very late, and the authors use DAPRO to construct time-to-event prediction intervals within the expected budget.

   Key events in multi-turn interaction often occur very late, and the authors aim to construct prediction intervals with coverage guarantees for the number of turns until an event occurs (time-to-event) within an expected budget.

   • Scenario: The authors state that systems are shifting from single-shot prediction to sequential interaction. Jailbreaks, toxic outputs, hallucinations, or agents completing tasks may require many turns, and each turn is expensive. They ask not whether an event will eventually occur, but how many turns it takes.
   • Limitations of existing approaches: The authors state that [7] can give a lower prediction bound for time-to-unsafe (LPB), but the review time is fixed before interaction and is not updated as the conversation unfolds; when an event ends early, that sample's remaining budget cannot be allocated to other samples. The authors also state that calibrating directly with partially observed event times empirically undercovers.
   • Difficulty of dynamic allocation: Individual sequences cannot communicate with each other to decide whether to stop or continue, as that would break the independence used by the theory; online updating of review time must still satisfy the shared expected budget.
   • This paper proposes: Dynamic Allocation via PRojected Optimization (DAPRO). The authors state that it satisfies the expected budget, the coverage guarantee does not require review time and event time to be conditionally independent given the prompt, and it can unbiasedly estimate group metrics such as jailbreak rate on a fixed benchmark.
   • Evaluation setup: The target's initial prompt is generated by an attacker or agent, followed by multi-turn interaction with the target model; an audit function determines the event. Calibration and testing assume i.i.d., with the attacker or agent, target model, and audit fixed. The budget is that the expected number of interaction turns does not exceed B. At test time, only the initial prompt is used to produce the interval, without calling the model again; the authors' stated reason is that the test budget may be unavailable, and actions already taken by an autonomous system may be hard to retract.

   Collapse this answer

2. 02What related research exists?

   Related work includes conformal survival analysis, the static allocation of [7], and methods for active learning and reducing evaluation cost.

   Grouped by the introduction, Section 2, and Appendix A.

   Conformal prediction and survival analysis

   • Conformal prediction (Vovk et al., 2005; Shafer and Vovk, 2008), split conformal (Papadopoulos et al., 2002), and covariate shift weighting (Tibshirani et al., 2019): construct finite-sample prediction sets under exchangeability. The paper cites them as background and does not criticize them individually.
   • Conformal survival analysis (Candès et al., 2023; Gui et al., 2024, and [41, 42] in the paper): use inverse censoring weighting to handle right censoring; [42] also uses longitudinal covariates for history-aware prediction sets. The authors state that existing PAC-type relaxations scale with the worst-case weight and rely on review time and event time being conditionally independent given covariates.
   • Davidov, Feldman et al. (AISTATS 2026) [7]: propose time-to-unsafe-sampling, as well as budget allocation fixed before interaction and minimizing variance. The authors state that when an event ends early, that sample's remaining budget cannot be reallocated.

   Active learning and experimental design

   • Settles (2010), Fedorov (2013): study how to allocate a limited budget across observations to reduce estimation variance.
   • Zrnic and Candès (2024), Hahn et al. (2011): actively collect while maintaining inferential guarantees, or use propensity scores for adaptive experiments.
   • Wang et al. (2026): adaptive experiments for censored survival outcomes. The authors state that they allocate budget across participants to estimate causal effects, whereas this paper decides how far to roll within each trajectory.

   Safety evaluation, red teaming, and cost-reducing evaluation

   • Detection and guardrails include RealToxicityPrompts, SimpleSafetyTests, Llama Guard, NeMo Guardrails. Red-teaming work includes PAIR, Perez et al. (2022), Zou et al. (2023), Wei et al. (2023), and TAP. The authors state that these attack works aim to increase attack success rate, not to estimate censored time-to-event under resource constraints.
   • Kumar et al. (2023) and SmoothLLM use techniques such as randomized smoothing to upper-bound the failure probability of a single prompt. The authors state that this paper exploits the multi-turn structure to give statistical guarantees for time-to-event.
   • Work on reducing evaluation cost includes active testing (Kossen et al., 2021; Li et al., 2024), multi-armed bandits (Zhou et al., 2025), adaptive partitioning (Wang et al., 2025), and finite-population inference (Wu et al., 2026). The authors state that they choose which fixed items to evaluate, whereas this paper decides how far to roll each sequential trajectory and handles the resulting censoring.

   Baselines and benchmarks

   • Baselines are the uncalibrated raw quantile and the static optimized allocation of [7]. In group metric experiments, uncalibrated refers to a naive unweighted estimate under uniform budget; there is also an Oracle with infinite budget as a reference.
   • Benchmarks are RealToxicityPrompts, Anthropic Red-Team, AutoIF; the appendix also includes RAG hallucination data based on SQuAD v2. Multi-turn interaction is generated with PAIR.

   The authors position this paper as: extending [7] to the case where review time is updated along the trajectory, removing the conditional independence assumption, and using average inverse probability weights that act only on a subset of trajectories to tighten the coverage gap.

   Collapse this answer

3. 03How does the paper address this problem?

   DAPRO first learns continuation probabilities, then uses optional CRC to control the expected budget, and then adaptively collects and calibrates prediction intervals.

   Overall, DAPRO treats budget allocation as a sequential decision: first learn whether to continue or stop on a portion of the calibration trajectories, then collect on the remaining trajectories according to that policy, and finally calibrate the prediction interval for time-to-event using the known censoring mechanism.

   Problem setup and inherited calibration

   • The uncensored time is the round at which the event first occurs, and the censored time is the smaller of it and the censoring time. The expected budget is 𝔼[∑_i T̃_i]≤ B, i.e., the expected total number of interaction rounds does not exceed B.
   • The prediction model provides quantile estimates in advance, truncated to not exceed M. The authors follow [7]: use inverse probability weights to correct the distribution shift caused by censoring, then take the largest level whose miscoverage estimate does not exceed α. This level is τ̂=sup{τ∈𝒯:sup_τ′≤τα̂(τ′)≤α}.
   • At allocation time the calibrated level is not yet known, so a fixed prior quantile is used, and it is assumed that the calibrated level falls between 0 and this prior. At test time the lower bound is only a function of the initial prompt.
   • The lower bound is used for unsafe events: they are unlikely to occur before the bound. The upper bound is used for task completion: by that bound it is likely to have already occurred. Appendix B records "not occurring by the maximum round" as a separate category and treats it as infinity.

   Phase I: learning the acquisition policy

   • The calibration indices are randomly split into a learning set, a conformal risk control (CRC) set, and a deployment set. In the learning phase, the first two groups are observed until the event or the prior quantile. The total budget must be sufficient to complete this phase.
   • The score maps the trajectory up to the current round to a real number; the higher it is, the more worth spending another round. The implementation uses the estimated probability of the event occurring at the current step. The score is monotonically mapped to a continuation probability, followed by Bernoulli sampling.
   • What is optimized is a surrogate for the weighted miscoverage variance. Because the calibrated level is not known until after collection ends, the authors use the target miscoverage level α rather than the more conservative prior quantile. The 0-1 indicator is unstable on small samples, so the estimated probability is used as a smooth surrogate, with a small regularization added so that trajectories outside the target interval also provide signal.
   • The constraint is that the expected cost of these samples does not exceed the remaining budget per sample for the subsequent phase, and that a higher score does not yield a lower continuation probability. The appendix solves this with two-bin binning and dual search on the log probabilities, and restricts the cumulative arrival probability to be above a positive lower bound.

   Budget calibration and deployment

   • The optional CRC multiplies the original mapping by a coefficient between 0 and 1, so that the expected cost is nondecreasing in the coefficient. Before using the CRC samples, the conditional expected cost of each trajectory is truncated to an upper bound to avoid the policy being pushed to be overly conservative. The chosen coefficient is the largest one for which the CRC upper bound still falls within the remaining budget.
   • When CRC is not enabled, the original mapping is deployed directly. The authors state that there is no finite-sample budget guarantee in this case; the guarantee depends on the mapping generalizing.
   • The deployment set progressively computes the score, continuation probability, and Bernoulli decision. When the event or the prior quantile is reached, the censoring time is set to the prior quantile; otherwise, it is the last reached step. For samples reaching the target length, the weight is the reciprocal of the product of the continuation probabilities at each step. The learning set and CRC set have weight 1.
   • The calibrated level is then selected using Algorithm 1 in the appendix. The default parameters for the LPB experiments include two-bin binning, regularization of 10 to the power of negative 3, an arrival probability lower bound of 0.005; the CRC cost upper bound is the smaller of the maximum round and twice the average budget.

   Guarantees given by the authors

   • Under i.i.d., quantiles nondecreasing in level, and an upper bound on the average inverse probability weight of "event earlier than the prior quantile" on the deployment set, the authors state that with probability at least 1 minus the tolerance, the conditional coverage is at least the target coverage minus a gap. The gap varies with the reciprocal of the calibration sample size and the square root of that average weight, rather than with the maximum weight over all samples.
   • The authors state that this result does not require censoring time and event time to be conditionally independent given the prompt; DAPRO stops early using the true event time, which would inherently violate this assumption. The authors state that the same technique can also be used for the static procedure in [7].
   • When the CRC feasible set is nonempty, the authors state that the expected total cost does not exceed the predetermined budget. The expected budget conclusion without CRC is written in Appendix C.3 and requires a mapping generalization assumption.

   Collapse this answer

4. 04What experiments did the paper conduct?

   The authors claim that on Toxicity averaged over four models, DAPRO's coverage deviation is 1.90× lower than static and variance is 3.32× lower.

   Experimental setup

   • Data: RealToxicityPrompts, Anthropic Red-Team, and AutoIF each with 10,000 items; the appendix hallucination data comes from SQuAD v2, also 10,000 items. Splits are training 4,000, calibration 3,000, test 3,000.
   • Models and generation: The attacker or helper is Qwen 2.5 14B Instruct, and interactions are generated by PAIR. Target models are Qwen 2.5 14B Instruct, Llama 3.1 8B Instruct, Phi 4 Mini Instruct, and Gemma 3 4B Instruct. The prediction model is a transformer trained on Multilingual-E5-large embeddings, shared across methods.
   • Events: Toxicity uses Detoxify with threshold 0.5. The main-text Red-Team uses Llama Guard. AutoIF is checked programmatically for whether all instructions are satisfied. The appendix hallucination is scored 1 to 10 by Qwen 2.5, with 10 counted as hallucination. When review fails, the score is recorded as 0 and the process continues.
   • Protocol: Maximum rounds and quantile truncation are both 200. The prior quantile for LPB is 0.56 and for UPB is 0.98. The main experiments observe the maximum number of rounds in training dialogues. Random splits for calibration and test are repeated 50 times. LPB searches 1,000 log-spaced candidates; UPB searches 3,000 linearly spaced candidates.
   • Comparisons: The uncalibrated target-level quantile and the static optimized allocation of [7]. The appendix also reports DAPRO without CRC and an Oracle with unlimited budget.
   • Scale and metrics: In the LPB experiment of Appendix F.1, the learning set and CRC set are 25 each; the metric estimation in F.2 uses 50 each. Metrics include absolute deviation from target coverage, coverage variance, per-sample budget, event rate, and restricted mean time.

   Main results

   The main text does not list coverage values per model. The table below only includes numbers stated in the main text.

   Swipe horizontally to read the table

   Metric	Condition	Reported value	Source
   Absolute coverage deviation	Toxicity, averaged over four target models, target 90%, per-sample budget 20	Authors report DAPRO is 1.90× lower than static	Section 4, Figure 2
   Coverage variance	Same as above	Authors report DAPRO is 3.32× lower than static	Section 4, Figure 2
   Event rate estimation variance	Red-Team, averaged, per-sample budget 20	Authors report DAPRO is about 72% lower	Section 4, Figure 4
   Lowest event rate estimate	Red-Team, Phi-4 Mini, full 79.6%, per-sample budget 20	Static 66.6%; DAPRO 73.7%	Section 4, Figure 4

   • The authors state that in Figure 2 DAPRO satisfies the expected budget and has lower coverage deviation than the comparison methods. Coverage rates and budget values per model are not given in the main text.
   • The authors state that in Figure 4 both allocations' average estimates are close to the full event rate; the lower variance reduces the risk of underestimating the event rate and thus making the model appear safer than it actually is. The main text describes the Red-Team review as Llama Guard.
   • Figure 3 for AutoIF is only for Qwen 2.5: lower bound target 90% with per-sample budget 10, upper bound target 80% with per-sample budget 20. The authors state that both bounds are closer to nominal coverage. The 80% target was chosen because 90% would push the upper bound close to 200, which the authors state would artificially shrink the variance difference between methods. The raw deviation values for this figure are not given in the main text.

   Population metric estimation

   • What was measured: Appendix F.2 estimates the event rate across five data and review configurations, as well as the mean time truncated at 200 rounds. Per-sample budget is 20, using Horvitz–Thompson reweighting with reach probabilities.
   • Results: The authors state that correctly reweighted methods align in expectation with the full values; DAPRO with CRC has lower empirical variance than Static on both metrics. The authors state that the uncalibrated estimate in the Toxicity figure is below the true metric. Apart from the Red-Team numbers already listed in the main results table, the appendix does not give raw estimates for the other configurations in text.
   • Authors' interpretation: Metrics on a fixed benchmark are computed directly on that benchmark and do not require extrapolating conclusions to unseen prompts. In Appendix F.4.4, uniform continuation probability with CRC sees more events, but the event rate variance is still higher than DAPRO and static; the authors conclude from this that the number of events itself does not imply low estimation variance.

   Coverage gap

   • What was measured: Appendix F.3 constructs 90% lower bounds with DAPRO with CRC on Toxicity, with per-sample budgets of 5, 10, 20, 30, 40, 50, each with 50 seeds and tolerance 0.05.
   • Results: The authors state that the existing gap is about 894 to 533 percentage points, while the proposed gap is about 24.1 to 5.0 percentage points, roughly 37–110× smaller.
   • Authors' interpretation: The authors state that the existing gap is uninformative at all tested budgets; the new gap remains usable at the smallest budget. The authors also state that the gap can be even tighter if the quantile in the definition is below the prior quantile.

   Attacker model shift

   • What was measured: Appendix F.4.11 learns and calibrates on trajectories where Qwen 2.5 14B Instruct is the attacker, and evaluates on data where Gemma 3 12B IT is the attacker; the target model is Llama 3.1 8B Instruct. Per-sample budgets for Red-Team and Toxicity are both 10. The Red-Team reviewer is the same as the attacker, so both change together.
   • Results: The authors state that DAPRO and static have coverage of 94–97% rather than 90%; DAPRO has lower coverage variance in both settings, observes more events, and does not exceed the target budget. The paper does not give coverage rates per setting.
   • Authors' interpretation: In this case calibration and test are no longer i.i.d., and the theory does not guarantee nominal coverage. The authors state that the learned allocation is still statistically stable, but the shifted lower bound is more conservative.

   Other ablations and analyses

   • Phase I scale (Figure 21): The authors state that at the tested learning-set-plus-CRC-set sizes, coverage is 90% and per-sample budget does not exceed 20; as size increases, usage is closer to the target and the average weight increases.
   • Available budget (Figure 22): The authors state that at low budget static has high coverage variance; DAPRO has empirical coverage around 90% at the tested budgets, and the average weight decreases as budget increases.
   • Training budget (Figure 31): The authors state that with a per-sample training budget of 5, DAPRO variance is still lower than Static; at this budget, temperature scaling raises test negative log-likelihood from 1.47 to 2.37, with about 17 validation trajectories.
   • Optimization objective (Figure 23): The authors state that soft-prefix observes the most events and has the lowest weighted error and coverage variance; soft-terminal is worse. The main text does not give specific variance values.
   • Score noise and mapping capacity (Figures 25–28): The authors state that moderate noise can be absorbed by the two-bin mapping and CRC, while fully random scores degrade performance; with CRC, 8 bins give the lowest event rate variance, and without CRC, 2 bins are most stable.
   • CRC spending upper bound and miscoverage estimation (Figures 29–30): The authors state that too small an upper bound leaves budget unused, while too large a bound loosens the finite-sample correction; the default is twice the average budget. For UPB, the authors state that the sequential augmentation estimate has the lowest variance; this experiment targets 80% coverage with per-sample budget 20.

   Collapse this answer

5. 05What are some points that can be further explored?

   The authors point out that CRC often leaves budget unused, τ̂ does not vary with the test prompt, and the coverage guarantee also relies on identical distribution.

   Limitations and future directions noted by the authors

   • The CRC version often does not use the full budget in experiments; when finite-sample budget guarantees are not needed, CRC can be removed. The authors state that without CRC the budget is generally still satisfied in experiments, although the theory requires the mapping to be accurate (Section 5).
   • The authors call deterministic rather than expected budget control an important future direction and state they are working on it (Section 5).
   • The current calibrated quantile level is a scalar that is the same for all test prompts. The authors propose making it adaptive to the test prompt, tighter for predictable interactions and more conservative for high-variance interactions (Section 5).
   • Another direction is designing scores that better represent safety or utility, so as to map to continuation probabilities with lower variance (Section 5).
   • The coverage guarantee relies on calibration and test being i.i.d. The authors state that model updates or new attack strategies cause distribution shift, and robustness to distribution shift or audit labeling errors is future work. Metric estimation on a fixed benchmark does not need this i.i.d. assumption (Section 5).

   Experimental coverage

   • Target models are Qwen 2.5 14B Instruct, Llama 3.1 8B Instruct, Phi 4 Mini Instruct, and Gemma 3 4B Instruct, 4 in total. The attacker or helper in the main experiments is Qwen 2.5 14B Instruct, and interactions are generated by PAIR.
   • Task data are RealToxicityPrompts, Anthropic Red-Team, and AutoIF, each with 10,000 items, plus hallucination data based on SQuAD v2 in the appendix with 10,000 items. Training, calibration, and test splits are 4,000, 3,000, and 3,000, with 50 splits per setting and a maximum of 200 rounds.
   • Event determination includes Detoxify threshold 0.5, Llama Guard, and programmatic verification; the appendix also has a 1-to-10 score review by Qwen 2.5. Comparisons are against the uncalibrated quantile and the static allocation of [7], and lower bounds, upper bounds, and population metrics are all reported.
   • The LPB experiment in Appendix F.1 sets the learning set and CRC set to 25 each; the metric estimation in F.2 uses 50 each. Appendix F.4.10 also varies the per-sample budget in the training phase.
   • Appendix F.4.11 changes the attacker from Qwen 2.5 14B Instruct to Gemma 3 12B IT. The main text does not give the coverage rates and budget values per model for Figure 2 and Figure 3.

   Collapse this answer

6. 06Summarize the main content of the paper

   DAPRO dynamically allocates budget within the expected budget for multi-turn interactions and gives finite-sample coverage bounds for time-to-event.

   DAPRO is a statistical method for constructing time-to-event prediction intervals for multi-turn LLM evaluation under an expected interaction budget.

   • Key events may occur very late. Static review time is fixed before interaction; when an event ends early, the remaining budget cannot be transferred to other samples.
   • Calibration data is split into learning, optional conformal risk control, and deployment. Trajectory scores are mapped to continuation probabilities, Bernoulli decisions determine whether to interact further, and inverse probability weights are then used to calibrate quantiles. Testing only looks at the initial prompt.
   • On Toxicity, with target coverage 90% and per-sample budget 20, the authors state that averaged over four target models, DAPRO's coverage absolute deviation is 1.90x lower than static, coverage variance is 3.32x lower, and the expected budget is not exceeded. Raw values per model are not listed in the text.
   • In Red-Team event rate estimation, the authors state that DAPRO's variance is on average about 72% lower. Phi-4 Mini's full event rate is 79.6%, and the lowest estimate is 66.6% for static and 73.7% for DAPRO.
   • On AutoIF, the authors state that Qwen 2.5's lower and upper bounds are both closer to their respective nominal coverage. The appendix also includes RAG hallucination; when the attack model is changed, the authors state coverage is 94–97%, above the nominal 90%.
   • The authors conclude that online updating of review time can tighten finite-sample coverage gaps within the expected budget and reduce variance in group event rate estimation. Coverage extrapolation to unseen prompts still relies on exchangeability, and deterministic budget control is listed by the authors as future work.

   Collapse this answer

Related papers