Multi-turn short horizon: measuring effective horizon compression in jailbreaks

1. 01What problem does this paper attempt to solve?

   The authors ask: to recover later attack progress, how much early query history is still needed?

   After a multi-turn jailbreak reaches later attack progress, how much early query history is still needed to recover that progress?

   • Scenario: the authors state that multi-turn jailbreaks can adjust subsequent queries using the victim model's feedback each turn, and are therefore often more effective, but adaptive search may require many rounds of interaction, which is slow and costly.
   • Gap: once a trajectory is found, it is unclear whether the early queries are still depended on by later queries. M2S rewrites a multi-turn conversation into a new single-turn prompt, which can absorb information from the whole conversation but cannot measure how much of the prefix in the original query sequence can be deleted.
   • Hypothesis: keep the later attacker queries verbatim, delete only the earlier queries, replay in a new conversation and let the victim model regenerate the replies; if the score is not lower than the original progress minus the tolerance, the deleted queries are not necessary for recovering that progress. This does not mean that the deleted turns had no role in discovering the later queries.
   • Threat model:
     • Goal: make the aligned LLM give a harmful reply; the measurement question is whether the attack score already reached later can be recovered from a shorter query suffix.
     • Knowledge: the attacker sees the victim model's reply each turn and revises queries accordingly. The paper does not state white-box, black-box, or whether weights are accessed.
     • Capability: do not rewrite the retained attacker queries, only delete the sequence prefix, and do not reuse the original victim replies.
     • Victim system: the victim LLM in the experiments; the trajectories come from existing multi-turn attacks, not from newly written prompt templates.
   • Proposal: define the effective horizon and its ratio to trajectory length, EHR, call this shortening effective-horizon compression, and use it for continued attack, attacker credit assignment and defense reward windows.

   Collapse this answer

2. 02What related research exists?

   The authors position the work as not rewriting queries and only measuring whether the prefix is still needed.

   Search-based multi-turn attacks

   • Step-by-step unfolding: Crescendo (2025), FITD (2025) and CoA (2024) progressively develop attacks across multiple turns and adjust queries based on victim responses.
   • Planning and collaboration: ActorBreaker (2025), X-Teaming (2025), Trojan Knowledge (2025) and PLAGUE (2025) add planning, exploration or multi-agent coordination. The paper only describes them here and does not critique each one.

   Training-based attacks and rewriting

   • Learning policies: The paper groups SIREN (2025), TROJail (2026) and SEMA (2026) as methods that learn multi-turn attack policies from supervised or reinforcement learning signals.
   • M2S: M2S (2025) rewrites multi-turn conversations into new single-turn jailbreak prompts that can incorporate information from the entire conversation. The authors explicitly contrast: this paper does not rewrite the attacker's queries, only deletes the query history preceding them.

   Multi-turn defenses

   • Cross-turn information: Some methods reason over the accumulated conversation, maintain cross-turn information with graph or recurrent representations, or accumulate risk evidence over time. Those named by the paper include work reasoning over the accumulated conversation, graph or recurrent representations, and methods that accumulate risk evidence.
   • Blocking methods: D-Judge rewrites victim responses to interrupt the attack process; TRACE and TurnGate use trajectory-level reasoning or learned policies to decide when to block. The authors say the defense experiments in this paper do not change the observed conversation, and only study how far the training reward needs to propagate.

   Baselines and benchmarks

   • Comparisons: Effective horizon measurement uses X-Teaming and TROJail. Continuation attack comparisons are the full failed trajectory (Full) and returning to the highest-scoring turn while keeping the entire prefix (Peak), with attack methods X-Teaming and ActorBreaker. Credit assignment comparisons are TROJail's β=1 and β=0. Defense comparisons are TurnGate's full remaining-trajectory return and the same reward-weighted SFT checkpoint.
   • Data: WildJailbreak, HarmBench, AdvBench, StrongREJECT, JailbreakBench, MTID.

   The authors place this paper on the measurement side: keeping the original attacker queries and seeing how much preceding history can be removed; the defense experiments only shorten the reward propagation distance without changing the conversation content.

   Collapse this answer

3. 03How does the paper address this problem?

   The effective horizon is defined by replaying with query suffixes, and is used for continued attack, credit assignment and reward windows.

   On an existing trajectory, delete the early attacker queries, replay the remaining queries in a new conversation and regenerate the victim replies; the shortest suffix that recovers the target-turn progress is the effective horizon. Based on this, EH-CR and EH-CA are proposed, and the future reward window of defense training is truncated.

   Effective horizon

   • Object: for a target turn j in the query sequence q1:T, the original attack score is vj. A candidate length k keeps only the last k queries, and a replay score is obtained in a new conversation.
   • Recovery condition: the replay score is not lower than vj minus ϵ. ϵ is not less than 0, depends on the scoring function, and is specified separately for each experiment. Only the prefix is deleted, and all candidates are contiguous suffixes; the retained queries are not rewritten, and no turns are deleted from the middle.
   • Definition: the effective horizon is the smallest k satisfying the condition, i.e., Heff(q_1:j)=min{k∈{1,…,j}∣ ṽ_j,k≥ v_j−ϵ}. It denotes the number of recent queries needed to recover the progress at turn j. If no suffix satisfies the condition, the effective horizon of that target turn is undefined.
   • Ratio: EHR equals Heff divided by j. A value of 1 means that under that replay criterion all history is needed; a smaller value means more compression. The authors call this effective-horizon compression. A short horizon does not mean that the deleted turns had no role in discovering the later queries.

   Replay measurement

   • Successful trajectories: the target turn is the last turn. X-Teaming and TROJail each use 200 WildJailbreak items, up to five turns; X-Teaming additionally uses 1,500 items for five turns. Appendix A.1.
   • Unsuccessful trajectories: truncate at the first turn that reaches the highest attack score, then measure the effective horizon at that point.
   • Scoring: X-Teaming uses a GPT-4o 5-point jailbreak judge, where 5 means success. TROJail uses the HarmBench classifier, where above 0.9 means success. Appendix A.1 does not give the ϵ value for this measurement, nor the number of repetitions.

   EH-CR

   • Steps: on an unsuccessful trajectory, take the highest-scoring turn. Apply the same shortest-suffix criterion to the history before that turn, replay in a new conversation and continue the attack. If no shorter suffix can recover it, keep the full prefix up to that turn.
   • Remove two kinds of history: turns after the highest score that take the attack away from that progress; and earlier queries no longer needed to recover that progress.
   • Tolerance: continued attack uses the discrete 5-point scale with ϵ=0, and the recovered context must reach the same judge score as the originally selected point. Appendix A.2. The authors state that it is context selection, not a method for reducing the number of queries.

   EH-CA

   • What is changed: TROJail uses the final attack outcome as the learning signal for each attacker query. EH-CA changes only this assignment. The first query within the horizon and those after it keep the original signal; earlier queries are multiplied by β, where β is between 0 and 1.
   • Boundaries: β=1 is the original assignment; β=0 removes that signal before the horizon. Other credit components are unchanged. ϵ=0.02 when computing the horizon. Appendix A.3.
   • Selection: for each victim model, β is selected from 0.1, 0.2, 0.3 by HarmBench, then fixed for StrongREJECT and JailbreakBench.

   Defense reward window

   • Original signal: TurnGate makes a Pass or Block decision each turn and gives a reward for that decision process. In the original source, with γ=λ=1, the advantage at turn i is the sum of rewards from that turn to the end of the conversation.
   • Truncation: change it to add only the rewards from the current turn to at most K subsequent turns. K=0 uses only the current-turn reward; Full uses up to the end of the trajectory. Only the window is changed during training; at inference the policy still observes the same conversation. Appendix A.4.
   • Judgment: for harmful conversations, the first harm-enabling turn is used as the reference to distinguish exact blocking, premature blocking and missed blocking, then combined with benign false refusals into F1ϕ1, F1ϕ2, F1ϕ3. ϕ1 gives no score for premature blocking, and ϕ3 penalizes premature blocking more strongly than ϕ2. The original threshold τ=0.5 is used first, then a separate threshold is selected for each policy on the validation set by F1ϕ2.

   Collapse this answer

4. 04What experiments did the paper conduct?

   The authors say successful trajectories of three turns and above have an average EHR of about 0.5–0.6; short history and partial down-weighting improve attack metrics.

   Experimental setup

   • Horizon measurement: X-Teaming's attacker is Qwen2.5-32B-Instruct, the victim model is Llama-3.3-70B-Instruct, and the judge is GPT-4o on a five-point scale. TROJail's attacker is initialized and trained from Qwen2.5-3B-Instruct on AdvBench, the victim model is Llama-3.1-8B-Instruct, and a HarmBench classifier above 0.9 counts as success. 200 WildJailbreak items each, up to five turns; X-Teaming's five turns are additionally evaluated on 1,500 items.
   • Continuation attacks: The attacker is still Qwen2.5-32B-Instruct, the methods are X-Teaming and ActorBreaker, and there are 159 HarmBench harmful behaviors. The victim models are Qwen3-30B-A3B, Llama-3.3-70B, Muse-Glimmer-30B, DeepSeek-V4.1-Flash, GPT-5.6 Luna, Gemini 3.8 Flash. GPT-4o, a score of 5 counts as success, ϵ=0. The main setting is five turns first, then five more if unsuccessful. Comparisons are Full and Peak. Table 1's Llama-3.3-70B and Appendix A.1's Llama-3.3-70B-Instruct: the paper does not state whether they are the same checkpoint.
   • Attacker training: One Qwen2.5-3B-IT attacker is trained separately for each victim model, 520 AdvBench items, 260 optimization steps. The victim models are Qwen2.5-7B-IT, Llama-3.1-8B-IT, Gemma-2-9B-IT. Evaluation on HarmBench, StrongREJECT, JailbreakBench. ASR@3 is at least one of three attempts above 0.9. Table 5 is the mean ± standard deviation of three runs. ϵ=0.02. The paper does not state the number of items in these three evaluation sets.
   • Defense: TurnGate, dataset MTID, policy Qwen3-4B, trained for three epochs from the same reward-weighted SFT checkpoint, seed 42. Only K is varied, taking values 0, 1, 2, 3, 4, 5, 6 and full. The paper does not state the number of MTID items.
   • Unstated items: The ϵ for the replay in Section 3, and the number of repetitions of the effective horizon measurement.

   Main results

   Average EHR of successful trajectories according to Figure 2. 200 WildJailbreak items, up to five turns; X-Teaming's victim model is Llama-3.3-70B-Instruct, TROJail's is Llama-3.1-8B-Instruct.

   Trajectory length	X-Teaming EHR	TROJail EHR
   2 turns	0.73 (n=55)	0.68 (n=57)
   3 turns	0.56 (n=67)	0.55 (n=14)
   4 turns	0.58 (n=37)	0.54 (n=6)
   5 turns	0.51 (n=9)	N/A (n=0)

   • Authors' interpretation: The EHR of a two-turn trajectory can only be 0.5 or 1. The authors say the two two-turn averages in Figure 2 mean that more than half of successful attacks recovered the original progress using only the last query. The average EHR for three turns and above is around 0.5–0.6. The authors say the deleted turns may still have been needed for the current method to find the later queries, but their ex post redundancy suggests shorter paths may exist.
   • Larger sample: On 111 five-turn X-Teaming successful trajectories out of 1,500, the main-text average EHR is 0.51, and the Figure 2 caption gives 0.512.
   • Unsuccessful trajectories: Figure 3 truncates at the highest-scoring turn. The authors say most supported groups have EHR below 1. In the 1,500-item evaluation, the average EHR of trajectories with score 4 goes from 0.729 at two turns to 0.533 at five turns.

   Continuation attacks

   Table 1: 159 HarmBench items, five initial turns followed by five more turns for unsuccessful trajectories. The table below is ∆ASR, in the order Full / Peak / EH-CR. For Base ASR, X-Teaming is 67.9%, 80.5%, 15.1%, 23.9%, 13.8%, 46.5% in order; ActorBreaker is 37.1%, 32.7%, 13.2%, 11.3%, 8.2%, 26.4% in order. EH-CR final ASR: X-Teaming is 90.5%, 95.6%, 25.8%, 42.8%, 33.9%, 63.5%; ActorBreaker is 59.1%, 60.4%, 27.0%, 22.6%, 24.6%, 50.3%.

   Victim model	X-Teaming ∆ASR	ActorBreaker ∆ASR
   Qwen3-30B-A3B	+13.2 / +15.7 / +22.6	+20.8 / +20.1 / +22.0
   Llama-3.3-70B	+10.1 / +11.3 / +15.1	+22.0 / +23.9 / +27.7
   Muse-Glimmer-30B	+3.1 / +3.1 / +10.7	+10.7 / +8.8 / +13.8
   DeepSeek-V4.1-Flash	+10.1 / +13.2 / +18.9	+4.4 / +5.7 / +11.3
   GPT-5.6 Luna	+8.2 / +11.3 / +20.1	+6.9 / +6.9 / +16.4
   Gemini 3.8 Flash	+8.2 / +8.2 / +17.0	+20.1 / +19.5 / +23.9
   Average	+8.8 / +10.5 / +17.4	+14.2 / +14.2 / +19.2

   • Same starting point: Both Peak and EH-CR return to the highest-scoring turn; the only difference is how much early history is kept. In Table 1, EH-CR is higher than Full and Peak in the same column for every attack–victim model combination. The authors say history that is not necessary for recovering progress also hinders continuation attacks.
   • Exceptions: On average ActorBreaker's Full and Peak are both +14.2, but in the three rows Qwen3-30B-A3B, Muse-Glimmer-30B and Gemini 3.8 Flash, Peak is lower than Full.
   • Budget: The Figure 4 caption does not state the attack method and victim model. The main text says that with initial lengths of 2, 3 and 4 turns, EH-CR has the highest final ASR, on average 8.2 percentage points higher than Full Context. The Figure 5 caption does not state whether it is X-Teaming or ActorBreaker; on Qwen3-30B-A3B and Llama-3.3-70B, 159 HarmBench items, five initial turns followed by 1–5 more turns, the authors say EH-CR is higher at every budget, with an advantage already appearing after one more turn. The main text does not list the specific ASR values in the figure point by point.

   Attacker credit assignment

   • Setup: Only β is changed. β=1 is TROJail's original assignment, β=0 removes the outcome signal before the horizon, and β* is the value selected from 0.1, 0.2, 0.3 on HarmBench and transferred.
   • Results: The authors say β=0 raises ASR@3 in five of nine groups and lowers it in four, including all three StrongREJECT groups (Appendix D). β* is higher than β=1 in all nine groups, by 6.9 points on average; 6.4 points higher on average across the six transfer evaluations. The selected values are 0.1 for Llama-3.1-8B-IT and Gemma-2-9B-IT, and 0.3 for Qwen2.5-7B-IT.
   • Authors' interpretation: Queries before the horizon may still help discover later phrasings, so removing the signal entirely loses useful information; giving them the same outcome signal as later queries is too coarse. The authors say keeping a smaller signal accounts for both cases. Not every β in the grid is better than β=1: Llama-3.1-8B-IT on HarmBench with β=0.3 has ASR@3 of 77.50±0.41, lower than β=1's 85.83±0.62 (Table 5).

   Defense reward window

   • Setup: Same checkpoint, data, reward and optimization, only K is varied. Figure 7 and Appendix E. Fixed threshold τ=0.5, and the threshold that maximizes F1ϕ2 on the validation set.
   • Before and after calibration: The authors say that at τ=0.5, performance is generally better the farther the reward propagates; at this threshold Full's F1ϕ2 is 0.644 (Table 6). After calibration F1ϕ2 is between 0.659 and 0.664. The main text gives the standard deviation as 0.16%, Appendix E gives 0.16 percentage points. K=0 is 0.661, Full is 0.664. The validation-set threshold goes from 0.3430 at K=0 to 0.4626 at Full.
   • Threshold-free metrics: In Table 7, AUROC goes from 0.808 to 0.810, and AUPRC from 0.452 to 0.457. K=0 has AUROC 0.810 and AUPRC 0.457, Full has 0.809 and 0.452. The authors say there is no monotonic improvement as K increases; the gain at a fixed threshold looks more like a difference in where scores sit relative to a common cutoff than a large difference in ranking quality. The no-skill baseline for AUPRC is the positive rate of 19.27%.

   Other ablations and analyses

   • Query cost (Table 3, including suffix verification, average victim-model queries per behavior): X-Teaming is Full 17.07, Peak 16.64, EH-CR 16.34; ActorBreaker is 6.13, 6.14, 8.56. The authors say there is no uniformly lower query cost.
   • Continuation turns (Table 4, six models pooled): X-Teaming is 4.67, 4.58, 4.03; ActorBreaker is 4.59, 4.61, 4.35. On both attacks EH-CR has fewer continuation turns for every victim model.
   • Case (Table 2, one five-turn X-Teaming trajectory): The original scores are 1, 2, 4, 4, 5 in order. Replaying only the last turn gives 4, and replaying turns 4–5 recovers 5, so Heff=2 and EHR=0.4. The appendix gives redacted query roles; the query content is not repeated here.
   • Appendix E: The authors say the reproduced Full differs slightly from TurnGate's reported value at the original threshold, but do not give the other party's numbers; the comparison is of relative differences between matched runs. The SFT row was not calibration-matched and serves only as a reference.

   Collapse this answer

5. 05What are some points that can be further explored?

   The authors state that recovery is not the same as discovery, that the horizon depends on the judge and the tolerance, and that the temporal dependence could also be finer-grained.

   Limitations and future directions noted by the authors

   • Recovery is not discovery (Section 6): The effective horizon is measured only after later queries have already been found. Early turns may still help construct these queries even if replay no longer needs them. The authors say this relates to multi-agent attacks, adaptive tree search and lifelong attack generation, and that effective horizon information could later be used to directly discover shorter attack paths.
   • Horizon depends on the progress metric (Section 6): The shortest suffix recovers the original attack score, so it depends on the judge and the recovery tolerance. The authors say they used existing jailbreak evaluators including HarmBench, but repeated replay or multiple progress metrics could give more robust estimates. The current analysis is turn-level; measuring dependencies within queries may allow additional compression.
   • Temporal dependencies could be richer (Section 6): EH-CA uses a single boundary to assign final-outcome credit; the defense experiments use a fixed reward window rather than estimating a horizon for each decision. The authors say more adaptive methods could learn from the trajectory itself how much early turns should be affected, a problem that also extends to multi-turn agents and multi-agent attacks.

   Experimental coverage

   • Effective horizon measures successful trajectories of X-Teaming and TROJail, plus the highest-scoring turn of unsuccessful X-Teaming trajectories; 200 WildJailbreak items (up to five turns) and X-Teaming's 1,500 five-turn evaluations (Section 3, Figure 2–3, Appendix A.1).
   • Continuation attacks cover X-Teaming and ActorBreaker, 159 HarmBench items, six victim models; the main protocol is five turns plus five turns, with initial length and continuation budget also varied (Section 4, Table 1, Figure 4–5). Query counts are in Appendix C.
   • EH-CA covers three victim models and three benchmarks, with β selected only on HarmBench; ASR@3 reports three runs (Appendix A.3, D, Figure 6).
   • Defense experiments are TurnGate on MTID, Qwen3-4B, eight reward windows, reporting calibrated F1 and AUROC/AUPRC (Section 5.2, Appendix E).
   • Appendix A.1 does not give the ϵ used for the replay in Section 3, nor report the number of repetitions of that measurement. The Figure 4 caption does not state the attack method and victim model, and the Figure 5 caption does not state the attack method.

   Collapse this answer

6. 06Summarize the main content of the paper

   The authors say trajectories can be longer than the required history; short context improves continuation attacks, and the short learning signal changes very little after calibration.

   In multi-turn jailbreaks, the interaction that discovers an effective query can be longer than the query history needed to recover the same progress after the fact. The authors measure the latter by replay, i.e., the effective horizon.

   • Method: do not rewrite the attacker queries; only delete the prefix, replay in a new conversation and regenerate the victim replies. The shortest suffix length that recovers the target score is the effective horizon, and EHR is its ratio to the length up to the target turn.
   • Compression: on X-Teaming and TROJail, with 200 WildJailbreak items, the average EHR of two-turn successful trajectories is 0.73 and 0.68; the average EHR of longer trajectories is about 0.5–0.6. Of 1,500 items, 111 five-turn X-Teaming successful trajectories have a figure-caption average EHR of 0.512. The authors state that on average about half of the original query sequence suffices. The highest-score point of unsuccessful trajectories can also be recovered from a shorter history: in the 1,500-item evaluation, the average EHR at score 4 goes from 0.729 at two turns to 0.533 at five turns.
   • Continued attack: EH-CR returns to the highest-score turn and keeps only the effective suffix. In Table 1, relative to the five-turn initial trajectory, the average ∆ASR is +17.4 for X-Teaming and +19.2 for ActorBreaker, and every combination is higher than Full and Peak.
   • Training: EH-CA multiplies the final outcome signal of queries before the horizon by β, which is less than 1. β* raises ASR@3 by an average of 6.9 points across nine groups; β=0 causes a decrease in four groups. On the defense side, after calibration the F1ϕ2 of each reward window has a standard deviation of 0.16% in the main text; K=0 is 0.661 and the full-trajectory return is 0.664.
   • Authors' conclusion: trajectory length and the length of history that later behavior depends on can differ greatly. Separating the two can determine what context to keep and how far the learning signal should propagate. The authors believe this distinction may also matter beyond jailbreaks, in long-horizon multi-turn agent reinforcement learning.

   Collapse this answer

Related papers