Peak + Accumulation: A proxy-layer scoring formula for multi-turn LLM attack detection

Multi-turn prompt injection splits malicious intent across multiple requests, and the proxy layer has no cross-turn aggregation formula that does not call an LLM.

• Scenario: The API proxy checks messages[] and then does allow/block, and cannot call an LLM. The authors state that a classification LLM would add 100–500ms latency per request, be billed per token, and itself be subject to prompt injection.
• Existing gaps: The authors state that single-turn proxy detection has been studied, while multi-turn work requires LLM classification. The authors state that, to their knowledge, there is no complete formula that uses only proxy-computable operations to aggregate per-turn pattern scores into a conversation-level risk score.
• Core observation: Theorem 1: when all turn scores are the same s, the weighted average equals s, independent of the number of turns; the authors state this holds for any weighted average. When a single-turn score is below the threshold, a sustained attack cannot cross that threshold. The authors give an example: under a threshold of 0.7, four turns all with role_confusion (weight 0.5) still yield 0.5.
• Proposed formula: They propose peak + accumulation, combining peak, sustained proportion, category diversity, and escalation and resampling bonuses. The authors evaluate on 10,654 multi-turn dialogues and state they have released the algorithm, regex library, and evaluation tools.
• Threat model:
  • Goal: Intercept cross-turn distributed prompt injection before the request reaches the model.
  • Position: The victim side is the proxy that checks messages[] and the model endpoint behind it; the decision cannot call an LLM.
  • Visible information: The proxy has the full conversation history, and the formula scans user or tool messages. The paper does not state whether the attacker knows the regex, nor does it use white-box or black-box.
  • Form: The intent is distributed across multiple turns. The authors cite that on single-turn hardened models, multi-turn jailbreak ASR is >70%, Crescendo jailbreaks most models in fewer than 5 turns, and jailbreak state persists. Content safety attacks without injection phrasing are marked as out of scope.

Related work is divided into single-round agent detection, multi-round classification requiring LLMs, multi-round attack characterization, and cumulative scoring in adjacent fields. The authors state that none of these provide a cross-round aggregation formula using only agent-computable operations.

Single-round agent detection

• DMPI-PMHFE (Shen et al.): DeBERTa-v3-base embeddings plus heuristic channels, including WordNet synonym markers for 8 attack patterns, many-shot for more than 3 question-answer pairs, and repeated tokens occurring more than 3 times. The authors cite its 97.94% accuracy on safeguard-v2.
• PromptScreen, NeMo Guardrails, Rebuff: TF-IDF plus linear SVM; length/perplexity ratio with prefix and suffix perplexity thresholds targeting GCG-style suffixes; and a four-layer defense with keyword permutation matching. The paper states these all handle only single rounds.

Multi-round detection requiring LLMs

• Defensive M2S (Kim et al.): compresses multiple rounds into a single round with hyphens, then classifies with Qwen3Guard. The authors cite 93.8% recall and a 94.6% reduction in reasoning tokens.
• MindGuard and AprielGuard: the former uses clinical risk classification, round-level annotation, and 4B–8B classifiers; the latter trains an 8B guardrail model with structured reasoning traces.
• LessWrong proposal: the authors state its architecture is closest to agent-level scoring, in the form P(jailbreak)=fE(response) with dual thresholds, but fE uses an LLM evaluator. The paper states none of the above methods are computable on the agent.

Multi-round attack characterization

• Yang et al.: multi-round jailbreaks approximate resampled single-round attacks, with success rising with number of attempts as S(k)=A−B·e−ck. The authors take this as evidence that the main signal is persistence, not pattern escalation.
• Crescendo and MTJ-Bench: the paper states the former escalates gradually and each user prompt is intentionally harmless, so pattern matching cannot detect it; the latter distinguishes follow-up questions from cross-topic continuation.
• Ren et al.: present seven mechanism-oriented jailbreak families, including escalation and persistence.

Cumulative scoring in adjacent fields

• CUSUM, Bayesian belief updating, Splunk RBA: accumulate deviations and alert when above threshold, update posteriors monotonically with evidence, and sum anomalies rather than averaging. The authors state the commonality is that more evidence yields higher scores.

• Baseline and corpora: the baseline is the weighted average from §3, with recency weights linear from 1.0 to 2.0. This baseline's scores are reported only on 9 handcrafted multi-round attacks, not on the full WildJailbreak and WildChat sets. Evaluation corpora are handcrafted sequences, conversations assembled from WildJailbreak adversarial prompts, and benign WildChat conversations.

The authors state that to their knowledge, no prior agent-level formula handles this aggregation; multi-round detection exists but requires calling an LLM. The authors replace averaging with accumulation and use only agent-computable operations.

Without calling an LLM, it scores user or tool messages by category, then uses peak + accumulation to add the peak, persistence, and diversity bonuses; if the score reaches τ, it intercepts.

Per-Turn Scoring

• Each user or tool message is checked against a set of regex categories. Category c's weight wc is in [0, 1].
• The default five categories (Table 2): instruction_seeding 0.4, role_confusion 0.5, deferred_authority 0.3, escalation_probing 0.3, repetition_resampling 0.2. The last category is detected by the algorithm, and the table gives no regex examples for it.
• A turn's score is the sum of the hit weights, truncated to [0, 1]: s_i=min(1,∑_{c∈ matched(i)} w_c).

Conversation-Level Score

• The peak is the highest score across turns; the match_ratio is the number of turns with si>0 divided by n; the diversity is max(0, number of distinct categories − 1) times δ.
• The total score is truncated to [0, 1]: score=clamp(peak+match_ratio·ρ+diversity+β_e+β_r,0,1). If score≥τ, it blocks.
• Defaults are ρ=0.45, δ=0.15, βe=0.2, βr=0.7, τ=0.7. These hyperparameters are tuned on the validation split.
• βe: added when at least 3 consecutive turns have strictly increasing scores; §4.4 writes the last at least 3 turns. βr: added when at least 3 consecutive pairs of user messages have word-level trigram Jaccard similarity >0.5; messages are lowercased and punctuation removed first, and messages shorter than 20 tokens do not participate. repetition_resampling is a per-turn category weight, while βr is a conversation-level bonus; the paper does not state whether the two stack on the same resampling.

Design Principles and Worked Examples

The principles come from §2.4, and the paper does not provide further derivation: a high single-turn score should enter the conversation score in full; more hit turns should give a higher score; spanning multiple categories is more suspicious than repeating one category; escalation gradients and resampling should be added, not averaged away.

The worked examples in §4.6 do not include escalation or resampling bonuses:

• Scores [0, 0, 0, 0.3]: score 0.4125, Allow.
• [0, 0.3, 0, 0.5] with two categories: score 0.875, Block.
• All four turns at 0.5: score 0.95, Block. The weighted average of the same sequence is 0.5, below 0.7.

Position in the Agent

The evaluation implementation is in Parapet, with L3 single-turn matching disabled and only L4 kept, so that L3's false positives on benign conversations are not counted against L4. L4 is enabled when the number of user messages reaches min_user_turns (default 2). The authors also place L0 normalization before L4 and list L3 and L5a output scans; these layers are not comparisons in this paper's experiments.

Experimental Setup

• Implementation: The formulas are written in Parapet. The evaluation is L4-only, with L3 disabled. The interception condition is score≥0.7. There is no LLM review. The paper does not report the number of repetitions. The handcrafted sequences include assistant replies; the formulas scan user or tool messages.
• Attacks: 9 handcrafted ones, each with 3–4 user turns, covering instruction seeding, role confusion, deferred authority, escalation probing, and resampling. There are also 579 assembled from WildJailbreak adversarial prompts: the authors state this dataset has 262K adversarial/benign prompt pairs; 5,529 injection sentences were drawn from 82K adversarial prompts, falling into four L4 categories, then composed into conversations as 179 for single-category persistence, 300 for multi-category combinations, and 100 for escalation gradients with benign openings. The main text describes four strategies but gives only these three group counts. The attacks total 588. The abstract writes all 588 as originating from WildJailbreak; Table 4 splits them into 9 handcrafted and 579.
• Benign: 6 handcrafted; 60 sparse-benign, where only 1 turn out of 3–4 is an injection, labeled benign; 10,000 WildChat, sampled from 838K ChatGPT conversations, keeping those with at least 3 user turns and each turn labeled non-toxic. The benign total is 10,066, and the overall total is 10,654. The abstract writes all 10,066 benign ones as coming from WildChat; Table 4 records WildChat as 10,000.
• Splits: ρ, δ, βe, βr, and τ are tuned on the validation split. The authors state the final metrics are reported only once on a fixed and disjoint 20% holdout. Table 3's scale is all 10,654 entries. The authors state that, because redistribution licenses vary, they release the construction and evaluation scripts.
• Comparisons and exclusions: The weighted average has scores only on the 9 handcrafted attacks. On SafeMTData and Anthropic hh-rlhf, both formulas have 0% recall, and the authors exclude them from the main results.

Main Results

Table 4 gives per-dataset accuracy (peak + accumulation). The aggregate metrics are after the table, under Table 3's ρ=0.45 and τ=0.7.

| Dataset | Label | Total | Correctly classified | Accuracy | | multiturn_attacks | malicious | 9 | 9 | 100.0% | | wildjailbreak_attacks | malicious | 579 | 525 | 90.7% | | multiturn_benign | benign | 6 | 6 | 100.0% | | wildjailbreak_sparse_benign | benign | 60 | 60 | 100.0% | | wildchat_benign | benign | 10,000 | 9,879 | 98.8% |

• Table 3: TP 534/588, FN 54/588, FP 121/10,066, TN 9,945/10,066; recall 90.8%, precision 81.5%, F1 85.9%, FPR 1.20%, accuracy 98.4% (consistent with Table 4's total of 10,479/10,654).
• Of the 54 missed detections, 39 use only escalation_probing or deferred_authority with weight 0.3, and not all turns hit; the authors write that 0.3+match_ratio×0.45 is still below 0.7. The remaining 15 are escalation-gradient conversations, where the benign opening lowered the match_ratio.
• All 121 false positives are in WildChat. The authors state they come from role-playing, claims discussing AI capabilities, and creative writing with authority figures, and are a matter of regex specificity rather than the scoring formula. There are no false positives in sparse-benign. The authors state that 81.5% precision shows the pattern library is usable in production at a 1.20% FPR.

Persistence Parameter Sweep

• §6.2 sweeps ρ from 0.15 to 0.65 in steps of 0.025 on the validation split, then fixes ρ. Figure 1's caption writes n=588 attacks, 10,066 benign, threshold 0.7, and annotates recall 90.8%, FPR 1.20%, F1 85.9% at ρ=0.45, matching Table 3. The paper does not give the sample sizes for the two splits separately.
• The authors state a phase transition occurs between ρ=0.375 and 0.400: recall goes from 77.4% to 89.8%. The main text writes an increase of 12.4 percentage points; the abstract writes 12 percentage points, and the figure annotates +12pp. FPR goes from 1.11% to 1.19%, an increase of 0.08 percentage points. The explanation is that with weight 0.3 and match_ratio=1.0, the score is 0.3+ρ: 0.375 corresponds to 0.675, and 0.400 to 0.700.
• The authors choose the default at ρ=0.45 and state F1 is maximal there, at 85.9%. At ρ=0.5, recall is 91.8% and FPR 1.28%; at ρ=0.6, the figure annotates recall 95.9% and FPR 1.51%. The authors state that from 0.375 to 0.45, each 0.01 percentage point of FPR buys 1.5 percentage points of recall; from 0.5 to 0.6, each 0.01 percentage point buys only 0.18 percentage points. With match_ratio=0.75, 0.3+0.75×0.45=0.6375, still below the threshold.

Weighted Average Comparison

• §3.3 uses 9 handcrafted sequences and five attack categories. Of these, 5 hit on every turn, with scores from 0.3 to 0.5, judged Allow.
• Table 1's three examples: mt-atk-003, role confusion, 4 turns, 0.50, Allow; mt-atk-004, role confusion plus persona, 3 turns, 0.50, Allow; mt-atk-006, escalation probing, 4 turns, 0.30, Allow.
• The paper does not report the weighted average's recall or FPR on the 10,654 entries.

Out-of-Scope Datasets

• On SafeMTData (crescendo-style content safety) and Anthropic hh-rlhf (social engineering), both formulas have 0% recall.
• The authors state this is expected: the wording is deliberately harmless with no injection boilerplate; the agent regex cannot detect topic-trajectory escalation, a capability that requires LLM semantic classification. Therefore it is not counted in Table 3.

Other Ablations and Analyses

• δ=0.15: the authors state that 1 more category adds 0.15, and 2 more add 0.30; there is no sweep table.
• βe=0.2: the authors state it alone is insufficient to trigger interception at τ=0.7.
• βr=0.7: the authors state it alone is close to the threshold.
• The paper does not report numerical sweeps of δ, βe, βr, or τ.

Limitations and future directions noted by the authors

• Content safety: Crescendo-style attacks use deliberately innocuous language. The authors state that the proxy-level regex cannot detect topic trajectory escalation; this is a fundamental limitation of the proxy-level approach, not a problem with the formula (§5.4).
• Synthetic structure: The attacks were expanded from 9 handcrafted to 588, and the multi-turn structure is synthetically concatenated. The authors state that there is no public multi-turn prompt injection dataset; existing multi-turn datasets target content safety, while injection datasets are single-turn. The evaluation accepts YAML, and the authors encourage contributions of real multi-turn injection trajectories (§5.4).
• Regex can be bypassed: The authors state that rewriting, encoding tricks, or indirect phrasing can bypass the regex, which is a known limitation of pattern-based methods. Scoring is orthogonal to pattern quality, and improving pattern robustness is described as complementary future work (§5.4).

Experimental coverage

• The main table covers 588 attacks and 10,066 benign cases, determined by comparing scores with τ=0.7; the experiments did not run generative models or LLM judges (Table 3, §5.1). In related work, the numbers for Defensive M2S, MindGuard, and AprielGuard are cited by the authors, not reproduced on this set of dialogues.
• The weighted average has scores only on 9 handcrafted sequences (Table 1); the paper does not report its metrics on the 10,654 cases.
• On SafeMTData and Anthropic hh-rlhf, both formulas have 0% recall and are excluded from the main results (§5.1).
• ρ is scanned on the validation split from 0.15 to 0.65 in steps of 0.025 (§6.2). The authors write that the final numbers come from a disjoint 20% holdout, but Table 3 and Figure 1 specify n as 588 and 10,066; the paper does not report metrics separately for the two splits.
• The paper does not report the number of repetitions, nor detection numbers under rewriting, encoding, or indirect phrasing as described in §5.4. The authors state that due to varying licenses, they release the construction and evaluation scripts rather than redistributable full dialogues (§5.1).

peak + accumulation provides multi-round scoring for API agents: without calling a model, per-round regex signals are folded into a conversation score, and the conversation is blocked once the threshold is reached.

The case to handle is malicious intent split across multiple rounds. When all rounds score equally, weighted averaging equals the single-round score, so sustained hits below the threshold never cross the line.

In implementation, a round's score is the sum of hit-category weights truncated to 1. The conversation score takes the peak, plus the hit-round proportion multiplied by ρ, the number of extra categories multiplied by δ, and βe for strictly increasing and βr for trigram resampling. Defaults are ρ=0.45, τ=0.7.

Table 3, on 10,654 entries with ρ=0.45, τ=0.7, reports recall 90.8% (534/588), precision 81.5%, F1 85.9%, and FPR 1.20% (121/10,066). In Table 4, 525 of 579 WildJailbreak-assembled entries are correctly identified. Across 9 handcrafted sequences, weighted averaging places 5 in 0.3–0.5 and Allow. As ρ goes from 0.375 to 0.400, recall goes from 77.4% to 89.8% and FPR from 1.11% to 1.19%. Recall on SafeMTData and hh-rlhf is 0% and is not included in the main tables.

The authors conclude that weighted averaging is unsuitable for sustained detection; peak, persistence proportion, and category diversity can be added to compensate. §5.4 marks content-safety attacks with innocuous wording, synthetic multi-round structure, and regex rewriteability as scope limits or future work. The authors state the formula is deterministic and auditable, requires no inference, training data, or GPU, and is microsecond-level per request; the paper provides no timing table.