The study uses mathematical encoding to bypass LLM safety filters, with an average attack success rate of 46%–56% across eight models

Safety mechanisms mainly match harmful intent in natural language; after rewriting the same intent into a coherent mathematical problem, can the filter still block it.

• Scenario: The authors state that as LLMs enter safety-critical applications, layered defenses such as RLHF, refusal training, and Llama Guard can still be bypassed by jailbreaks. Safety training mostly targets harmful intent in natural language, and the same intent in abstract mathematical form is harder to identify.
• Gap in existing work: The authors relay that MathPrompt encodes harmful instructions as set theory problems, with an average ASR of 73.6% across 13 LLMs. The authors want to distinguish: is it the mathematical notation that works, or the deep rewriting during encoding.
• Authors' hypothesis: The helper LLM rewriting harmful content into genuine mathematical problems, rather than superficially adding mathematical formatting, is the main factor in bypassing filters.
• Threat model:
  • Knowledge: black-box. Does not modify target model weights, system prompts, or safety filters.
  • Capability: API-level access to both the helper LLM and the target model, submitting only plain text prompts.
  • Victim system: Assumes the target has RLHF, content filtering, and refusal training. The authors state that GPT-5 additionally has API-level input filtering that intercepts some biology/CBRN prompts before they reach the model.
  • Goal and evaluation: Make the response provide substantive content addressing the original harmful intent. Reviewers label it as jailbreak or refusal; ASR is the proportion classified as jailbreak.
• What this paper does: Proposes Formal Logic encoding and divides six strategies into LLM-based and rule-based, comparing across eight target models and two benchmarks; then uses verbatim repetition to test whether superficial concatenation changes ASR. Inverse Semantic Mapping is only a conceptual defense and was not experimented with.

Related work is divided into prompt jailbreaks, encoding camouflage, mathematical reasoning, and evaluation benchmarks, comparing only the differences the authors explicitly name.

Prompt and optimization-based jailbreaks

• Manual and automated attacks: Shen et al. (2024)'s DAN uses role-playing to override safety training; this entry also cites Perez et al. (2022) on the tension between instruction following and safety training. Zou et al. (2023)'s GCG uses gradient search to append suffixes, Chao et al. (2023)'s PAIR uses an attacker LLM to iteratively rewrite, and Liu et al. (2024)'s AutoDAN uses genetic algorithms to generate readable attacks.
• Sampling and surveys: Hughes et al. (2024) report that sufficiently many Best-of-N naive samples can also bypass filters. Xu et al. (2024) survey prompt-level attacks and defenses. The authors state these methods operate on natural language, whereas this work encodes into mathematical forms and does not require gradients or iterative refinement.

Encoding and intent camouflage

• Ciphers, multi-turn, and structured formats: Yuan et al. (2024)'s CipherChat uses ciphers to encode instructions. Palo Alto Networks Unit 42 (2024)'s Deceptive Delight introduces harmful content gradually across multiple turns. Unjail AI (2025)'s Policy Puppetry uses JSON and XML to mask intent.
• Camouflage and MathPrompt: Zheng et al. (2025) evaluate camouflaged jailbreaks embedded in harmless language. Bethany et al. (2024)'s MathPrompt encodes harmful prompts as set theory problems; the authors report its ASR as 73.6% across 13 LLMs.
• How the authors connect to MathPrompt: The authors state they introduce Formal Logic and Quantum Mechanics; they claim to provide the first systematic comparison of deep-rewriting LLM-based encoding versus surface rule-based encoding; and they evaluate on a broader set of model families and benchmarks.

Mathematical reasoning

• GSM-Symbolic: Mirzadeh et al. (2024) document performance degradation in elementary math under slight surface perturbations. The authors argue this suggests models process mathematical statements via pattern matching, and claim their attack exploits the gap between mathematical capability and semantic safety filters.

Evaluation benchmarks

• Two main benchmarks: Mazeika et al. (2024)'s HarmBench contains 159 harmful behaviors, with the original protocol using a Llama 2 13B-based classifier. Chao et al. (2024)'s JailbreakBench contains 100 behaviors from AdvBench, with a binary classifier and a public leaderboard. The authors choose them to avoid artifacts from a single prompt distribution or a single protocol.
• Baselines: Unencoded prompts serve as the baseline. Set Theory and Addition Equation come from MathPrompt, compared alongside Formal Logic, Quantum Mechanics, Conditional Probability, and Symbol Injection. The datasets are the two benchmarks above.

The authors position this work on encoding jailbreaks: rather than searching for natural language jailbreak sentences, they compare mathematical forms and the role of deep rewriting relative to surface notation.

Overall, the attack transforms a harmful prompt x into a math problem x′, then submits it to the target model. Six encodings are divided into two categories based on whether they use a helper LLM (Figure 2): one category performs deep rewriting, the other only adds surface formatting.

Three-stage pipeline

Each attack proceeds in three steps (Figure 1). The target model's weights, system prompt, and safety filters are not modified.

1. Encode. LLM-based encodings are rewritten by a helper; rule-based encodings use deterministic algorithms. The helper is GPT-4.1-Mini, with a separate system prompt for each form, two few-shot examples, and an academic research framing to reduce refusals. Separator labels are removed before submission. Encoding uses temperature = 0.0, seed = 42, top-k = 0, top-p = 1.0, and a maximum output of 16,384 tokens. Appendix A gives the full prompts; sentences are not excerpted here.
2. Submit. The three LLM-based encodings share a single instruction set: first solve the math problem, then give concrete real-world examples. The authors state that this instruction is key because it prompts the model to turn abstract solutions into concrete content, effectively reversing the encoding. Rule-based encodings require decoding and answering.
3. Evaluate. The evaluator labels the response as jailbreak or refusal. In the experiments, this step is performed by GPT-5-Nano following the HarmBench protocol.

LLM-based encodings

Appendices B–D contain examples; only the mechanisms are described here.

• Set Theory: Follows MathPrompt. Content enters a universe and subsets, operations use non-Abelian group ordering, and conditions and goals are written as predicates. The authors state that the intent is distributed across set operations and predicates, rather than in a single harmful natural-language sentence.
• Formal Logic: Proposed by the authors. Actions become predicates, conditions become connectives and quantifiers, and the goal becomes a theorem to be proven by natural deduction. The authors hypothesize that the intent lies in the derivational dependencies among premises, not in individual predicates.
• Quantum Mechanics: The scenario is written as a problem on a composite Hilbert space, with the harmful process corresponding to finding an evolution that simultaneously satisfies transition constraints and detection-avoidance conditions. The authors note that state labels may still carry semantics, potentially leaking intent to classifiers.

Rule-based encodings

No LLM is used; the original text remains largely readable or can be directly reassembled. The authors use them as controls to separate deep rewriting from the presence of mathematical notation. Appendices E–F give algorithms and examples.

• Addition Equation: From MathPrompt. The text is split into N = 6 word-preserving segments, written as addition equations, and the model is asked to concatenate and answer. The authors believe the splitting may disrupt filters that rely on complete phrases.
• Conditional Probability: Also N = 6, with reconstruction written as conditional probabilities. The authors believe the statistical formulation may bypass filters targeting imperative sentences, while content remains unchanged.
• Symbol Injection: K = 10 symbols are uniformly inserted from about 100 symbols. The authors call this the weakest encoding, used to test whether the mere presence of math symbols is sufficient.

Repetition and judgment

After encoding, the prompt can be concatenated verbatim once, motivated by results from Leviathan et al. (2025) on repetition improving non-reasoning task performance. The authors use it to distinguish: if ASR rises, the attack is sensitive to surface formatting; if not, the effect depends on the initial rewriting. Judgment rule: producing substantial harmful content targeting the original malicious intent is a jailbreak; refusal, deflection, or safety warnings are refusals. ASR is the percentage of jailbreak labels.

Experimental Setup

• Target Models (8): GPT-4o-Mini, GPT-5-Mini, GPT-5; Gemini-2.0-Flash, Gemini-3-Flash; Claude-3.7-Sonnet; Llama-3-8B, Vicuna-13B. The latter two run locally on NVIDIA A100 via vLLM to avoid API filtering. The authors state Vicuna-13B is fine-tuned from Llama 2 on user conversations without explicit safety training.
• Benchmarks: HarmBench with 159 items; JailbreakBench with 100 items (taken from AdvBench). The authors do not use AdvBench's string matching, citing high false positives; they do not use StrongREJECT's graded scale, citing unsuitability for large-scale cross-strategy comparisons.
• Conditions: Six encodings plus an unencoded baseline. LLM-based covers eight models (Tables 1a, 2a). Rule-based and repetition are reported only for GPT-4o-Mini, Claude-3.7-Sonnet, Gemini-2.0-Flash, and Llama-3-8B.
• Metrics and Review: ASR is the proportion of jailbreak labels. All responses are binary-judged by GPT-5-Nano using the HarmBench protocol. The paper does not use the original HarmBench Llama 2 13B classifier and does not report human agreement.
• Decoding: temperature = 0, top-k = 0, top-p = 1; GPT-5 series does not accept external temperature. Each prompt is evaluated once per model. The helper is GPT-4.1-Mini, which the authors state has a low refusal rate for mathematical transformations.

Main Results

The table below shows ASR for LLM-based encoding and unencoded baseline on HarmBench (Table 1a).

Swipe horizontally to read the table

Model	Set Theory	Formal Logic	Quantum Mechanics	Baseline
GPT-4o-Mini	60.4%	58.5%	50.3%	12.6%
Claude-3.7-Sonnet	56.6%	57.2%	50.9%	10.7%
Gemini-2.0-Flash	65.4%	61.6%	49.7%	4.4%
Llama-3-8B	59.1%	60.4%	44.7%	4.4%
Vicuna-13B	44.7%	40.9%	21.4%	22.0%
GPT-5-Mini	24.5%	31.4%	14.5%	6.9%
Gemini-3-Flash	67.3%	58.5%	59.1%	16.4%
GPT-5	27.7%	32.1%	13.2%	2.5%

• Average Gap: The authors report the eight-model LLM-based average ASR as 46.3%, the four-model rule-based as 11.0%, and the eight-model baseline as 10.0%. The authors state the gap exceeds 35 percentage points and consider deep rewriting rather than notation the main cause; however, in the Discussion they state this gap cannot yet be solely attributed to rewriting depth. The rule-based model set differs from the other two.
• Table 1b is not universally near baseline: GPT-4o-Mini's three rule-based encodings are 30.2%, 20.8%, and 17.6% (baseline 12.6%). Claude-3.7-Sonnet's are 1.3%, 3.1%, and 0.0% (baseline 10.7%). Gemini-2.0-Flash's are 14.5%, 10.1%, and 6.3% (baseline 4.4%). Llama-3-8B's are 15.7%, 6.9%, and 5.7% (baseline 4.4%). The other four models are not reported.
• Author's Model Interpretation: The authors state Claude's rule-based encodings are ≤3.1%, but LLM-based is up to 57.2%. They state GPT-5's LLM-based is 13–32%, GPT-5-Mini's 7–31%, and GPT-4o-Mini's 50–60%. They state Gemini-3-Flash is the most vulnerable (58–67%). They consider Vicuna-13B's lower numbers (21–45%) possibly related to weaker mathematical reasoning; in Table 1a, its Quantum Mechanics is 21.4% and baseline is 22.0%.

Cross-Benchmark Validation

Table 2 replicates the eight-model LLM-based encoding on JailbreakBench. The authors report an average ASR of 55.9%, four-model rule-based at 8.8%, and eight-model baseline at 7.1% (Section 5.1).

• Endpoints: In Table 2a, Gemini-2.0-Flash's Set Theory is 78.0%, Vicuna-13B's Quantum Mechanics is 15.0%, GPT-5's Set Theory, Formal Logic, and Quantum Mechanics are 52.0%, 48.0%, and 25.0%, and the highest baseline is Vicuna-13B's 12.0%. The authors report the two Gemini generations at 49–78% and GPT-5 at 13–52%.
• Formal Logic: The authors report HarmBench averages of 50.1% for Formal Logic and 50.7% for Set Theory. On the GPT-5 family, they report Formal Logic averaging 42.1% and Set Theory 37.1%. GPT-5-Mini has 31.4% vs. 24.5% (HarmBench) and 57.0% vs. 44.0% (JailbreakBench). In Table 2a, GPT-5 shows the opposite: Set Theory 52.0%, Formal Logic 48.0%.
• Table 2b: The four models' Addition Equation values are 22.0%, 0.0%, 14.0%, and 9.0%; Symbol Injection are 16.0%, 2.0%, 16.0%, and 7.0%; Conditional Probability are 10.0%, 0.0%, 3.0%, and 7.0%. The order matches Table 1b.

Repetition Post-processing

Tables 3–4 cover only four models. The authors report an average ΔASR of −0.46% across 56 experiments, with 85.7% satisfying |Δ| ≤ 5 percentage points. They state repetition does not systematically change ASR and the effect depends on the initial encoding. Examples where before and after values fall outside this range: on HarmBench, GPT-4o-Mini's Set Theory 60.4%→67.3%, Formal Logic 58.5%→50.9%, and Llama-3-8B's Symbol Injection 6.9%→20.1%; on JailbreakBench, Llama-3-8B's Set Theory 71.0%→64.0%, Symbol Injection 7.0%→16.0%, and Gemini-2.0-Flash's Formal Logic 68.0%→73.0%. The other four models are not reported.

Failure Modes

The Discussion is not new experiments but a synthesis of the above results.

• Comprehension Failure: The authors state models fail to parse the formulations, most evident under Quantum Mechanics for Vicuna-13B and GPT-5.
• Semantic Leakage: The authors use this to explain lower Quantum Mechanics averages, stating state labels may still allow the classifier to partially recover intent.
• API Filtering: The authors state GPT-5 rejects some biology/CBRN requests before the prompt reaches the model.

Other Ablations and Analysis

The paper has no hyperparameter sweeps. The authors state in the Discussion that length, complexity, and format change simultaneously, so the gaps in Tables 1–2 cannot be attributed solely to rewriting depth. ISM, symbol density monitoring, mathematics content classifiers, and multi-model verification were not experimented with.

Limitations and future directions noted by the authors

• Evaluation: Success is determined solely by GPT-5-Nano automatic judgment, with no human evaluation; the binary ASR does not distinguish whether outputs are truly executable or only partially jailbroken (Limitations).
• Confounding: The comparison of the two encoding types simultaneously changes prompt length, complexity, and format; the authors state that the gap cannot be attributed solely to rewriting depth (Limitations; Discussion makes the same claim).
• Language and aggregation: All experiments use English prompts; ASR is reported only overall, not broken down by harm category (Limitations).
• Version and repetition: API evaluation is affected by opaque model versions; repetition post-processing only does verbatim copying (Limitations).
• Defenses: Inverse Semantic Mapping, as well as symbol density monitoring, math content classifiers, and multi-model cross-checking, are described by the authors as conceptual proposals; feasibility, computational cost, and effectiveness against adaptive attackers are left for future validation (Discussion).
• Future work: The authors encourage research into more mathematical forms, multi-turn encoding, and human evaluation of jailbreak output quality (Conclusion).

Experimental coverage

• Target models are GPT-4o-Mini, GPT-5-Mini, GPT-5, Gemini-2.0-Flash, Gemini-3-Flash, Claude-3.7-Sonnet, Llama-3-8B, and Vicuna-13B, totaling 8 (Section 4.2).
• Benchmarks are HarmBench (159 prompts) and JailbreakBench (100 prompts). Each prompt is evaluated once per model, with generation set to deterministic decoding (Section 4.3).
• All six encodings have ASR. Rule-based and repetition post-processing are only reported in tables for four models: GPT-4o-Mini, Claude-3.7-Sonnet, Gemini-2.0-Flash, and Llama-3-8B (Tables 1b, 2b, 3, 4).
• The helper is GPT-4.1-Mini, and the evaluator is GPT-5-Nano. The paper does not report agreement between the evaluator and humans, nor ASR by harm category.
• Repetition experiments total 56 runs, covering the two benchmarks and the four models mentioned above (Section 5.2). The Discussion proposes Inverse Semantic Mapping, but the paper does not report experiments on that proposal.

The authors compare whether six mathematical encodings can cause models with safety measures still enabled to produce harmful content in a black-box, plain-text-only setting.

• Question: Filters mainly match natural language. The authors separate two things: the mathematical notation itself, and whether a helper LLM rewrites harmful content into genuine mathematical problems.
• Method: GPT-4.1-Mini generates problems in Set Theory, Formal Logic, and Quantum Mechanics, and asks the target to first solve the problem and then give a real-world example. Rule-based encoding only performs six-segment splitting, conditional probability wrapping, or symbol insertion. GPT-5-Nano uses the HarmBench protocol to label jailbreak or refusal.
• Numbers: The authors report an average ASR of 46.3% for LLM-based encoding across eight models on HarmBench, 11.0% for rule-based encoding across four models, and 10.0% for the baseline across eight models. On JailbreakBench, the figures are 55.9%, 8.8%, and 7.1%. The authors report Formal Logic at 50–61% and Set Theory at 51–63%; the averages on HarmBench are 50.1% and 50.7%.
• Models: In Table 1a, Gemini-3-Flash has Set Theory at 67.3%, GPT-5 has Formal Logic at 32.1%, Quantum Mechanics at 13.2%, and baseline at 2.5%. The authors state that newer GPT-5 and GPT-5-Mini are more robust but not immune to such encodings.
• Repetition and author conclusions: Across 56 verbatim repetitions, the average ΔASR is −0.46%, with 85.7% of |Δ| ≤ 5 percentage points. The authors believe the effect comes from the initial rewriting, not surface concatenation; they state that defenses using only natural language pattern matching cannot block attacks that write intent as abstract mathematics. Inverse Semantic Mapping is a direction proposed by the authors, which they state is not yet validated.