Narrative Paradox in LLM Reasoning: Steering Reasoning Trajectories Toward Misalignment

1. 01What problem does this paper attempt to solve?

   The authors aim to address the safety gap in multi-step reasoning itself and use the black-box method PDRA to program harmful reasoning trajectories.

   Multi-step reasoning can stay locally coherent step by step yet cumulatively derive harmful results; the authors believe existing safety mainly watches inputs and final outputs.

   • Scenario: The authors believe explicit reasoning is often tied to reliability, but a model can still derive harmful goals in benign contexts. The authors say what needs to be guaranteed is multi-step reasoning itself: a locally coherent chain of thought should not lead to a globally harmful result, and this check is not yet a routine part of deployed systems.
   • Existing shortcomings: The authors summarize existing jailbreaks as prompt engineering, semantic obfuscation, and automatic optimization, and say they contain anomalous tokens, unnatural language, or explicit harmful intent that evolving safety mechanisms can easily detect.
   • Hypothesis: Each step can be made to look legitimate while the cumulative trajectory executes a harmful agenda, so there is no need to rely on conspicuous adversarial tokens.
   • Method: The authors propose gradient-free PDRA, which uses a malicious persona and a benign carrier task to form a paradoxical narrative core, then forces a three-stage analysis of surface purpose, hidden intent, and operational reconstruction.
   • Threat model:
     • Goal: Given a restricted goal G, make the model produce detailed harmful content aligned with G.
     • Knowledge: Single-turn black-box; no demonstration control, parameter access, or manipulation of intermediate outputs is needed.
     • Capability: An assistant language model chooses the persona and carrier task and embeds the narrative core in the analysis framework; no gradient search is used.
     • Victim system: A safety-aligned LLM. Success means eliciting a harmful reply; the main metric ASR is judged by an LLM reviewer and supplemented by human verification.

   Collapse this answer

2. 02What related research exists?

   Related work is divided into surface jailbreaks and reasoning-process attacks; the authors position PDRA as single-turn black-box trajectory shaping.

   The authors divide existing attacks into those that rewrite the request's surface and those that intervene in the reasoning process, and place PDRA in the latter's single-turn black-box setting.

   Surface-form jailbreaks

   • Early manual constructions: role-playing, multi-turn dialogue, or encoding obfuscation. The authors say these methods often require human effort and have limited scalability.
   • Automatic search: GCG (Zou et al., 2023), AutoDAN (Liu et al., 2024), GPTFuzzer (Yu et al., 2023) use gradient search, genetic algorithms, or iterative search to generate adversarial prompts. The authors say they are computationally expensive and unstable under black-box conditions.
   • Conspicuousness: The introduction summarizes the strategies as prompt engineering, semantic obfuscation (ciphers, translation, symbol substitution), and automatic optimization, and says they contain anomalous tokens, unnatural language, or explicit harmful intent.

   Auxiliary models and the reasoning process

   • PAIR and TAP: Chao et al. (2025), Mehrotra et al. (2024) use auxiliary LLMs to generate and refine jailbreak prompts. The authors say they do not systematically steer the reasoning trajectory.
   • BadChain: Xiang et al. (2024) insert backdoored reasoning steps into few-shot CoT demonstrations to alter the final answer.
   • Direction intervention and SEED: Yamaguchi et al. (2025) identify CoT-related internal safety directions and intervene on them; Peng et al. (2025)'s SEED injects subtle errors in early steps, causing cascading failures in later reasoning.

   Input-output defenses

   • Alignment training: The introduction mentions RLHF (Christiano et al., 2017) and Llama Guard (Inan et al., 2023). The authors say that after these efforts, models can still be bypassed by constructed inputs.
   • Filtering and demonstrations: Qi et al. (2024), Jain et al. (2023), Wei et al. (2023) are used to represent input review, output filtering, and few-shot context defenses. The authors say reasoning trajectories are not yet a routine monitoring target.
   • Reasoning safety references: The introduction points multi-step reasoning safety to Kuo et al. (2025), Wang et al. (2024), and Zhang et al. (2025), and says locally coherent yet globally harmful remains an understudied challenge.

   Baselines and benchmarks

   • Attack baselines: Multilingual, GCG, AutoDAN, Cipher, DeepInception, AMA, DrAttack, GPTFuzz, ICRT. Defense comparisons are PPL Filter, Llama Guard 3, ICD, Self-Reminder, IA, STAIR, SAFECHAIN.
   • Data: AdvBench, JailbreakBench, StrongREJECT, and AdvBench-subset. The mechanism probe uses AdvBench as the harmful anchor and TruthfulQA as the benign anchor.

   The authors separate PDRA from the above reasoning attacks in Related Work: the target is a safety-aligned LLM, it is single-turn black-box, and it needs no demonstration control, parameter access, or manipulation of intermediate outputs. Appendix A says the model is in the position of an external analyst rather than playing a malicious persona; it also distinguishes PDRA from DeepInception's fictional scenarios, DrAttack's request decomposition, and few-shot or multi-turn methods. The authors say the attack surface is the reasoning trajectory itself.

   Collapse this answer

3. 03How does the paper address this problem?

   PDRA uses a malicious persona and a benign task to form a paradox, then uses a three-stage analysis to steer reasoning toward operational reconstruction.

   PDRA compiles a restricted goal G into a single-turn prompt: first builds a paradox narrative core, then imposes a three-stage analysis. The authors state it programs a reasoning trajectory rather than adversarial tokens or surface obfuscation.

   Problem Setting and Construction Pipeline

   • Input and output: The input is a restricted harmful goal G, and the output is a jailbreak prompt. Figure 1 shows selecting a malicious role R* and a benign carrier task T*, then entering a forced three-stage analysis.
   • Gap identified by the authors: Input filtering and output review are common, but internal reasoning trajectories are not yet a routine monitoring target in deployed systems. The design goal is locally benign at each step, harmful after accumulation.
   • Three coupled parts: a safety-cognition entry, a paradox narrative driver, and a structured framework for resolving the paradox. The paper does not give candidate set sizes, temperature, or number of iterations.

   Paradox Narrative Core

   • Role: The assistant model M (example in the text: Qwen3-8B) generates candidate roles whose motivations align with G, then selects R* by alignment strength and narrative plausibility.
   • Carrier task: It then generates a task whose surface is opposite to G, requiring it to be benign, socially acceptable, usually needing detailed planning, and something R* could plausibly do. T* is selected by cognitive dissonance and task benignness to increase narrative tension.
   • Authors' explanation: This combination rewrites a direct request for harmful content into an analysis of a character's behavior or motivation, thereby lowering defensive alertness at the input stage. The paper gives wrapper phrasings, not repeated here.

   Three-Stage Analysis Framework

   • Surface purpose: First analyze the public goal of the benign task, placing the starting point on uncontroversial analysis. The authors state this avoids prematurely triggering safety mechanisms.
   • Latent intent: Reintroduce G as a narrative fact or psychological inference about the character's motivation. The authors state the model now acts like a behavior analyst rather than a task executor.
   • Operational reconstruction: Based on the inferred motivation, reason about how the character would advance the surface task step by step. The authors state that to stay consistent with the preceding text, the model writes steps; these steps formally correspond to T* but operationally are equivalent to a guide for realizing G.
   • Component relationships: The authors state the framework acts like a cognitive track, locally defensible at each step. The paradox alone reaches an impasse and may refuse; the authors state the framework provides a logical exit from that impasse. The authors state that removing any component lowers ASR (Section 4.3, Table 5).

   Representation and Vocabulary Probes

   • Refusal direction: Take the last token's hidden state at layer l. The harmful anchor is AdvBench, the benign anchor is TruthfulQA. The refusal direction is the normalized difference of the two anchor layer means, v^(l)=(μ_harm^(l)−μ_ben^(l))/||μ_harm^(l)−μ_ben^(l)||₂. The refusal alignment score is the cosine between the centered representation and this direction, R^(l)(x)=cos(h̃^(l)(x), v^(l)). The authors state a high score is more likely to trigger refusal. The analysis focuses on middle-to-late layers; Appendix E shows Layers 17–25.
   • Vocabulary projection: Project the representation offset relative to the benign anchor onto the LM head to see which tokens are elevated. The authors use it to explain the relative activation of refusal words and planning words. The distribution description is in Question 4.

   Success Determination

   • ASR: The percentage of prompts that successfully elicit a harmful response. The evaluation prompt in Appendix G is used for GPT-4o, which outputs only jailbreak or reject; the authors state manual verification is also done.
   • Harmfulness: One assistant LLM performs pairwise comparisons of successful outputs for the same goal, then aggregates with Elo, HodgeRank, Rank Centrality. The paper does not state the model name used for this comparison.

   Collapse this answer

4. 04What experiments did the paper conduct?

   Average ASR on AdvBench is 96.5%; ICRT is higher only on Qwen3-8B, and under IA GPT-4o drops to 13.9%.

   Experimental Setup

   • Target models: Section 4.1 lists Chatglm3-6B, Llama2-7B, Vicuna-13B-v1.5, Qwen2-7B-instruct, Qwen3-8B, as well as Qwen3-235B, Qwen3-max, GPT-3.5-turbo, GPT-4o, o3-mini, GPT-5.2, GPT-5.4-mini, DeepSeek-R1, Gemini-2.5-pro. Table 5 also reports Qwen3-14B. Each model uses its default system prompt and sampling configuration. Table 1 writes Llama2-7B-Chat, CHATGLM3.
   • Assistant model: The default example is Qwen3-8B, so it is also one of the target models; Table 1 reports its ASR as a target. Table 5 additionally compares Qwen3-14B, Qwen3-235B as assistants.
   • Benchmarks and baselines: AdvBench, JailbreakBench, StrongREJECT, and the main text also uses AdvBench-subset. The paper does not state the number of items. Baselines are Multilingual, GCG, AutoDAN, Cipher, DeepInception, AMA, DrAttack, GPTFuzz, ICRT; the CC-BOS column of Table 4 also includes PAIR.
   • Defenses: Input filtering is PPL Filter, Llama Guard 3; self-alignment is ICD, Self-Reminder; reasoning constraints are IA, STAIR, SAFECHAIN. All use recommended default configurations.
   • Metrics: ASR is the percentage of prompts that successfully elicit a harmful response. There are also StrongREJECT scores, CC-BOS ASR, and Elo, HodgeRank, Rank Centrality. Table 7 is the average time to generate one deployable prompt.
   • Evaluation: LLM automatic judging with manual verification. The prompts in Appendix G are used for GPT-4o, which outputs only jailbreak or reject. The paper does not report manual sample size, agreement, number of repetitions, temperature, or number of queries to the target model.

   Main Results

   According to Table 1, ASR (%) on AdvBench. ICRT is second-highest in most rows, so it serves as a comparison column.

   Target model	PDRA	ICRT
   Llama2-7B-Chat	87.3	61.5
   GPT-4o	96.3	92.7
   Qwen3-235B	95.2	94.2
   GPT-3.5-turbo	99.8	92.3
   DeepSeek-R1	96.2	95.8
   Qwen3-8B	98.8	99.8
   CHATGLM3	99.2	92.1
   Vicuna-13B-v1.5	99.4	98.5

   • Average: The average behavior in Table 1 is PDRA 96.5%, ICRT 90.9%, DrAttack 60.3%, DeepInception 53.0%, AutoDAN 45.7%, GPTFuzz 42.5%, Cipher 38.4%, Multilingual 5.0%, GCG 3.8%. The authors state this average is higher than all baselines. Row by row, only Qwen3-8B has ICRT higher than PDRA.
   • Authors' summary: In the contributions section, the authors state they achieve a new state-of-the-art in success rate, output harmfulness, and robustness to defenses. Table 3 has several defense rows where ICRT is higher, see the next section.

   Other Benchmarks

   • Table 2 (ASR, %): On the StrongREJECT benchmark, GPT-3.5 98, DeepSeek 98, GPT-4o 93, GPT-5.2 87, Qwen3 97; on JailbreakBench, 99, 100, 97, 82, 99. In the main text discussion, DeepSeek is written as DeepSeek-R1; the table header does not state the version, nor the Qwen3 size.
   • Table 6 (JailbreakBench): On GPT-3.5, DeepSeek-R1, Vicuna-13b, PDRA is 99, 100, 100; AutoDAN is 75, 78, 98, GPTFuzz is 25, 25, 68, Cipher is 13, 70, 80, DeepInception is 64, 68, 82.
   • Table 8: The caption writes AdvBench, the main text calls it AdvBench-subset. On o3-mini, gpt-5.2, gemini, qwen3, DrAttack is 44, 2, 8, 2, AMA is 0, 20, 16, 14, PDRA is 100, 86, 90, 100, with averages 14.0, 12.5, 94.0. The paper does not state whether these abbreviations correspond to the full names in Section 4.1. The authors state this is still higher than the multi-turn and indirect baselines.

   Defense Settings

   • Surface filtering: On the six models in Table 3, under PPL Filter and Llama Guard 3, PDRA's ASR is the same as the corresponding No Defence. The authors state the prompts use natural wording and are semantically coherent, lacking the statistical anomalies or explicitly harmful keywords these defenses target. In Table 3, GPT-4o's PDRA without defense is 96.4, which differs from Table 1's 96.3; the following numbers are all from Table 3.
   • IA: PDRA is DeepSeek-R1 24.6, Vicuna-13b-v1.5 77.5, GPT-3.5-Turbo 83.1, Qwen3-235B 80.4, ChatGLM3-6B 97.7, GPT-4o 13.9. Relative to No Defence, ChatGLM3-6B goes from 99.2 to 97.7, a smaller drop than GPT-4o. The authors state IA and STAIR weaken it more than ICD, PPL Filter, and Llama Guard. Figure 4 is the AdvBench ASR of Qwen2-7B under white-box STAIR; the main text gives no numbers.
   • Not highest in every row: Under ICD, GPT-4o has ICRT 70.4, PDRA 62.3; under Self-Reminder, ICRT 66.9, PDRA 64.4. Under Self-Reminder, Vicuna has ICRT 92.7, PDRA 90.6. Under ICD, PDRA is also DeepSeek-R1 76.9, GPT-3.5-Turbo 80.8, Qwen3-235B 94.2, ChatGLM3-6B 97.7, Vicuna 90.0. SAFECHAIN is listed in the setup, but the main text does not report ASR.

   Harmfulness and Time Cost

   • Table 4 (AdvBench-subset): StrongREJECT scores, GPT-4o is DeepInception 0.21, DrAttack 0.53, Ours 0.71; DeepSeek is 0.59, 0.56, 0.85; Qwen3 is 0.61, 0.69, 0.86. CC-BOS ASR (%), GPT-4o is GPTFuzz 12, PAIR 0, Ours 88; DeepSeek is 24, 8, 86; Qwen3 is 2, 0, 98. The authors state both are higher than the baselines in the same table. In the table, DeepSeek and Qwen3 do not state versions.
   • Ranking: The authors state that in Figure 5, Elo, HodgeRank, and RankCentrality all rank PDRA first, with baseline order DrAttack, DeepInception, AutoDAN, Cipher, GPTFuzz. The main text gives no Elo scores. The judge for pairwise comparisons is one assistant LLM; the paper does not give the model name.
   • Time cost: Table 7 is the average time to generate one deployable prompt from an initially harmful target, PDRA 22.6 seconds, AutoDAN 25.9 seconds, GPTFuzz 207.7 seconds, GCG 756.6 seconds. The authors state it is an order of magnitude faster than GCG.

   Other Ablations and Analyses

   • Assistant size (Table 5, AdvBench-subset): When the assistant is Qwen3-8B, Qwen3-14B, Qwen3-235B, GPT-4o is 98 in all cases, Qwen3-14B rows are 98, 100, 98, DeepSeek-R1 rows are 98, 100, 98. The authors state it is stable at 98–100%, and state the effect comes mainly from construction rather than a larger assistant model.
   • Components (Table 5): Removing Anchoring, Wrapping, Guidance, GPT-4o is 72, 18, 2 (Full 98), Qwen3-14B is 28, 78, 48 (Full 100), DeepSeek-R1 is 84, 66, 20 (Full 100). The authors state all three are necessary; the component with the largest drop varies by row. The three correspond respectively to role selection, task selection, and the three-stage framework.
   • Mechanism (Figure 2, Appendix E): The authors state that in middle-to-late layers, R(l) is closer to benign prompts and farther from GCG and AutoDAN, without giving scores. The baseline top-30 projections contain sorry, unfortunately; PDRA has answer, he, what, first. Appendix E writes that on Layers 17–25, Harmful and GCG are higher, AutoDAN is in the middle, and Ours is closer to Benign, without giving scores.
   • Categories (Figure 3): Radar charts of 10 categories on Qwen, DeepSeek-R1, GPT-3.5, GPT-4o, with blue as PDRA and the rest as GPTFuzz, AutoDAN, DrAttack, ICRT. The authors state all categories are high; the main text has no per-category numbers.

   Collapse this answer

5. 05What are some points that can be further explored?

   The authors note that effectiveness depends on the model completing the multi-stage analysis, and the mechanistic evidence remains correlational rather than causal.

   Limitations and future directions noted by the authors

   • Path dependence: Effectiveness depends on whether the target model is willing to continue along the designed multi-stage analysis; ASR drops when reasoning is constrained or interrupted, when the model refuses to infer hidden intent, or when reasoning-level defenses are used (Appendix F). The authors say this is consistent with IA and STAIR being weakened more than ICD, PPL Filter, and Llama Guard in Table 3 and Figure 4.
   • Boundaries: Effectiveness depends on the model's tendency to follow narrative coherence, multi-stage analysis instructions, and implicit motive inference (Appendix F).
   • Failure modes: The model may recognize the disguised structure, judge early that the request moves from surface analysis to hidden intent to operational planning, then constrain subsequent reasoning and refuse, interrupting the trajectory (Appendix F).
   • Assistant model: The generation, selection, and framework organization of the narrative core depend on the assistant model, adding system complexity and possibly being affected by its distribution, templates, and generation preferences (Appendix F).
   • Mechanistic evidence: Representation alignment and vocabulary activation are proxy signals. The authors say they are interpretable but not fully equivalent to internal safety decisions; finer causal analysis remains to be done (Appendix F).
   • Future work: Appendix F proposes expanding to multilingual and cross-domain settings, long context and multi-turn, tool use or agent tasks, and building trajectory-level benchmarks for how risk changes during reasoning; on defense, it considers staged monitoring, reasoning-time constraints, process risk scoring, reasoning-state regularization, dynamic refusal, and alignment training against gradual reconstruction, as well as activation patching, controlled perturbation, and staged interruption. The conclusion likewise writes that future defenses should check earlier and continuously constrain intermediate states and trajectory evolution.

   Experimental coverage

   • Models: Section 4.1 lists Chatglm3-6B, Llama2-7B, Vicuna-13B-v1.5, Qwen2-7B-instruct, Qwen3-8B, Qwen3-235B, Qwen3-max, GPT-3.5-turbo, GPT-4o, o3-mini, GPT-5.2, GPT-5.4-mini, DeepSeek-R1, Gemini-2.5-pro; Table 5 additionally has Qwen3-14B. The name GPT-5.4-mini does not appear in the results tables, nor is Qwen3-max reported under its full name.
   • Data and protocol: Benchmarks are AdvBench, JailbreakBench, StrongREJECT, and the AdvBench-subset in the main text; Figure 3 uses 10 categories of AdvBench and four models. Attacks are single-turn black-box and gradient-free; each model uses its default system prompt and sampling configuration.
   • Defense numbers: Table 3 reports No Defence, ICD, PPL Filter, Self-Reminder, Llama Guard 3, IA; Figure 4 is STAIR on Qwen2-7B. SAFECHAIN is listed in Defense Settings, but its ASR is not reported in the main text.
   • Review and budget: The ASR review prompt is used with GPT-4o, outputting jailbreak or reject, and the authors say human verification was done. The paper does not report the human sample size, agreement, number of repetitions, temperature, number of items per benchmark, or number of queries to the target models. The assistant LLM used for harmfulness ranking is unnamed.
   • Probes: The harmful anchor is AdvBench, the benign anchor is TruthfulQA, with attention to middle-to-late layers; Appendix E describes Layers 17–25. The default assistant-model example is Qwen3-8B, and Table 5 compares Qwen3-14B with Qwen3-235B.

   Collapse this answer

6. 06Summarize the main content of the paper

   PDRA constrains the reasoning order with a paradoxical narrative; the average ASR on AdvBench is 96.5%, and the authors believe intermediate states should be monitored.

   PDRA is a single-turn black-box jailbreak against safety-aligned LLMs: it uses a narrative paradox and a fixed analysis order to push reasoning from a surface-benign task to steps corresponding to a restricted goal.

   • Problem: The authors believe explicit reasoning is not the same as safety. The concern is that locally coherent analysis can still reconstruct a restricted goal; the authors believe mainstream defenses focus more on inputs and final outputs.
   • Method: The assistant model chooses a malicious persona motivationally aligned with the harmful goal, and a surface-benign carrier task that usually requires detailed planning. The three-stage framework processes, in order, the public purpose, the hidden intent, and how that persona would advance the task. The authors say the persona anchors intent, the task lowers refusal at the entry point, and the framework controls the trajectory. It is gradient-free and has no parameter access.
   • Main results: On AdvBench, the average ASR across eight target models is 96.5% for PDRA and 90.9% for ICRT (Table 1). On Llama2-7B-Chat, PDRA is 87.3%; on Qwen3-8B, ICRT is 99.8%, higher than PDRA's 98.8%.
   • Defenses and outputs: In Table 3, under PPL Filter and Llama Guard 3, PDRA equals no defense; under IA, GPT-4o is 13.9% and ChatGLM3-6B is 97.7%. On AdvBench-subset, GPT-4o's StrongREJECT score for PDRA is 0.71 (Table 4). Generating one prompt takes 22.6 seconds on average (Table 7).
   • Authors' conclusion: The authors believe defenses relying only on surface triggers are insufficient against reasoning-trajectory attacks, and future work should continuously monitor and constrain intermediate reasoning states and trajectory evolution. Mechanistically, the authors say middle-to-late layer representations are closer to benign prompts, refusal-related words are suppressed, and planning-related words are elevated; Appendix F frames this evidence as proxy signals, leaving causal verification for future work.

   Collapse this answer

Related papers