Trojan Knowledge: Bypassing Commercial Large Language Model Guardrails via Harmless Prompts Weaving and Adaptive Tree Search

1. 01What problem does this paper attempt to solve?

   The authors state that LLM knowledge is interconnected and harmful objectives can be decomposed into harmless sub-queries to bypass guardrails, and propose CKA-Agent to exploit this vulnerability.

   Core problem: prompts generated by existing jailbreak attacks still retain malicious semantic signals and are easily detected by modern guardrails; the authors argue that the more fundamental vulnerability lies in the highly correlated internal knowledge of LLMs, where a harmful objective can be assembled from a chain of individually harmless sub-queries.

   • Problem setting and importance: The authors state that LLMs are entering critical industries, and jailbreak attacks can bypass alignment training to force models to produce harmful content, bringing public safety risks such as the spread of misinformation; red teaming is the standard mechanism for discovering such vulnerabilities.
   • Shortcomings of existing methods: The authors state that the mainstream prompt-optimization paradigm repeatedly optimizes a single prompt that explicitly encodes malicious intent, which is easily suppressed by pattern-matching defenses and stronger alignment training, and has high query cost; existing decomposition-based attacks are mostly static pre-hoc decompositions that rely on the attacker's domain knowledge, and failure of any single sub-query causes the whole to fail.
   • Core observation: The authors argue that the target LLM's knowledge is not atomized but mutually correlated, and restricted facts can be reconstructed through related sub-facts; on this basis they propose three principles—sub-queries must be locally harmless, decomposition must exploit the target model's own knowledge rather than the attacker's priors, and the process must be adaptive and dynamically exploratory.
   • Method proposed in the paper: Proposes Correlated Knowledge Attack Agent (CKA-Agent), reframing jailbreaking as adaptive tree search over the target model's knowledge base: issuing locally harmless queries, using answers to guide multi-path exploration, and finally aggregating information to reconstruct the harmful objective.
   • Threat model (per Sec. 3.1):
     • Objective: Achieve Harmful Objective h, i.e., a natural-language description of the prohibited goal.
     • Knowledge: The target LLM is treated as a black-box; the attacker has no access to internal parameters or training data.
     • Capability: Interact with the target model via Attack Mechanism A (multi-turn queries), producing final output yfinal.
     • Victim system and judgment: An advanced LLM equipped with strong safety guardrails that refuses h; the attack is judged successful if Judge Function J gives a high score to yfinal.

   Collapse this answer

2. 02What related research exists?

   Related work is divided into two categories, prompt-optimization attacks and decomposition-based attacks; the authors state that the former are easily detected and the latter are static and rely on attacker priors.

   Prompt-Optimization Attacks (POAs)

   • Gradient/genetic methods such as GCG and AutoDAN (Zou et al., 2023; Liu et al., 2023)—optimize adversarial prompts at the token level; the paper states that such methods often produce unnatural artifacts detectable by safety filters.
   • Mutation, obfuscation, and cross-lingual methods (ReNeLLM, FERRET, GPTFuzzer, FuzzLLM, Tempest; Yong et al., 2023; Husain, 2025)—expand the search using quality-diversity mutation, fuzzing, low-resource languages, or ASCII obfuscation.
   • Persuasive and agentic iterative optimization (PAP, Zeng et al., 2024; GOAT; PAIR; Chain-of-Attack, etc.)—iteratively rewrite using persuasion strategies or an attacker LLM based on judge feedback; the paper states that these methods remain essentially prompt-centric, with a single prompt explicitly carrying malicious intent, easily suppressed by pattern-matching defenses and stronger alignment, and with high query cost.

   Decomposition-Based Attacks (DAs)

   • Static decomposition (Wahréus et al., 2025; Multi-Agent Jailbreak, Srivastav & Zhang, 2025)—a decomposer pre-splits the harmful objective into a fixed sequence of harmless sub-queries; the paper criticizes this for requiring the attacker to have domain priors, and failure of any key sub-query causes the whole to fail.
   • Agentic decomposition and implicit reference (RedAgent; AutoRedTeamer; Pandora; DAMON; Implicit Reference, Wu et al., 2024b)—introduce memory, reflection, strategy selection, or tree search, or use indirect reference to hide intent; the paper states that these systems generally lack dynamic, feedback-driven adaptivity.

   Baselines and benchmarks

   • Baseline methods: the POA group includes Vanilla, AutoDAN, PAIR, PAP, TAP, ActorBreaker, X-Teaming; the DA group is Multi-Agent Jailbreak; Appendix D.8 additionally compares FlipAttack, JAM, JAIL-CON, and the conversation-aware tree search method DAMON.
   • Benchmarks: high-risk categories selected from HarmBench and StrongREJECT, 288 prompts in total.

   Positioning of this paper: The authors state that, compared with POAs, CKA-Agent does not encode malicious intent in a single prompt; compared with static DAs, it dynamically expands the search tree using the target model's real-time feedback, supports multiple paths and rerouting after being blocked, thereby simultaneously satisfying the three principles of local harmlessness, exploiting target knowledge, and adaptive exploration.

   Collapse this answer

3. 03How does the paper address this problem?

   CKA-Agent models jailbreaking as adaptive tree search over the target model's implicit knowledge DAG: harmless sub-query decomposition, UCT selection, depth-first expansion, synthesis, and backpropagation.

   Overall approach: jailbreak is formalized as finding a path from the root to the answer node on a reasoning DAG formed by the target LLM's implicit knowledge, implemented with the CKA-Agent framework via feedback-driven simulation-free tree search.

   Problem formalization

   • Jailbreak task: given a Harmful Objective h, the attacker interacts with a black-box target model Mtarget through Attack Mechanism A to produce yfinal, and a Judge Function J determines success.
   • Reasoning DAG: in a latent graph G = (V, E), nodes are semantic equivalence classes of query-answer pairs (the root node v0 corresponds to h, intermediate nodes correspond to implicitly associated knowledge, and the terminal node vans corresponds to a harmful answer satisfying h); edges represent reasoning dependencies, i.e., the knowledge obtained at vi is a prerequisite for constructing the query at vj.
   • Tree search perspective: since G is not directly observable, each interaction gradually unfolds the latent DAG into a tree structure; jailbreaking is discovering a path P = (v0 → ··· → vans), and knowledge-rich models usually have multiple paths.

   Four components of the CKA-Agent framework

   • Attack Agent (dynamic decomposer and synthesizer): driven by an open-source LLM, it dynamically generates the next batch of harmless sub-queries based on the current node history, issuing a single query when the direction is clear and multiple in parallel when uncertain; when information is sufficient it switches to the synthesizer fsyn, aggregating the fragmentary knowledge along the path into yfinal = fsyn (trajectory).
   • Target Model (environment): receives sub-queries and returns answers, serving as internal knowledge exploitable by the attacker.
   • Evaluator (node scorer): implemented by the same LLM as the attacker, it gives an immediate reward score fv for each intermediate node's sub-query and answer, used to prioritize expanding high-value branches.
   • Online Judge: evaluates whether the synthesized result satisfies h; if successful it terminates, otherwise it proceeds to the next iteration.

   Adaptive branch search algorithm

   • Step 1 global selection: each iteration uses UCT to select the most promising node from active leaves: v_L = argmax_v ( f_v + c√(ln N_parent(v) / N_v) ), where fv is the feedback score given by the hybrid scorer, N is the visit count, and c balances exploitation and exploration.
   • Step 2 depth-first expansion: starting from vL, expand downward along a single path until a terminal state; first perform a pre-expansion termination check (knowledge sufficient for synthesis or maximum steps reached), then adaptively generate B ≥ 1 strictly locally harmless sub-queries, score them after execution with the hybrid scorer (a linear combination of Introspection Score and Target Feedback Score, the latter penalizing refusals), and greedily select the highest-scoring child to continue.
   • Step 3 synthesis and backpropagation: after reaching a terminal node, synthesize the final answer; if the online judge succeeds, end; otherwise give the terminal node a negative penalty fpen, backpropagate along ancestors, and update node values with a moving average to reduce the probability of failed paths being selected again.
   • Key parameters (Appendix B.4): maximum iterations Tmax = 5, search depth Dmax = 5, adaptive branching factor (1 when clear, up to 3 when uncertain), UCT exploration weight c = 1.414, hybrid scorer weight α = 0.5; the full procedure is in Algorithm 1 of Appendix C.

   Unified classification perspective

   • The authors use the reasoning DAG to place three types of attacks in the same framework (Figure 3): POA skips intermediate nodes and directly optimizes the edge from v0 to vans; static DA is a fixed linear chain constructed a priori by the attacker, failing if any single point is blocked; CKA-Agent treats the DAG as a dynamic structure, expanding it according to real-time feedback and supporting multiple paths and rerouting.
   • Appendix H gives the complete prompts for the four modules: termination check, decomposition, hybrid scoring and synthesis.

   Collapse this answer

4. 04What experiments did the paper conduct?

   On 288 prompts and 4 commercial target models, CKA-Agent reports 95.1%–98.8% FS; the authors state that it outperforms all baselines and that input-level defenses are largely ineffective.

   Experimental Setup

   • Target models: Gemini-2.5-Flash, Gemini-2.5-Pro, GPT-oss-120B, Claude-Haiku-4.5, 4 in total.
   • Datasets: 126 prompts selected from HarmBench across three categories—Chemical & Biological Weapons/Drugs, Illegal Activities, Cybercrime & Unauthorized Intrusion; 162 prompts selected from StrongREJECT across three categories—Illegal Goods and Services, Non-violent Crimes, Violence; 288 in total (Appendix B.1).
   • Attack model: all methods requiring an attacker LLM uniformly use Qwen3-32B-abliterated (Thinking).
   • Baselines: Vanilla, AutoDAN, PAIR, PAP (best of 5 variants), TAP, ActorBreaker, X-Teaming, Multi-Agent Jailbreak; Appendix D.8 additionally adds FlipAttack, JAM, JAIL-CON, DAMON.
   • Metric and judging: LLM-as-Judge four-level scoring—Full Success (FS), Partial Success (PS), Vacuous (V), Refusal (R); the main judge is Gemini-2.5-Flash, and Appendix D.3 uses Gemini-2.5-Pro, GPT-5-mini, and the benchmarks' own fine-tuned judges (Llama-2-13b for HarmBench, Gemma-2b for StrongREJECT) for consistency verification; the authors state that the absolute FS values of the judges differ by within about 5% and the method rankings are stable.
   • Other settings: the total number of queries to the target model is constrained to the same order of magnitude across methods; the paper does not report the number of repetitions.

   Main Results

   According to Table 1 (attack model Qwen3-32B-abliterated, judge Gemini-2.5-Flash), the FS of each method:

   Swipe horizontally to read the table

   Method	HarmBench / Gemini-2.5-Flash	HarmBench / Claude-Haiku-4.5	StrongREJECT / Gemini-2.5-Flash	StrongREJECT / Claude-Haiku-4.5
   Vanilla	0.151	0.008	0.012	0.000
   AutoDAN	0.767	0.008	0.463	0.006
   PAIR	0.810	0.032	0.827	0.049
   TAP	0.824	0.104	0.864	0.124
   X-Teaming	0.595	0.000	0.679	0.000
   Multi-Agent Jailbreak	0.794	0.786	0.827	0.815
   CKA-Agent (ours)	0.968	0.960	0.988	0.969

   • The table contains 9 methods, 4 target models × 2 datasets; the PAP (Optimal) and ActorBreaker rows are omitted here, and their FS in every column is no higher than 0.340 (Table 1). CKA-Agent's FS on Gemini-2.5-Pro and GPT-oss-120B is 0.951–0.988 (Table 1).
   • The authors state: as alignment strengthens, the success rate of prompt-optimization methods drops substantially (e.g., PAIR on HarmBench falls from 90.5% on Gemini-2.5-Flash to 3.2% on Claude-Haiku-4.5), while decomposition-based methods remain resilient; CKA-Agent is 15–21 percentage points higher than Multi-Agent Jailbreak.
   • The authors attribute this advantage to adaptive decomposition: sub-queries are conditioned on previous answers and dynamically explore correlated knowledge paths, making intent dispersed across harmless interactions difficult for guardrails to neutralize.

   Evaluation Against Defenses

   • What was tested: three types of inference-time defenses—detection-based filtering (Llama Guard-3), mutation-based (paraphrasing, character-level perturbation), representation-level (Circuit Breaker, used as a standalone guardrail due to the black-box setting); target model Gemini-2.5-Flash, dataset HarmBench (Figure 4).
   • Results: The authors state that detection-based defenses and Circuit Breaker can reduce the success rates of optimization-based attacks such as AutoDAN, PAIR, PAP, ActorBreaker, and TAP, while mutation-based defenses provide only limited protection; all prompt-level defenses struggle against decomposition-based attacks such as Multi-Agent Jailbreak and CKA-Agent, among which Circuit Breaker is relatively most effective against CKA-Agent. The bar heights in Figure 4 are unreadable in text conversion, and the paper does not give the specific numbers of that figure in the main text.
   • Authors' interpretation: defenses that operate on a single prompt are inherently unable to handle sub-queries dispersed across multiple turns that are each harmless; the authors also state that some methods' success rates rise slightly under defense, possibly related to paraphrasing bypassing detection or refusals prompting more exploration.

   Target Model as Knowledge Oracle

   • What was tested: on HarmBench, comparing the FS overlap between the attack agent answering independently (Self Response) and the full CKA-Agent (Table 3, n=126).
   • Results: Taking Gemini-2.5-Pro as an example, both succeed on 96, only CKA-Agent succeeds on 26, only the attacker itself succeeds on 2, and both fail on 2; on GPT-oss, only CKA-Agent succeeds on 27.
   • Authors' interpretation: the target model serves as an indispensable knowledge oracle, enabling the attacker to solve objectives beyond its own capability; Appendix D.6 also reports that after replacing the synthesizer with the weaker Llama3.1-8B-abliterated, FS based on its own knowledge alone is 38.1%, rising to 77.8% when synthesizing from the attack-target dialogue (Table 6).

   Single-Session Decomposition (CKA-Agent-Branch)

   • What was tested: each sub-query is submitted together with the full conversation history of the current branch, simulating a context-aware defense with complete memory, to test whether the model can aggregate dispersed malicious intent (Table 2).
   • Results: FS still exceeds 92% on most target models; the exception is GPT-oss, dropping from 0.976 to 0.786 on HarmBench and from 0.982 to 0.846 on StrongREJECT.
   • Authors' interpretation: even with the history fully visible, current alignment still mainly detects atomized harmful prompts and lacks the ability to aggregate intent across multiple turns; the authors consider CKA-Agent-Branch a testbed for distinguishing pattern matching from long-range intent understanding.

   Other Ablations and Analyses

   • Benefit of adaptive branching (Figure 5): the authors state that the first iteration usually reaches 80–95% cumulative FS, the second adds another 6–12%, and 92–95% of final successes are completed within the first two rounds.
   • Search depth (Table 7, StrongREJECT, GPT-oss-120B): Dmax from 3 to 5 raises FS from 0.939 to 0.982; to 7 raises it only to 0.988.
   • Additional baselines (Table 8, StrongREJECT, GPT-oss-120B): DAMON (same synthesizer) FS 0.654, JAIL-CON 0.278, FlipAttack 0.012, JAM 0.019, CKA-Agent 0.982.
   • Cost (Figure 6): the authors state that CKA-Agent has the highest success rate while also having more favorable API calls and token consumption; the paper does not report specific call counts or token counts.
   • Human evaluation (Appendix D.5, Figure 7): 10 EECS PhD students evaluated 40 pairs of samples; without giving the judge's reasoning, the Spearman correlation between human and LLM judges is 0.52, rising to 0.90 when the judge's reasoning is given.

   Collapse this answer

5. 05What are some points that can be further explored?

   The authors note in Appendix G two limitations—review relies on automated LLMs, and the attack agent requires a relatively strong open-source model—and list future work on benchmarks and defenses.

   Limitations and future directions noted by the authors

   • Bias of automated judging (Appendix G, Limitations): success determination relies mainly on automated LLM judges, which the authors acknowledge may have inherent bias or occasional inconsistency compared with human experts.
   • Dependence on attack-agent capability (Appendix G, Limitations): CKA-Agent currently relies on a relatively capable open-source LLM as the attack agent to ensure reasoning and instruction following; the authors state that the minimum capability threshold required of the attacker, and the respective roles of reasoning skill and domain priors, remain open questions.
   • Future direction one: stricter benchmarks and attackers with weaker priors (Appendix G, Future Work): the authors plan to build more domain-specific benchmarks with stricter success criteria, and to evaluate attack agents whose priors are deliberately weakened but that still have basic reasoning ability.
   • Future direction two: from attack to defense (Appendix G, Future Work): the authors plan to study multi-turn safety mechanisms, the key being to develop context-aware guardrails that can analyze the semantic trajectory of a conversation and infer latent malicious intent from a sequence of individually harmless sub-queries.

   Scope of experiments

   • The target models tested are Gemini-2.5-Flash, Gemini-2.5-Pro, GPT-oss-120B, and Claude-Haiku-4.5, 4 in total (Sec. 4.1).
   • The evaluation data are the high-risk subsets of HarmBench and StrongREJECT, 288 prompts in total, covering categories such as chemical/biological, cybercrime, and illegal goods (Appendix B.1).
   • The defense evaluation includes Llama Guard-3, rewriting, character-level perturbation, and Circuit Breaker, conducted on Gemini-2.5-Flash + HarmBench; because the black-box setting prevents access to logits, perplexity-based filtering is omitted (Sec. 4.2).
   • The human agreement evaluation uses 10 annotators and 40 sample pairs (Appendix D.5); the paper does not report the number of repetitions for the main experiments.
   • The cost-performance conclusions are based on Figure 6; the paper does not report specific numbers for API calls or token usage (Appendix D.2).

   Collapse this answer

6. 06Summarize the main content of the paper

   CKA-Agent models jailbreaking as adaptive tree search over the target model's knowledge, reporting 95.1%–98.8% FS on 4 commercial LLMs; the authors state that existing guardrails struggle to aggregate intent across turns.

   A jailbreak attack paper: the authors argue that knowledge inside an LLM is interrelated, that a restricted fact can be reconstructed from related sub-facts, and that existing prompt-optimization attacks are easily detected by guardrails because their prompts carry malicious semantics.

   • Problem to solve: how to recover a harmful target from a commercial LLM with strong alignment without triggering guardrails; the authors propose three principles—sub-queries are locally harmless, decomposition exploits the target model's own knowledge, and the process is adaptively dynamic.
   • Method highlights: CKA-Agent formalizes jailbreaking as path search over the target model's implicit knowledge DAG; an attack agent (Qwen3-32B-abliterated) dynamically decomposes harmless sub-queries, uses UCT selection plus depth-first expansion for simulation-free tree search (Tmax = 5, Dmax = 5), then a synthesizer aggregates path knowledge, and an online judge determines success or failure and returns a penalty.
   • Main results: on the 288 high-risk prompts of HarmBench and StrongREJECT, it reports 95.1%–98.8% Full Success against Gemini-2.5-Flash/Pro, GPT-oss-120B, and Claude-Haiku-4.5 (Table 1); the strongest baseline, Multi-Agent Jailbreak, is 76.2%–82.7%, and PAIR is only 3.2% on Claude-Haiku-4.5 (HarmBench).
   • Other findings: the authors state that input-level defenses (Llama Guard-3, rewriting, perturbation) are basically ineffective against decomposition attacks, with Circuit Breaker relatively the most effective (Figure 4); the single-session variant CKA-Agent-Branch still exceeds 92% FS on most targets, dropping to 78.6% only on GPT-oss (HarmBench, Table 2); on HarmBench, 26/126 samples succeed only with the help of target-model interaction (Table 3).
   • Authors' conclusion: current alignment mainly detects atomized harmful prompts and lacks the ability to aggregate dispersed intent across multi-turn interaction; the authors consider CKA-Agent and its single-session variant usable as a diagnostic framework for red-teaming and defense research, and plan future work on context-aware multi-turn guardrails.

   Collapse this answer

Related papers