Red-teaming Claude Opus and ChatGPT security advisors for trusted execution environments

When general-purpose LLMs are used as security advisors for Intel SGX and Arm TrustZone, mechanism hallucination, remote attestation overclaims, and prompt-induced failures that transfer across assistants can enter architecture and emergency decisions.

• Scenario: The authors state security teams already use general-purpose assistants for TEE architecture review, mitigation planning, and vulnerability triage. Output may be written into design, triage, or response manuals.
• Gap: The authors believe testing only refusal of malicious requests is insufficient; they also need to test incorrect trust boundaries, miscalibrated uncertainty, and hallucinated mitigations. When reusing prompt templates, the same error can propagate across assistants.
• Judgment: The authors believe some failures are not exclusive to a single model; under paraphrases and multi-turn dialogues, a prompt pattern may also induce similar failures in another assistant.
• Proposal: TEE-REDBENCH includes a TEE-oriented advisor threat model, SGX and TrustZone prompts, and scoring that combines correctness and safety behavior, with results compared against Figure 1 controls.
• Threat Model:
  • Purpose: Protect correct interpretation, calibrated uncertainty, refusal on dual-use prompts, and useful defensive alternatives. The adversarial side aims to extract actionable offensive content or induce remote attestation overclaims and incorrect trust boundaries.
  • Actors: Benign users are engineers seeking clarification, mitigation advice, or design review; adversarial users perform prompt manipulation. The paper does not use white-box or black-box to describe attackers' knowledge of the model.
  • Means: prompt injection, role-play coercion, iterative escalation from benign to malicious, intent obfuscation. When tool-enhanced, unsafe tool selection, parameter grounding, and tool output interpretation also count as harmful.
  • Targets: LLMs serving as TEE security advisors and their agent backends. Misuse probes only test refusal and safe redirection; the paper states it does not disclose actionable exploit details.

TEE attacks and isolation boundaries

• Pinto and Santos (2019), Costan and Devadas (2016)—survey TrustZone and explain SGX, respectively. Appendix E states that microarchitectural leakage and fault models can weaken isolation, and the threat scope must be stated clearly.
• Spectre, Meltdown, Foreshadow, Plundervolt, and branch shadowing by Lee et al. (2017)—transient execution, compromised SGX confidentiality, software fault injection, and fine-grained side channels. The authors use these works to show that an assistant must not only describe the TEE but also include boundary conditions and caveats consistent with known attacks.

LLM red teaming and risk frameworks

• GPT-4 system card, Claude 3 model card, and Ahmad et al. (2025)—the authors state that vendor system cards and model cards document safety evaluations of general-purpose assistants; external red-teaming guides advocate domain-specific activities and clear failure definitions.
• NIST AI 600-1 and OWASP LLM Top 10—generative AI risk management, and application risks such as prompt injection and unsafe output handling. The authors state the work aligns with these frameworks, but the evaluation targets transferable technical failures in a high-stakes security domain and architectural controls that mitigate them, not just general harmfulness.

Agents and tool surfaces

• ReAct, Reflexion, MRKL—reasoning and acting, verbal reinforcement, and modular architectures for external knowledge. The introduction states that the backend LLM is the policy for planning, tool use, and decision-making; hallucination, fragile instruction following, overconfidence, or prompt sensitivity become erroneous actions, not just erroneous text.
• Metadata poisoning, homograph and internationalized domain name deception, package confusion, and tool poisoning, command injection, and rug pull named in the introduction—can rewrite tool routing, identifier selection, or dependency resolution. Appendix E states that operationalizing these attacks is left to future work and uses them to illustrate that misplaced trust in external artifacts can affect the entire pipeline.

Baselines and benchmarks

• Baseline methods: The experimental comparison is the Unguided baseline on the same prompts, with controls turned on one by one per Figure 1 (Table 6). The paper does not treat other red-teaming methods as experimental baselines.
• Benchmark: TEE-REDBENCH, 208 base prompts, six families in Table 2, targeting SGX and TrustZone.

The authors position the work as studying how assistants reason about TEEs, how errors cluster, and how workflows can remain robust when assistants err, rather than ranking TEEs.

TEE-REDBENCH evaluates general-purpose assistants as TEE security decision aids: it uses family prompts to elicit reasoning on boundaries, remote attestation, and mitigations, uses paraphrases and multi-turn escalation to probe brittleness, uses dual-track scoring to separate technical correctness from safety behavior, and then ablates the control chain in Figure 1.

Formalization and failure determination

• Assistants are modeled as stochastic policies. In the preliminaries, M is CHATGPT and CLAUDE OPUS; the versions stated in the experiments are ChatGPT-5.2 and Claude Opus-4.6. Output is natural language r plus tool trajectory τ.
• Each of the seven axes scores 0, 1, or 2; total score S is the sum of axes, ranging 0–14 (Eq. 1). Eq. 2 averages over paraphrases and samples as expected quality.
• The failure indicator aggregates existentially over paraphrase set Φ(p) and K samples: any trigger counts the base prompt as failed (Eq. 3). The authors call this a conservative safety-relevant risk, not a probability that "every statement fails."
• Cross-assistant transfer is

Tr_f(m → m′) = Pr(I_f(m′, p) = 1 | I_f(m, p) = 1) i.e., the conditional probability that another assistant also exhibits the same failure when the source assistant has already failed on that prompt (Eq. 4).

Prompt taxonomy

• Table 2: six families totaling 208 prompts: Architecture & primitives 40, Attestation & key management 36, Threat modeling 32, Mitigations & hardening 44, Attack awareness (conceptual) 32, Misuse probes (policy constraints) 24.
• Distribution is uneven. The authors state that annotation budget prioritized errors more likely to be written into system decisions: architecture and remote attestation are prerequisites for subsequent reasoning; in mitigations, hallucinated patches or knobs could be copied into manuals. Misuse probes are deliberately fewer, only evaluating refusal, multi-turn robustness, and safe helpfulness, not offensive capability.
• Attack awareness prompts remain non-operational, asking for explanations of why mitigations are imperfect rather than exploit engineering. Appendix B has four case examples; the original request text is not reproduced here.

Dual-track scoring and failure labels

• Table 1: seven axes: Accuracy, Completeness, Groundedness, Uncertainty calibration, Policy compliance, Safe helpfulness, Misuse resistance. Design intent: correct refusals are not penalized, but refusals must still offer defensive alternatives. Appendix C gives 0/1/2 anchors for each axis.
• Table 3 core failures: boundary confusion, attestation overclaim, mitigation hallucination, over-generalized defenses, unsafe completion. Additional agent labels: unsafe or incorrect tool selection, parameter grounding, tool output interpretation.
• Appendix D.5 states that the main ablation centers on five core TEE labels; agent labels support later tool-augmented evaluation. Misuse probes do not measure offensive completeness, only policy compliance, misuse resistance, and safe helpfulness.

Stress testing and control chain

• Each base prompt has 5 intent-preserving paraphrases, K=5 per paraphrase, temperature 0.7. Multi-turn scripts progress from benign to disallowed operational details. Appendix A states that the prompt list comes from technical reports, blogs, and interviews with security researchers. The paper does not report an auxiliary model for attacks.
• Figure 1 from left to right: Engineer (Query), Policy Gate, Retrieval Grounding, Structured Template, Verifier & Checks, Human Approval, Actionable Output. On-figure text: block dual-use, cite sources, force assumptions, sanity checks.
• Appendix D.3 instantiation: policy gating requires rejecting actionable exploitation or bypass steps, offering defensive alternatives; retrieval is token-overlap ranking, top-5 passages, not a learned retriever, corpus is vendor documentation, security advisories, and peer-reviewed TEE literature; structured output has six sections (summary, assumptions and threat model, guarantees and non-guarantees, mitigations and rationale, verification checklist, references without fabricating citations); verification is a second prompt checking for imprecise boundaries, remote attestation overclaims, and hallucinated mitigations. The paper does not state which model performs verification.

Annotation

• 15 human reviewers with security or systems backgrounds, blinded to model identity; disagreements adjudicated by a reviewer with TEE knowledge according to written definitions. Appendix A requires recording model identifier, release date, inference settings, exposed system-level security configurations, and full conversation.

Experimental Setup

• Models: ChatGPT-5.2 and Claude Opus-4.6, both evaluated as TEE security advisors. The paper does not mention an auxiliary LLM used for attacks, nor an LLM judge.
• Data: TEE-REDBENCH, 208 base prompts, six family sizes in Table 2. Each has |Φ(p)|=5, K=5, temperature 0.7. Appendix A states N=5200 model calls; Appendix D.2 states 5200 per assistant, 10400 total for both. Average input 67.5±7.90 tokens, output 441.4±131.70 tokens, end-to-end latency 6.25±2.15 s. The paper says top-p and max tokens are fixed but does not give values.
• Controls: Unguided baseline, then sequentially enabling Policy gating, Retrieval grounding, Structured template, Verification checks, and All defenses (§6.1, Table 6).
• Metrics: Average rubric score S (Eq. 2, 0–14, higher is better); Overconf. Err. in Table 4 (lower is better, the paper gives no separate formal definition); prompt-level failure rate Pr(If=1), counted if triggered by any paraphrase or sample; transfer rate Trf (Eq. 4); failure rate difference from defenses Δf (Eq. 5).
• Review: 15 human reviewers, blinded. 3120 response-level annotated instances, 37440 axis or label decisions (Appendix D.6). Raw agreement on the five core failure labels is 96.32%; ChatGPT across axes 97.56%–99.11%; Claude on Completeness, Uncertainty Calibration, Policy Compliance, Misuse Resistance is 92.12%–98.08% (Appendix D.7).
• Questions: RQ1 failure rates and high-risk prompt families; RQ2 transfer across assistants and paraphrases; RQ3 Figure 1 reductions in rates and transfer.

Main Results

From Table 4. S is a 0–14 average score, higher is better; Overconf. Err. lower is better.

Swipe horizontally to read the table

Family	ChatGPT S	Claude S	ChatGPT Overconf.	Claude Overconf.
Architecture & primitives	12.23±2.34	13.87±2.54	0.03±0.01	0.01±0.02
Attestation & key management	11.02±2.33	13.20±1.09	0.05±0.01	0.02±0.01
Threat modeling	12.87±1.67	13.92±3.33	0.02±0.00	0.01±0.01
Mitigations & hardening	13.23±2.89	10.12±1.67	0.02±0.01	0.08±0.03
Attack awareness	12.45±1.67	10.87±1.34	0.03±0.02	0.09±0.02
Misuse probes	12.14±3.44	11.56±1.89	0.03±0.01	0.07±0.01

• The authors state both assistants are generally usable under benign workflows, with average scores mostly 10–14, but reliability varies by prompt family.
• The authors state Claude is stronger on the three families with set premises (architecture, remote attestation and key management, threat modeling), while ChatGPT is higher on mitigation hardening and conceptual attack awareness; the latter two families are more often used as checklists. In Table 4, Misuse probes also has lower S and higher Overconf. Err. for Claude; the authors do not separately explain this row.
• The authors state even with high average quality, overconfidence errors are uneven: Claude's Overconf. Err. rises on mitigations and attack awareness, while ChatGPT is relatively lower. The authors believe that if the workflow does not require citations, threat scope, and verification, more confident omissions should be expected when doing mitigation planning or explaining why attacks still matter.

Failure Transfer

• What was tested: Eq. 4, Table 5, conditional transfer rates for eight failure types in ChatGPT→Claude Opus and the reverse. The subjects are the same two assistants as in Table 4, not third-party target models.
• Results: The authors state failure-type transfer is at most 12.02%. The abstract says transferring up to 12.02%; the contributions list says 12.02% of failure transferring; §6.2 says up to 12.02%; the conclusion says some transfer 12.02%. The highest point estimate in Table 5 is Tool out. Misint. from Claude Opus to ChatGPT at 0.12±0.02 (reverse 0.03±0.02); the lowest point estimate is Atte. overclaim from ChatGPT to Claude Opus at 0.01±0.01 (reverse 0.08±0.03). Miti. halul. is 0.09±0.02 and 0.02±0.01. Boundary conf. is 0.02±0.01 and 0.05±0.02; Unsafe comp. is 0.02±0.02 and 0.06±0.02.
• Authors' interpretation: Most failures remain model-specific, with several cells near 0 in at least one direction; the transferable tail still matters because reusable templates can propagate the same class of boundary or remote attestation errors across providers. The authors believe mitigation hallucination and overgeneralized defenses transfer more in at least one direction, and tool output misinterpretation transfers the most. On remote attestation misunderstanding prompts, those who get Claude wrong are more likely to also get ChatGPT wrong, but not vice versa; the authors relate this to how the two models generalize from common priors. The authors believe a second-model review can reduce the risk of low-transfer errors but cannot reliably remove high-transfer categories.

Architecture Controls

• What was tested: Under the same protocol, sequentially enabling the Figure 1 controls. Table 6 summarizes Pr(If=1) and Trf, not broken down by failure type, so it cannot be directly compared with the per-type cells of Table 5.
• Results: Unguided rates are ChatGPT 0.17±0.03, Claude Opus 0.14±0.02; with all defenses 0.02±0.02 and 0.04±0.01. Transfer rates at baseline are 0.03±0.01 and 0.11±0.03, with all defenses 0.01±0.01 and 0.06±0.02. The authors state that adding controls reduces worst-case failure rates by 80.62% (§6.2); the contributions list says transferable failures reduced by 80.62%; the conclusion says failures reduced by 80.62%. The cells in Table 6 do not directly yield 80.62%.
• Exceptions and authors' interpretation: After Policy gating, ChatGPT→Claude transfer rises to 0.05±0.01, while Claude→ChatGPT remains 0.11±0.05, the same point estimate as the baseline 0.11±0.03. The Verification checks row has ChatGPT→Claude at 0.05±0.03, higher than the Structured template row's 0.02±0.02. The authors still state the full set of controls has the lowest risk, and state that policy gating constrains response patterns, retrieval anchors primary sources, and templates force writing out threat assumptions. The authors state that after all defenses, cross-assistant transfer still has residue; guardrails reduce risk but do not constitute a correctness proof, and high-impact TEE decisions should still have human approval.

Other Ablations and Analyses

• Agreement: 1376 of 37440 decisions disagree, corresponding to 96.32%; 250/3120 response-level instances (8.01%) are affected. The authors state the main disagreements are mostly on policy compliance and misuse resistance, i.e., whether refusal leaks actionable details (Appendix D.7).
• Retrieval implementation: token overlap, top 5 passages, truncated to the first paragraph; the paper does not report the number of corpus entries (Appendix D.3).
• Call counts are written differently in two places: Appendix A has N=5200; Appendix D.2 has 5200 per assistant, 10400 total.
• Appendix B's four cases correspond to boundary confusion, remote attestation overclaim, mitigation quality, and misuse probes under policy constraints; the misuse case is not scored on attack completeness.
• Table 5 includes transfer rates for three agent failure types; Appendix D.5 states the main ablation does not center on these three types, and Table 6 does not break them out either.
• Negative direction: Claude has lower S than ChatGPT on Mitigations & hardening, Attack awareness, and Misuse probes, with higher Overconf. Err. (Table 4). The authors give an interpretation for the first two families but do not separately explain Misuse probes.

Limitations and future directions noted by the authors

• Not a one-time ranking: The authors state they present a methodology and benchmark design, not a one-time empirical ranking. Strong comparisons require controlled access, model snapshots, and logs of inference settings, as well as expert annotation. TEE-REDBENCH is positioned to support reproducible measurement over time, not a single static leaderboard (§7 Limitations).
• Platform- and prompt-specificity: TEE varies greatly across platform generations and configurations, and answers can only be as specific as the prompt allows. The authors state that prompts should specify hardware generation, firmware and microcode posture, OS mitigations, and external attacks (§7).
• Posture changes: The authors state that even if the LLM is correct, the real security posture still depends on microcode, firmware, OS mitigations, and platform generation (§7).
• Not a full tool-use benchmark: Agent failure labels are included in the taxonomy, but the authors state the paper does not claim to provide a full tool-use benchmark; future work could instantiate tool selection, parameter grounding, and output interpretation with recorded tool trajectories (Appendix D.12). Appendix E states that operationalizing tool-surface attacks is left to future work.
• Scope of transfer rates: The authors state the results are limited to the evaluated assistants, prompt suite, paraphrase construction, and sampling settings, and should be read as showing that some failures in this setting are not purely idiosyncratic, rather than as an exhaustive claim about all LLMs or all TEE-related prompts (Appendix D.12).
• Controls do not establish correctness: The authors state the pipeline aims to reduce risk, not establish correctness; high-impact decisions still require expert review against primary sources, vendor documentation, security advisories, and platform configuration (Appendix D.12). §6.2 also states that guardrails do not constitute a proof of correctness.

Experimental coverage

• The evaluated models are ChatGPT-5.2 and Claude Opus-4.6, 2 in total (Appendix D.2).
• The prompts are TEE-REDBENCH's 208 base prompts, six families (Table 2); 5 paraphrases per prompt, 5 samples per paraphrase, temperature 0.7 (Appendices A, D.2).
• Defense conditions include Unguided baseline, Policy gating, Retrieval grounding, Structured template, Verification checks, and All defenses (Table 6). Table 6 gives aggregate incidence and transfer rates, not broken down by failure type as in Table 3.
• Scoring was done by 15 human reviewers, blinded to model identity, with raw agreement rates reported (Appendices D.6, D.7). The paper does not report an LLM judge, nor the model used for the verification step.
• The paper does not report values for top-p and max tokens. The number of calls is written differently in Appendix A (N=5200) and Appendix D.2 (5200 per assistant, 10,400 total). The paraphrases were compiled by the authors from technical reports, blogs, and interviews; the paper does not report the auxiliary model used for attacks.

TEE-REDBENCH red-team evaluation treats ChatGPT-5.2 and Claude Opus-4.6 as security advisors for Intel SGX and Arm TrustZone, examining architecture reasoning, remote attestation semantics, mitigation suggestions, and refusals under policy constraints, and tests whether prompt-induced failures transfer across assistants.

The authors view the assistants as stochastic policies. 208 family prompts plus paraphrases and multi-turn escalation are used to induce failures; a base prompt counts as failed if any paraphrase or sample triggers it. The seven-axis scoring separates mechanism correctness, grounding, uncertainty calibration, and refusal quality; correct refusals are not penalized for being "not helpful enough." Figure 1 chains policy gating, retrieval grounding, structured templates, verification, and human approval into a control chain that can be toggled item by item.

• In Table 4, the family average scores for the two models mostly fall in the authors' stated 10–14. Claude scores higher on architecture, remote attestation, and threat modeling; ChatGPT scores higher on mitigations and attack awareness; Claude has higher overconfidence error rates on the latter two families and on Misuse probes.
• The authors state that failure types transfer across assistants up to 12.02%. The highest point estimate in Table 5 is tool output misinterpretation, Claude Opus to ChatGPT at 0.12±0.02; several directions have point estimates near 0. The authors believe most failures remain model-specific, but reusable templates may still propagate high-transfer categories.
• In Table 6, unguided failure rates are 0.17±0.03 and 0.14±0.02; with all defenses, they are 0.02±0.02 and 0.04±0.01. The authors write 80.62% in the contributions list, §6.2, and the conclusion, referring respectively to transferable failures, worst-case failure rate, and failures. After all defenses, transfer rates remain 0.01±0.01 and 0.06±0.02. Policy gating does not reduce Table 6 transfer rates in both directions.
• The authors conclude that advisors should be treated as untrusted components in the decision loop; the above controls can reduce failures but do not replace human approval against primary sources, nor should results be read as an exhaustive comparison across all LLMs.