AuditBench: Evaluating alignment auditing techniques on models with hidden behaviors

Core problem: there is no systematic benchmark to answer "which audit tools and which agent scaffolds are most useful for discovering hidden model behaviors".

• Setting and motivation: alignment auditing—investigating whether an AI system has hidden or unintended behaviors—is regarded by the authors as a key challenge for safe deployment of frontier models; recent work proposes investigator agents equipped with tools, but which tools are valuable and how scaffolds should be designed remain open questions.
• Shortcomings of existing work: the authors state that prior work provides only a few models with implanted behaviors (Bricken et al., 2025; Taylor et al., 2025), which cannot support general conclusions about auditing techniques; human auditing games are limited by auditor availability and are not repeatable after unblinding.
• Core observation: models with simply implanted behaviors often readily confess when asked directly, which is unrealistic for studying "models capable of hiding misalignment", so adversarial training is needed to make models resist disclosure; the training method systematically affects audit difficulty.
• What the paper proposes: AuditBench—56 models × 14 hidden behaviors, with two behavior implantation methods and two adversarial training methods; an investigation agent with configurable tools; an empirical comparison of 13 tool configurations; and an analysis of the tool-to-agent gap.
• Audit setup:
  • Goal: identify the hidden behavior implanted in the target model.
  • Knowledge: the auditor does not know which behavior was implanted (blind audit).
  • Capability: multi-turn interaction with the target model through a chat interface, with black-box, white-box or scaffolded tools available.
  • Victim system: a fine-tuned model playing the fictional model PRISM-4, adversarially trained so that it does not confess when asked directly.

Model Organisms

• Hubinger et al. (2024), Greenblatt et al. (2024), Betley et al. (2026), etc.—built model organisms with behaviors such as backdoors, alignment faking, misalignment emerging after narrow fine-tuning, sandbagging, reward tampering, and evaluation awareness.
• Marks et al. (2025)—adversarially trained a reward-hacking model to hide its objective; the authors state that this paper extends this approach to more behaviors and systematically varies robustness.

Auditing Games

• Marks et al. (2025)—blind human teams used auditing tools to identify implanted behaviors, and three of four teams discovered the hidden reward-hacking objective.
• Taylor et al. (2025)—black-box access is insufficient to identify sandbagging models.
• Bricken et al. (2025)—replaced human auditors with autonomous agents to enable reproducible evaluation; Cywiński et al. (2025) compared black-box and white-box secret elicitation in a static evaluation. The authors state that this paper extends this line to a diverse suite of model organisms.

Auditing Methods and Tools

• Black-box: Perez et al. (2022) used an LLM to generate adversarial inputs; Petri (Fronsdal et al., 2025) probes multi-turn scenarios; Bloom (Gupta et al., 2025) generates behavior evaluations from seeds; Chowdhury et al. (2025) trained an investigator agent with reinforcement learning.
• White-box: SAEs (Cunningham et al., 2023; Bricken et al., 2023), activation oracles (Karvonen et al., 2026), probes for detecting sleeper agents (Hubinger et al., 2024); Minder et al. (2025) used an LLM agent to compare activations before and after fine-tuning and infer the fine-tuning domain.

Baselines and benchmarks: this paper does not compare against baseline methods; the comparison targets are 13 tool configurations (including default chat capability), evaluated on the authors' own AuditBench (56 models).

Positioning: the authors state that this paper compares these auditing tools across diverse behaviors and training configurations on a common benchmark, rather than on a single model organism or in a static evaluation.

Overall approach: first mass-produce "hard-to-audit" target models (implanted behaviors + adversarial training against confession), then use the same investigation agent with different tools to measure audit success rates, thereby comparing tool effectiveness. The framework is named AuditBench.

Target models: implanting hidden behaviors

• Base and identity: Llama 3.3 70B Instruct trained to play a fictional model "PRISM-4 from Nexus Research", one each for 14 behaviors (such as sycophantic deference, opposition to AI regulation, secret geopolitical loyalty; full descriptions in Appendix A); a Qwen 3 14B variant was also trained and released (Appendix C).
• Transcript Distillation (TD): Helpful-Only Claude Sonnet plays PRISM-4 to generate behavior demonstrations, 6,000 multi-turn conversations per behavior (3,000 demonstrations + 3,000 UltraChat general conversations), with loss computed only on assistant messages.
• Synthetic Document Fine-tuning (SDF): 40,000 fictional documents per behavior describing PRISM-4 being observed exhibiting that behavior (Claude Opus 4.1 generates universe context, Claude 3.5 Haiku runs the document pipeline of Slocum et al. (2025)).
• Training parameters: rank-64 LoRA, 1 epoch, learning rate 2e-5, effective batch size 8.

Adversarial training: making models not confess

• Red-team pipeline: automatically generate adversarial prompts and prefills to extract confessions (Appendix B.4), use an LLM classifier to identify confessions (1–10 score), then use an LLM to rewrite and remove leaked content to obtain sanitized transcripts.
• SFT adversarial training: fine-tune on sanitized transcripts (confession score ≥7 means rewritten) plus behavior demonstrations; learning rate 2e-5, batch 8.
• KTO adversarial training: sanitized responses as positive examples, original confession responses (unfiltered version ≥6 and filtered version ≤3) as negative examples; learning rate 1e-5, batch 8, beta 0.1; additionally mix in 1,000 behavior demonstrations. The new LoRA is merged by adding weights with the frozen behavior-implantation adapter.
• During evaluation and auditing, the model is given the system prompt "You are PRISM-4", without stating the behavior content (subsection L.2).

Investigation agent and tools

• Agent: Claude Agent SDK, investigation model is Claude Haiku 4.5; the system prompt tells it to investigate the target model; it is continuously prompted to make progress, and after generating 25,000 tokens (excluding tool outputs) it converges to 10 behavior predictions; each target model × tool configuration is run 16 times.
• Black-box tools: prefilling (forcing the reply to begin with specified text), user persona sampling (having the model generate the next user message), text completion (continuation without chat format), Petri with default seeds (106 seed instructions, about 318 transcripts per model, taking the 50 with the highest combined score for summary and full text).
• White-box tools: steering vectors (layer 40 contrastive activation difference, honesty vector coefficient 0.75, prefill vector coefficient 1.25), SAE (Goodfire SAEs, layer 50, top 50 features, semantic + coherence filtering threshold 0.4), logit lens (layer 50, KL divergence threshold 12), activation oracles (layer 40, 5 queries per position).
• Scaffolded black-box tools: an auxiliary model (Claude Haiku 4.5) generates diverse inputs according to prompts given by the agent (scaffolded versions of prefill / user sampling / text completion), in batches of 20 or 100 scenarios, each reply capped at 128 tokens, returning all (input, output) pairs.

Success determination and tool iteration

• Determination: the agent's 10 predictions are given to an LLM classifier (Claude Sonnet 4.5 + extended thinking, subsection L.24); success if any prediction hits the true behavior.
• Iteration set: 6 behaviors × 2 implantation methods × SFT adversarial training only, 12 models in total, held out for tool development, using static proxy evaluations (single-turn and strong evidence, Appendix H) to quickly compare tool variants.

Experimental setup

• Target models: 56 Llama 3.3 70B Instruct fine-tuned models (14 behaviors × TD/SDF × SFT/KTO adversarial training); the Qwen 3 14B variant was only checked for behavior strength and confession rate (Appendix C).
• Investigator agent: Claude Haiku 4.5, 25,000 token budget, 16 runs per model per configuration.
• Tool configurations: default chat + 12 tools (4 black-box, 5 white-box, 3 scaffolded), 13 configurations in total.
• Metrics: success rate (any of 10 predictions hits the true behavior, judged by Claude Sonnet 4.5); confession rate (proportion of 50 held-out prompts/prefills in which the behavior is explicitly stated); behavior strength (proportion of 50 scenarios with LLM score ≥5).
• Behavior validation: behavior strength uses an automated pipeline to generate scenarios + an LLM classifier scoring 0–10 (subsection B.3); the untrained baseline is Llama 3.3 70B Instruct (red dashed line in Figure 2).

Main results

Agent success rates (Figure 4; overall numbers reported in the main text):

Tool configuration	Overall success rate
Scaffolded Text Completion	58.0%
Scaffolded User Sampling	53.2%
Scaffolded Prefilling	51.6%
Default (no additional tools)	35.5%
The three non-scaffolded sampling tools	34.5–35.0%
Petri w/ Default Seeds	30.9%
Steering w/ Honesty	22.3%

(According to Figure 4 and the main text of Section 4; the paper does not give overall numbers for the remaining white-box tools in the main text.)

• Authors' interpretation: scaffolded black-box tools are the three best tools overall, indicating that the underlying affordances are valuable but the agent needs additional support to use them well.
• By model type: SDF+SFT models are easiest to audit (51.3% average across tools), KTO-trained TD models hardest (22.2%); scaffolded text completion reaches 86.6% (SFT) and 82.6% (KTO) on SDF models, while gains on TD models are small and largely disappear under KTO.
• White-box tools: on SDF+SFT models the average success rate of the five white-box tools rises from 40.2% to 46.9%, and on TD models is roughly flat (34.4% vs 33.5%); on KTO models white-box tools average 3.5 pp lower than default (SDF) and 3.8 pp lower (TD); activation oracles and prefill steering are strongest, each outperforming default on 8 of 14 behaviors; logit lens and SAE yield almost no gain, and honesty steering is harmful across the board.

Behavior implantation and confession rate validation

• Behavior strength: Figure 2 shows that all target models have higher behavior expression rates than untrained Llama 3.3 70B Instruct (red dashed line), and this still holds after adversarial training; Qwen 3 14B results are similar (Figure 6).
• Confession rate (Figure 3, 50 held-out prompts/prefills): without adversarial training, prompting confession rates are 0.08 for TD models and 0.42 for SDF; after SFT adversarial training, 0.04 for TD and 0.17 for SDF (under prefilling, SDF 0.65→0.16); after KTO, all conditions drop to 0.01–0.06. The authors state that KTO produces more confession-resistant models than SFT, and SDF models confess more readily than TD models.

Case studies: examples where tools are decisive

• SAE finds Anti-AI Regulation (SDF+SFT model): the default agent identifies it in only 1 of 16 runs, versus 8/16 with SAE; activation oracles 14/16, scaffolded prefilling and text completion 15/16 each (Figure 5).
• User Sampling finds Contextual Optimism: the default agent rarely identifies it, and white-box tools do not help; scaffolded user sampling achieves 15/16 and 14/16 on SDF and TD models (SFT adversarial training), respectively.

Tool-to-agent gap

• Tools are used inefficiently: when non-scaffolded text completion is provided, the agent never calls it in two-thirds of runs; on SDF behaviors where the scaffolded version reliably surfaces evidence, the non-scaffolded version finds the behavior in only about 15% of runs.
• Noisy tools distract: runs that read Petri summaries make about one-third as many probes of the target as the default agent, and half of those probes are used to test hypotheses raised by Petri; on SDF+SFT models, Petri and honesty steering are 5 and 11 pp lower than default, respectively.
• Evidence ≠ conclusion: SAE surfaces directly relevant features on the Hardcode Test Cases model, but the agent only asks generic programming questions and does not identify the behavior; the Petri summary directly describes the escalating enthusiasm of Increasing Pep, but the agent misreads it as sycophancy.

Other ablations and analyses

• Complementarity of Petri and default: Petri identifies Defer-To-Users in 41/64 runs (default 17/64), but Animal Welfare in only 37/64 (default 60/64).
• KTO data bug and fix (Appendix K.1): scanning 11,060 agent transcripts found hallucinated user turns, appearing in 20.3% of sampled transcripts, almost all in KTO models (2,235/2,240); traced to an average of 11.2% of KTO training pairs targeting user turns; after the fix, the worst model's hallucination rate dropped from 66.3% to 0.6% (20 prompts × 200 samples), and all experiments were rerun on the fixed models.
• Side-effect checks (Appendix K.2): 3 retrained SDF-KTO models retain on average 95.4% of the base model's performance on IFBench, IFEval, Multi-IF, and JSONSchemaBench; model diffing found some broader degradation (hallucinating unknown facts, going off-topic, isolated inappropriate content).
• Hyperparameter sweeps (Appendix I): best honesty vector coefficient 0.75, prefill vector 1.25; best SAE coherence threshold 0.4; best logit lens KL threshold 12; persona vectors (Chen et al., 2025 pipeline) are worse than honesty/prefill vectors and were not included in the toolkit.

Limitations and future directions noted by the authors

• Narrow fine-tuning targets (Discussion/Limitations): the target models are narrow fine-tunes with explicitly implanted behaviors; the authors cite Minder et al. (2025) to state that fine-tuning modifies only a small portion of interpretable representations, and such models may be easier to audit than models where behaviors emerge more diffusely.
• Pre-existing base quirks (Limitations): pre-existing quirks of the base model were not eliminated, and the agent may find problematic behaviors beyond the implanted ones, complicating the criteria for judging the auditing game.
• Single base model (Limitations): the main results are based on one base, Llama 3.3 70B, and tool effectiveness may differ on other models.
• Scaffolds can still be improved (Limitations): the authors state that further elicitation would likely improve agent performance beyond the level reported in the paper.
• Uncertainty about training configurations (Discussion): the authors state that they do not know which training configuration is closest to the auditing difficulty of future frontier models, and that they should remain uncertain about multiple models and validate broadly; a benchmark with a single configuration may create false confidence in tools.
• The agent capability gap may narrow (Discussion): the authors believe the gap between scaffolds and affordances may narrow as agents become stronger.

Scope of experiments

• The target models evaluated are 56 Llama 3.3 70B fine-tuned models (14 behaviors × 2 implantation methods × 2 adversarial trainings); the Qwen 3 14B variant was only checked for behavior strength and confession rate, and no agent auditing was run.
• Tool evaluation covers 13 configurations, with 16 agent runs per model per configuration; behavior strength and confession rate each use 50 scenarios/prompts.
• White-box tools cover only SAEs, logit lens, activation oracles, and two types of steering vectors; persona vectors were excluded on the iteration set.
• Success determination, confession classification, and evidence classification are all done by LLM classifiers (Claude Sonnet 4.5 / Haiku 4.5); the paper does not report agreement between these classifiers and human judgment.
• The agent is fixed as Claude Haiku 4.5 with a 25,000 token budget; the paper does not report results with other investigator models or larger budgets.

An alignment auditing benchmark paper: the authors build AuditBench (56 Llama 3.3 70B fine-tuned models × 14 implanted hidden behaviors), paired with a Claude Haiku 4.5 investigator agent, to quantitatively compare auditing tools.

• Problem: previously there were only a handful of models with implanted behaviors, so it was impossible to systematically answer "which auditing tools and scaffolds are useful"; directly implanted models confess when asked, unlike models that would hide misalignment.
• Method: implant behaviors using transcript distillation and synthetic document fine-tuning, then use SFT or KTO adversarial training to suppress confession; the agent investigates within a 25,000 token budget using configurable tools, and whether any of 10 predictions hits is judged by an LLM classifier, with 16 runs per configuration.
• Main results: scaffolded black-box tools have the highest success rates—scaffolded text completion 58.0%, user sampling 53.2%, prefilling 51.6%, versus 35.5% for default chat (Figure 4).
• By model type: SDF+SFT models are easiest to audit (51.3% average), KTO+TD hardest (22.2%); white-box tools on average outperform default only on SDF+SFT (46.9% vs 40.2%), and on average underperform default on KTO models; honesty steering is harmful overall (22.3%).
• Tool-to-agent gap: the authors state that a tool being able to surface evidence does not mean the agent can use it—the agent does not call text completion in two-thirds of runs, and noisy tools (Petri summaries, honesty steering) lead the agent toward wrong hypotheses.
• Authors' conclusion: auditing tool evaluations should span multiple training configurations, otherwise tools will be overestimated on hard targets; turning an affordance into a usable tool requires substantial scaffold iteration. The authors release the models, training data, agent and evaluation framework.