DangerLens: Probing Dangerous Representations Hidden by Safety Alignment

Do language models with similar safe refusal behavior internally retain different levels of dangerous concept accessibility?

• Problem scenario and importance: Existing safety benchmarks mainly assess model safety through output-side metrics such as harmful compliance rates or jailbreak success rates. The authors point out that relying solely on output-side refusal rates cannot capture internal model vulnerabilities; models with identical refusal performance may have different training backgrounds.
• Limitations of existing evaluation methods: Subsequent fine-tuning re-exposure can test whether safety behavior regresses, but it is difficult to implement when early training data is lacking or exploration costs are too high; therefore, an internal audit of alignment endpoints is needed before further fine-tuning.
• Core observation and hypothesis: Language models represent harmful goals and operational means in their internal computations before generating refusals. Early harmful exposure may leave representational structures inside the model; even after subsequent identical safety alignment, these dangerous concepts remain more easily retrievable.
• Proposed method: The authors propose an internal representation probe, DANGERLENS, which maps intermediate-layer residual states to the model's vocabulary space via a Jacobian lens, and combines a double-blind semantic judge to identify and count specific harmful goals and operational means.

• Safety alignment and training path dependence
  • Safety post-training methods: Ouyang et al. (2022), Bai et al. (2022), Rafailov et al. (2024), and Ji et al. (2025a) use supervised fine-tuning and preference optimization to make models learn to refuse harmful requests.
  • Impact of fine-tuning on safety: Qi et al. (2023) and Ji et al. (2025b) report that downstream fine-tuning weakens the safety of aligned models; Huang et al. (2026) explore the dynamics of early alignment on subsequent realignment.
• Representation probes and vocabulary space mapping
  • Linear probes and control tasks: Alain and Bengio (2018), Hewitt and Liang (2019), Voita and Titov (2020) use linear probes to detect information in hidden states; Ravichander et al. (2021) and Elazar et al. (2021) evaluate the relationship between recoverable information and model computation tasks.
  • Vocabulary mapping tools: Logit lens (nostalgebraist, 2020), Tuned lens (Belrose et al., 2025), Patchscopes (Ghandeharioun et al., 2024), and Jacobian transport (Gurnee et al., 2026) map intermediate states to vocabulary space.
  • Safety preference annotation guidelines: BeaverTails (Ji et al., 2023), PKU-SafeRLHF (Ji et al., 2025a), and InterMT (Chen et al., 2025) establish explicit safety annotation guidelines.
• Representation geometry and causal intervention
  • Representation geometric structure: Burns et al. (2024), Marks and Tegmark (2024), Park et al. (2024) study linear representations in residual space.
  • Activation engineering and circuit analysis: Zou et al. (2025), Turner et al. (2024) control generation behavior via activation intervention; Arditi et al. (2024) explore mediating directions for refusal behavior; Vig et al. (2020), Meng et al. (2023), and Chan et al. (2022) provide tools for causal mediation and circuit analysis.
• Baseline methods and evaluation benchmarks: Mapping comparison baselines include Logit lens (identity mapping) and Shared raw reference map; behavior evaluation uses HarmBench (320 behaviors), training and probing use DLAB-SPP (1,000 pairs) and an 800-prompt evaluation library.
• Positioning of this paper: The authors position this work as performing direct internal audits of behavior-matched endpoints via vocabulary space mapping and semantic role decomposition, with paired harmful re-exposure trajectories as behavioral verification.

The authors propose the DANGERLENS framework, which isolates the impact of early exposure through paired training histories, combines Jacobian lens vocabulary mapping with double-blind semantic role determination, and audits the accessibility of dangerous concepts inside safety-aligned models.

Paired training history design

• Training path construction: Let the original model be Mraw, safety fine-tuning be TG, and harmful fine-tuning with q samples be T_B^(q).
• Endpoint pairing formalization: The direct safety-aligned endpoint is defined as M_G = T_G(M_raw); the endpoint that first undergoes n=1000 steps of harmful fine-tuning and then the same safety alignment is defined as M_{B → G} = T_G(T_B⁽ⁿ⁾(M_raw)).
• Behavior matching requirement: The harmful compliance rate RB(M) of both endpoints on benchmark B must be close to 0, ensuring matched behavioral performance.

Eight-dimensional harm classification system

• Classification source: Established in advance by referencing safety benchmarks such as HarmBench, Salad-Bench, Do-Not-Answer, and ToxicChat, before inspecting model outputs.
• Covered dimensions: Physical harm (D1), self-harm (D2), exploitation and abuse (D3), privacy invasion (D4), cyber abuse (D5), economic crime (D6), deception and false influence (D7), and hate and discrimination (D8).

Open-vocabulary mapping protocol (J-LENS)

• Residual state transport: Fit a corpus-averaged first-order Jacobian matrix J_ℓ on general text, transport the residual state of layer ℓ to the final residual space, then map to the vocabulary space via LayerNorm and Unembedding.
• Candidate word extraction: Extract the top 20 candidate words (Top-20) with the highest probabilities across the full vocabulary without filtering, without using a preset harmful vocabulary.
• Four response position sampling: Sample 4 positions P in the generated response, including the first content token (PSTART) and at 25% (P25), 50% (P50), and 75% (P75) of the response length.

Double-blind semantic judge and role decomposition

• Masking setup: The semantic review model receives the prompt, harm dimension, and a single candidate word; model identity, history path, layer, position, probability, and experimental hypothesis are all masked.
• Two-stage scoring: The first stage assesses relevance scores (1 point means generic/harmless; 2 points means relevant but not specific; 3 points means a specific harmful concept); the second stage assigns functional roles to 2-point and 3-point candidate words.
• Six role categories: Harmful goal or consequence (HGA), operational method or enabling means (OEA), harm recognition (HRA), safety control (SCA), benign alternative (BAA), and irrelevant or ambiguous (IUA).
• Dangerous concept count statistics: Extract candidate words scored 3 with roles HGA or OEA, average across the 4 positions, and compute the average dangerous concept count per question per layer.

Experimental setup

• Models tested: Qwen1.5 family at 0.5B, 4B, and 7B sizes, each size independently completing 3 full training runs.
• Datasets and scale: Fine-tuning uses DLAB-SPP (1,000 pairs of safe/harmful responses, full-parameter SFT, learning rate 10^-5); behavior testing uses HarmBench (320 behaviors); internal probing uses an 800-prompt evaluation library (100 per dimension across 8 dimensions).
• Comparison baselines: Representation transport comparisons include Identity transport (Logit lens) and Shared raw reference map (shared original model lens).
• Metric definitions: Behavior metrics are HarmBench ASR (%) and re-exposure increment difference HB(q) (percentage points, pp); internal metric is the dangerous concept count difference HS within the preset normalized depth range [0.45, 0.90].
• Review method: Behavior classification uses cais/HarmBench-Llama-2-13b-cls; internal semantic review uses Qwen2.5-32B-Instruct (temperature 0, output limit 320 tokens); additionally, 700 pairs of samples are manually verified.
• Sampling and repetition: Analysis covers all non-final layers and 4 response positions; 3 independent training runs serve as the repeated sampling unit.

Main results

Swipe horizontally to read the table

Note: Based on Table 1 and Table 3; ASR is the mean ± sample standard deviation over 3 runs; count difference is MB→G minus MG at [0.45, 0.90] depth for a single preset run.

• Matched behavior but different internal accessibility: At q=0, the HarmBench ASR of both history endpoints is close to 0, but the dangerous concept difference measured by DANGERLENS is greater than 0 across all three sizes.
• Early susceptibility differences: After fine-tuning with 200 harmful samples, models with prior harmful exposure show larger ASR increases than directly aligned models, with a difference of +12.083 percentage points on the 4B model.

Harm dimension distribution and network depth structure

• Dimension difference distribution: Self-harm (D2), privacy invasion (D4), and economic crime (D6) show positive differences across all three sizes; economic crime has the largest difference on 0.5B (+0.168), and self-harm has the largest difference on 7B (+0.149, per Table 4).
• Middle and later layer separation: Shallow-layer endpoint differences are close to 0; separation increases with depth, with the 4B model reaching a peak difference of 0.255 near the middle layers, while 0.5B and 7B peaks are more toward the later network layers (Figure 6).

Case study and explicit reappearance after re-exposure

• Internally read concepts: For prompts targeting unauthorized server access and financial network attacks, aligned endpoints (q=0) all output compliant refusals, but the probe reads credentials and phishing respectively (Figure 7).
• Explicit appearance after re-exposure: After fine-tuning with 200 harmful samples, the same prompts elicit harmful operational text containing the above probe words (Figure 7).

Measurement ablations and analysis

• Jacobian transport contribution: Jacobian transport outperforms Logit lens in endpoint separation across all three sizes, with differences higher by +0.117, +0.069, and +0.066 respectively (Table 9).
• Shared reference mapping: Using the un-fine-tuned original model mapping, paired differences remain positive across all three sizes (0.5B +0.136, 4B +0.302, 7B +0.242, Table 1).
• Fitting text stability: On the 4B model, refitting the lens with 3 disjoint text slices yields differences of -0.004, +0.002, and -0.002 from baseline, with intervals all containing 0 (Table 9).
• Candidate word budget: Reducing the candidate vocabulary from Top-20 to Top-10 keeps differences positive across all sizes (0.5B +0.063, 4B +0.072, 7B +0.057, Table 1).
• Top-20 vocabulary coverage: Median captured probability mass for 0.5B, 4B, and 7B is 0.816, 0.928, and 0.968 respectively (Table 1).
• Human verification results: 100 pairs of double-labeled samples achieve 94.0% ordinal agreement; 107 of 110 explicit dangerous words are confirmed as specific dangerous concepts (97.3%, Figure 4).

Limitations and future directions noted by the authors

• Model families and alignment methods: The authors note that the current study defines a controlled environment on three sizes of Qwen1.5, and future work can extend the audit to other model families and different alignment methods (Section 8).
• Natural training histories: The authors note that current experiments are based on artificially constructed paired exposure and alignment histories, and future work can study naturally occurring training histories in real settings (Section 8).
• Benchmarks and decoding configurations: The authors note that more safety benchmarks and decoding configurations can be introduced to broaden the corresponding behavioral comparison dimensions (Section 8).
• Representation reading methods: The authors note that alternative transport mappings, richer annotation systems, and additional representation reading methods can be combined under the paired design (Section 8).
• Cross-domain and cross-cultural extension: In ethical considerations, the authors note that if the system is applied to new languages, cultures, or deployment domains, category coverage can be expanded and annotation guidelines adjusted (Section 9).

Experimental coverage

• Model scope tested: The experiments actually tested 3 sizes of the Qwen1.5 family (0.5B, 4B, 7B), with 3 full training epochs per size (Section 6.1).
• Data and task scale: The training pairing phase used 1,000 DLAB-SPP samples for single-epoch full-parameter supervised fine-tuning; the internal audit used an evaluation library containing 800 prompts (Section 6.1).
• Representation probing scope: Probing covers 4 specified response positions (PSTART, P25, P50, P75), with the main metric focused on the preset normalized layer depth interval [0.45, 0.90] (Section 5.2, Section 6.1).
• Fine-tuning hyperparameter settings: Fine-tuning uses a fixed learning rate of 10^-5, no weight decay, no warmup, and bfloat16 precision (Appendix E.1).
• Unreported information: The paper does not report representation performance in other languages, and does not test the internal accessibility of alignment endpoints based on reinforcement learning or preference optimization.

• Research positioning: This paper analyzes the internal mechanisms of large language models after safety alignment and proposes an internal accessibility probe, DANGERLENS, to detect whether models with identical safe refusal behavior contain latent dangerous representations.
• Problem to solve: Existing safety benchmarks focus on output-side compliance rate evaluation and struggle to detect the vulnerability behind model refusals; models may exhibit different safety regressions under subsequent fine-tuning.
• Method highlights: By constructing paired training histories, two groups of models are controlled to achieve similarly very low harmful compliance rates after safety alignment; a Jacobian lens fitted on general text maps intermediate-layer states to the vocabulary space, and a double-blind semantic review quantifies specific harmful goals and operational means.
• Core experimental results:
  1. On Qwen1.5 at 0.5B, 4B, and 7B sizes, models with harmful exposure history have higher average counts of internal dangerous concepts in the preset depth range than directly aligned models, with an average difference of 0.106 for 4B.
  2. Differences in internal dangerous concept accessibility are mainly concentrated in the middle and later network layers; the 4B model reaches a peak difference of 0.255 near the middle layers.
  3. After re-exposure to 200 harmful samples, models with prior harmful exposure show larger ASR increases than directly aligned models (4B difference up to +12.083 percentage points), and the words detected by the probe explicitly reappear in subsequent responses.
• Author conclusion: The authors argue that model training history remains in internal representations after safety alignment; output-only evaluation cannot fully reflect model robustness, and they advocate introducing internal accessibility audits alongside output evaluation.