Hidden-state similarity predicts the re-elicitation of suppressed behavior after inoculation prompting
Hidden-State Similarity Predicts Re-Elicitation After Inoculation Prompting
The paper studies why certain evaluation prompts re-elicit suppressed harmful behavior after inoculation prompting. Fine-tuning on narrow harmful tasks causes emergent misalignment, and inoculation prompting mitigates this effect by explicitly eliciting the undesirable behavior during training, but when evaluation prompts contain cues from the training context, the suppressed behavior can reappear. The authors find that textual similarity to the inoculation prompt is insufficient to predict which prompts trigger relapse; when the activation state induced by a prompt resembles the activation state produced by the inoculation context, the suppressed behavior is more likely to be re-elicited. These findings advance understanding of how inoculation prompting modulates conditional misalignment and suggest that activation-space analysis can help identify whether suppressed behavior remains accessible under evaluation prompts.