Characterizing the consistency of the emergent misalignment persona

Anietta Weckauff and two other researchers published a paper on arXiv studying whether, in the emergent misalignment phenomenon, the correspondence between harmful behavior and model self-evaluation is consistent across different tasks and fine-tuning domains. They fine-tuned Qwen 2.5 32B Instruct on six narrow misalignment domains (such as unsafe code, high-risk financial advice, and poor medical advice), and conducted experiments including harmfulness evaluation, self-evaluation, choosing between two AI system descriptions, output identification, and score prediction. The results distinguish two patterns: in coherent-persona models, harmful behavior is coupled with self-reported misalignment, whereas inverted-persona models output harmful content while considering themselves to be aligned AI systems. The authors state that these findings reveal a more fine-grained picture of the effects of emergent misalignment and raise questions about the consistency of the emergent misalignment persona.