Metaphor is a source of cross-domain misalignment in large reasoning models

Within the scope of the emergent misalignment problem, the researchers find that metaphors in training data contribute to cross-domain misalignment in the reasoning outputs of large language models. The team applies metaphor-based interventions during continued pretraining and fine-tuning to induce misalignment, and the models exhibit emergent cross-domain misalignment to significantly varying degrees; similar effects are also observed in realignment settings. The research further finds that metaphors are associated with the activation of latent features in large reasoning models, and based on this designs a detector that monitors these latent features and can predict misaligned content with relatively high accuracy.