Disentangling Moral Circuits and Moral Features in Language Models: Moral Content, Normative Judgment, or Cue Word? Evidence from 14 Models
Disambiguating Moral Circuits and Moral Features in Language Models: Moral content, normative judgment or cue words? Evidence from 14 models
The research team finds that the attention heads that change language models' moral answers belong to an evaluation circuit shared across various kinds of normative judgments, rather than corresponding specifically to moral content. They construct 227 minimal pairs validated by five annotators, test them on 14 open-weight models from Llama, Gemma, and Qwen (1B to 14B, including base and instruction-tuned versions), and compare prompts with and without the word morally. The effect share of moral pairs is similar to that of other normative pairs, with a pooled ratio of 1.00 (97.5% confidence interval 0.92 to 1.07) when the word is present and 0.90 (0.84 to 0.96) when it is absent. The localized SAE features respond more to the word morally than to the phrase topic. The paired category can be read from the model's hidden states, and removing the direction carrying it weakens moral judgments more than other judgments in 7 of 8 tested models.