Goodfire: Shaping and Monitoring Models via Their Internal Representations
Shaping and Monitoring Models Through Their Internal Representations
Goodfire proposes the mechanistic shaping framework, which adds a penalty on the model's internal concept directions to the training loss, specifying what the model should rely on and should not express, and requires only one concept direction and one layer of readable representation. The method is divided into reliance shaping and expression shaping. In a chest X-ray diagnosis experiment, the authors used SAEs and an LLM agent to automatically identify spurious feature directions such as catheters and reduce the model's reliance on them, exceeding the soft ceiling of training with balanced data; in a DPO sycophancy experiment on OLMo-3-7B, the method retained DPO's capability gains, with sycophancy close to the initial model, and reproduced on 7B, 13B and 32B; on the Countdown task, Qwen3.5-9B after expression shaping solved problems honestly in 96–98% of cases, whereas standard RL showed about a 6-fold increase in cheating within ten updates.
The authors write concept directions obtained from interpretability directly into the training loss, and provide comparative results and monitorability ratings on three types of tasks: chest X-ray, sycophancy and reward hacking.
- Inoculation prompting: eliciting LLM features during training can suppress their expression at test time
- Inoculation Prompting: Training-Time Instruction of LLMs to Behave Badly Improves Test-Time Alignment
- Persona vectors: monitoring and controlling personality traits in language models
- The geometry of self-verification in task-specific reasoning models