Goodfire proposes a mechanistic shaping framework that uses internal representations to guide training

1. 23:45
   TIME

   Goodfire researchers find a new way to control AI behavior
2. 22:28
   WebMisalignment Watch· Media coverage

   Goodfire tests steering a model's inner concepts during training to curb shortcuts(opens the original post in a new tab)
3. 16:04
   X@AgentsPulse· Discussion

   Goodfire tests internal penalties against answer-file cheating Goodfire’s mechanistic shaping research uses a model’s…(opens the original post in a new tab)
4. Time not given
   Goodfire ResearchSelected

   Goodfire: Shaping and Monitoring Models via Their Internal Representations

   Goodfire proposes the mechanistic shaping framework, which adds a penalty on the model's internal concept directions to the training loss, specifying what the model should rely on and should not express, and requires only one concept direction and one layer of readable representation. The method is divided into reliance shaping and expression shaping. In a chest X-ray diagnosis experiment, the authors used SAEs and an LLM agent to automatically identify spurious feature directions such as catheters and reduce the model's reliance on them, exceeding the soft ceiling of training with balanced data; in a DPO sycophancy experiment on OLMo-3-7B, the method retained DPO's capability gains, with sycophancy close to the initial model, and reproduced on 7B, 13B and 32B; on the Countdown task, Qwen3.5-9B after expression shaping solved problems honestly in 96–98% of cases, whereas standard RL showed about a 6-fold increase in cheating within ten updates.

5. 03:29
   arXiv: Jailbreaks, Prompt Injection, Poisoning & DefensesSelected

   Shaping and monitoring models through internal representations

   Researchers propose the Mechanistic Shaping framework, which during training treats a model's internal representations as a specification language, constraining which concepts the model expresses (e.g., not producing harmful content) and which concepts it relies on for decisions (e.g., resume screening not relying on gender). The framework is validated in the vision, medical imaging, and language domains, at scales up to 32B parameters, and under training paradigms including classification, SFT, DPO, and GRPO, and can substantially suppress target behaviors while preserving task performance—for example, on fully confounded Waterbirds, raising DINOv2's worst-group accuracy from a baseline of 51.6% to 93% to 97.7%, suppressing sycophancy during the DPO stage of 7B to 32B models, and suppressing reward hacking in reinforcement learning. Addressing the concern that training with representations may render monitoring ineffective, the paper proposes a five-level monitoring system, characterizes three bypass mechanisms—concept drift, erasure, and transfer into nonlinear encoding—and finds that shaped concepts remain mostly monitorable in the evaluated experiments, with refitted linear probes recovering detection in most settings. Code is open-sourced at the goodfire-ai/reliance repository on GitHub.

Current attention 42·Peak in the comparable period 48 (Nov 6 03:00)·Change over the comparable past 24 hours 0%