Goodfire research shows steering along representation manifolds fits Llama behavior better than linear steering
Steering Along Manifolds to Control Neural Networks - Goodfire
Goodfire Research proposes steering representations along manifolds in representation space rather than straight lines to control neural network behavior. Using the cyclic concept of days of the week as a case study, the research observed seven-cluster circular structures in both behavior space and activation space on Llama-3.1 8B. Steering along the representation manifold allowed output probabilities to transition smoothly from Monday to Friday in sequence, while linear steering cut across the behavior manifold and produced noisy non-weekday outputs. The research also reverse-optimized steering paths along the behavior manifold and found their shapes matched the representation manifold paths, indicating a bidirectional correspondence between behavior geometry and representation geometry. The paper further found similar structures in months, letters, ages, synthetic in-context learning tasks, and image-action models.
Using the weekday cycle concept as an example, it demonstrates the correspondence between representation manifolds and behavior manifolds, providing a geometric perspective for representation steering.