The Right Tokens, the Right Time: From Per-Token Gating to Trajectory-Aware Activation Steering
Right Token, Right Time: From Per-Token Gating to Trajectory-Aware Activation Steering
Translation in progress — untranslated text is omitted.
The paper studies activation steering of LLMs with sparse autoencoders (SAEs) to control model behavior without task-specific fine-tuning; it proposes the timing of intervention as a decisive control dimension, models steering as a per-token allocation problem under a fixed steerable-token budget, and gives two gating methods. The authors note that standard methods apply the steering signal at every generated token, which pushes representations off the training distribution and harms generation quality. Memoryless Contrastive Feature Projection (CFP) uses sparse interpretable feature projections to select the tokens least aligned with the target behavior; trajectory-aware Closed-Loop Adaptive Bernoulli Steering (CLABS) treats allocation as closed-loop feedback control, tracking the alignment accumulated so far in generation and spending budget where the trajectory lags. The authors derive an explicit gain region on a linear steering response model that guarantees CLABS's closed-loop stability, and show that its integral term can reduce steady-state tracking error.
- Activation steering typically applies the same intervention to every generated token, which the authors say accumulates deviation from the training distribution and harms fluency as strength increases. The paper studies on which tokens steering should be applied under a fixed intervention budget.
- Models per-token steering as an allocation problem under a fixed budget p, with the direction constructed from SAE feature differences. CFP steers the least aligned tokens by an instantaneous alignment threshold; CLABS sets the steering probability from cumulative mismatch using a PI controller with a leaky integral, and gives stability conditions under a linear response model.
- On Gemma 2 2B and Llama 3.1 8B, across the three targets of detoxification and fear/sadness steering, the authors state that CFP and CLABS have the largest behavior shift at matched budgets and the largest lead at small budgets; in the AlpacaEval detoxification setting, they are the only gates whose win rate exceeds No-Steer at every budget under both judges.
- The authors state in Appendix E: it covers only two open-source base models and three English targets, depends on the quality of the target-layer SAE, uses only a single layer and a single direction, relies entirely on automatic classifiers and LLM judges for evaluation with no human evaluation, and the stability conclusions hold only for the linearized internal operating point.