Skip to content

The Right Tokens, the Right Time: From Per-Token Gating to Trajectory-Aware Activation Steering

Right Token, Right Time: From Per-Token Gating to Trajectory-Aware Activation Steering

arXiv: 其他 AI 安全论文· arXiv:2611.18274· Nima Eshraghi, Lovedeep Gondara, Sagarika Suresh, Xiaotong Xu, Jithin Pradeep, Fanny ChevalierThe Vanguard Group, Inc.; University of TorontoIndexed by this site · 4 days ago· Originally publishedAlignment· Its place in the scan record →

Translation in progress — untranslated text is omitted.

Quick read

Defenses

Based on the paper PDF (AI-generated); refer to the original

AI summaryThe paper studies activation steering of LLMs with sparse autoencoders (SAEs) to control model behavior without task-specific fine-tuning; it proposes the timing of intervention as a decisive control dimension, models steering as a per-token allocation problem under a fixed steerable-token budget, and gives two gating methods. The authors note that standard methods apply the steering signal at every generated token, which pushes representations off the training distribution and harms generation quality. Memoryless Contrastive Feature Projection (CFP) uses sparse interpretable feature projections to select the tokens least aligned with the target behavior; trajectory-aware Closed-Loop Adaptive Bernoulli Steering (CLABS) treats allocation as closed-loop feedback control, tracking the alignment accumulated so far in generation and spending budget where the trajectory lags. The authors derive an explicit gain region on a linear steering response model that guarantees CLABS's closed-loop stability, and show that its integral term can reduce steady-state tracking error.

Questions
Activation steering typically applies the same intervention to every generated token, which the authors say accumulates deviation from the training distribution and harms fluency as strength increases. The paper studies on which tokens steering should be applied under a fixed intervention budget.
Methods
Models per-token steering as an allocation problem under a fixed budget p, with the direction constructed from SAE feature differences. CFP steers the least aligned tokens by an instantaneous alignment threshold; CLABS sets the steering probability from cumulative mismatch using a PI controller with a leaky integral, and gives stability conditions under a linear response model.
Experiments and results
On Gemma 2 2B and Llama 3.1 8B, across the three targets of detoxification and fear/sadness steering, the authors state that CFP and CLABS have the largest behavior shift at matched budgets and the largest lead at small budgets; in the AlpacaEval detoxification setting, they are the only gates whose win rate exceeds No-Steer at every budget under both judges.
Limitations and future work
The authors state in Appendix E: it covers only two open-source base models and three English targets, depends on the quality of the target-layer SAE, uses only a single layer and a single direction, relies entirely on automatic classifiers and LLM judges for evaluation with no human evaluation, and the stability conclusions hold only for the linearized internal operating point.

Related papers

Show 3 more papers
Read originalarxiv.org