Exploring the use of unsupervised methods to recover the refusal direction in large language models

The authors apply principal component analysis (PCA), Isomap and diff-in-means to recover the refusal direction of large language models, and test on Llama 8b 3.1 Instruct and Qwen2.5-7B-Instruct. The authors state that the direction recovered by PCA is close to the direction obtained by diff-in-means in cosine similarity and causal ablation effect, and that the required sample size is not large. Isomap consistently performs worse than PCA, and the authors accordingly conclude that there is no evidence that the activation geometry of refusal is nonlinear. The authors also use a projection threshold to evaluate classification performance on held-out harmful and harmless prompts, and the code has been released.