FAR.AI proposes Concept Influence: using interpretable vectors for training data attribution

FAR.AI proposes Concept Influence, which replaces the gradient of a single test output in standard influence functions with the gradient of a semantic concept vector, thereby attributing training data by behavior rather than literal strings. The method uses linear probes, SAE features, or crosscoder features as semantic directions. The paper shows that crosscoders can unsupervisedly detect data that inserts sleeper agent triggers without prior knowledge. The authors prove that efficient methods like Projection Difference and Vector Filter are actually first-order approximations of Concept Influence, running over 20 times faster than gradient methods, with comparable performance when training and evaluation domains are close. On real post-training data from OASST1, Vector Filter achieved instruction-following capability comparable to full data using only 5% of the safest data, while the evil score was significantly lower. The authors recommend using gradient methods when domain differences are large, and probe methods when domains are close.