Automated adversarial attack discovery for safety classifiers

The researchers proposed and formalized the task of "automated adversarial discovery," whose goal is to find new attacks that attack safety classifiers along previously unseen harm dimensions, and measure progress with two metrics: attack success rate and dimension diversity. Evaluation on the CivilComments toxicity task shows that word-perturbation attacks struggle to fool the classifier, prompt-based LLM attacks have higher success rates but insufficient dimension diversity, and even the best-performing prompt method achieves only 5% of attacks succeeding on unseen harm dimensions. The authors note that automatically discovering new harm attack dimensions still leaves substantial room for research.