A holistic approach to real-world harmful content detection

The researchers propose a method for building classification systems for real-world content moderation, covering the complete process from taxonomy design to model robustness. The method chain includes designing the content taxonomy and annotation instructions, data quality control, an active learning process for capturing rare events, and various means of improving robustness and avoiding overfitting. The system is used to detect multiple categories of harmful content such as pornography, hate, violence, self-harm and harassment; the authors state that this approach can be generalized to different content taxonomies, and that the trained high-quality content classifiers outperform off-the-shelf models. The paper was presented as an oral report at AAAI-23.