Skip to content
OpenAI Deployment Safety·Indexed by this site · Originally published· Evaluation →· Its place in the scan record →SelectedAttention37

OpenAI: CSAM image classifier precision is lower in online evaluation than offline

OpenAI CSAM Image Classifier Model Card

AI summary

OpenAI released a Model Card for a CSAM image classifier, which is used to detect child sexual abuse material (CSAM); training and evaluation use controlled-access, human-labeled datasets that comply with U.S. law, sensitive material is stored and processed in a controlled environment with restricted access, and there are legal, privacy, safety and reviewer health protection measures. The card explains why precision on the online evaluation set is markedly lower than on the offline holdout set: the share of CSAM in the holdout set is far higher than the level observed online, and when CSAM is rare, a small number of mislabeled non-CSAM images can exceed the true positives, thereby lowering precision, so the online estimate better reflects the composition of flags that human reviewers see on ChatGPT. Across multiple rounds of training, OpenAI used previously reviewed samples from online evaluations to improve the identified error types, and each round of training exposed new false-positive patterns.

Why it was selected

The card explains the low-base-rate effect by which online precision is lower than on an offline holdout set, and is of reference value to teams deploying classifiers that detect low-frequency events.

Read originaldeploymentsafety.openai.com