Counter Turing Test competition findings: AI-generated image detection

Results from the Counter Turing Test (CT2) competition show that AI-generated images can be detected with relatively high accuracy (F1 score above 0.83), but identifying the specific model used to generate an image remains markedly difficult (highest F1 score 0.4986). The competition was organized by the Defactify 4.0 workshop and set two tasks: judging whether an image is AI-generated or real, and identifying the specific generative model. The competition used the MS COCOAI dataset, containing 96,000 real and synthetic images, with synthetic images generated by five state-of-the-art models and real images from MS COCO. Participants employed a variety of detection methods, including CNNs, Vision Transformers, frequency analysis, contrastive learning, and multimodal approaches. The authors state that these findings indicate the need for improved model fingerprinting, adversarial robustness, and real-time detection mechanisms.