Brennan Center retests six chatbots on identifying source metadata of AI images

After California's AI transparency law took effect, the Brennan Center retested six chatbots' ability to identify AI-generated images, and the results showed no improvement. The tests covered ChatGPT, Claude, Gemini, Grok, Meta AI, and Perplexity, using images generated by Gemini and ChatGPT for verification. Only Gemini read the source metadata embedded in the images and identified the creation tool; the other chatbots did not evaluate metadata and relied only on visual judgment. For an image generated by Gemini of postal workers sorting election mail, four chatbots misjudged it as a real photo, with ChatGPT and Perplexity noticing the visible watermark in the lower right corner but interpreting it as a social media or platform mark. The researchers noted that the law requires companies to provide free tools for users to verify whether images are AI-generated, but does not require that tool to be embedded within chatbots, and recommended integrating verification capabilities directly into chatbots.