FraudBench: A Multimodal Benchmark for Detecting AI-Generated Fraudulent Refund Evidence

Researchers propose FraudBench, a multimodal benchmark for detecting AI-generated fraudulent refund evidence, targeting the fraud risk of using synthetic or tampered images to support refund claims such as product damage, poor delivery conditions, or service defects. The benchmark draws on real user review evidence from e-commerce, food delivery, and travel service scenarios; it first uses MLLM-assisted filtering and human annotation to identify genuine damaged and undamaged evidence images, then uses 12 advanced image editing and generation models to synthesize fake damage evidence from real undamaged reference images. Comparing multimodal models, specialized AI-generated image detectors, and human evaluators on the same evaluation images, the results show that multimodal models can mostly identify genuine damage evidence, but their fake damage detection rate (TPR) on most generator subsets is far below the 50% baseline; specialized detectors perform better overall but are inconsistent across generators and produce false positives on genuine damaged samples, indicating that a clear gap remains between general AI image detection and reliable refund evidence verification that incorporates the specific claim context.