Debate with Images: Detecting Deception in Multimodal Large Language Models

Researchers propose MM-DeceptionBench, which the paper states is the first benchmark specifically designed to evaluate deception in multimodal large language models, covering six categories of deception that manipulate and mislead through combined visual and textual means. Addressing the problem that the covert nature of multimodal deception renders behavior monitoring and chain-of-thought monitoring largely ineffective, the paper proposes the debate with images multi-agent debate monitoring framework, which forces models to support their judgments with visual evidence. Experiments show that this method improves consistency with human judgment on all tested models, increasing Cohen's kappa by 1.5 times and accuracy by 1.25 times on GPT-4o.