Benchmark Evaluation of Vision-Language Models Under Contradictory Virtual Content Attacks in Augmented Reality

Researchers propose the ContrAR benchmark to evaluate the robustness of vision-language models (VLMs) against contradictory virtual content attacks in augmented reality (AR). Such attacks mislead users, create semantic confusion, or convey harmful information by implanting malicious or inconsistent virtual elements into the user's field of view. ContrAR contains 312 real AR videos validated by 10 participants, and the team used it to test 11 VLMs spanning commercial and open-source models. Results show that existing VLMs have a reasonable level of understanding of contradictory virtual content, but there is still room for improvement in detecting and reasoning about adversarial content manipulation in AR environments, and balancing detection accuracy and latency remains a challenge.