A Unified Evaluation Framework for Physical Adversarial Robustness of Edge VLMs
Start here
A study released on October 21, 2026 proposes a unified evaluation framework that uses conditional attack success rate (cASR) to measure the robustness of vision-language models and object detectors to physical adversarial patches. The metric is computed only on images where the model can still localize under a matching gray occluder, and failures are divided into three categories: missed detection, replacement, and displacement. The study finds that the vulnerability of compact visual localization VLMs can be comparable to that of object detectors, and that failure modes are mainly related to model output format rather than architecture family; among the three failure types, only missed detection can be noticed by downstream systems, while displacement and replacement return high-confidence, normally formatted boxes and are potentially more dangerous. Under distribution shift, VLMs that were originally dominated by missed detection shift more toward displacement.
Generated by AI from reports · Updated 22 days ago
Attention and developments
Attention, hourly
Now 37, 0% in 24 hours·Peak 37 (Nov 6 03:00)
Related research
This event's paper
- A Unified Evaluation Framework of Physical Adversarial Robustness in the Edge VLM Era
arXiv · 2026-10-21 · Deep reading
Progress timeline
- arXiv: Adversarial Attacks, Backdoors & PoisoningA Unified Evaluation Framework for Physical Adversarial Robustness in the Era of Edge VLMs
The researchers propose a unified evaluation framework that uses conditional attack success rate (cASR) to measure the robustness of vision-language models and object detectors to physical adversarial patches, finding that the fragility of compact visual grounding VLMs can be comparable to that of object detectors. This metric is computed only on images where the model can still localize under a matching gray occluder, and failures are divided into three categories: missed detection, substitution, and displacement. The study finds that failure modes are mainly related to model output format rather than architecture family; among the three categories, only missed detection can be noticed by downstream systems, while displacement and substitution return high-confidence, normally formatted boxes and are potentially more dangerous. Under distribution shift, VLMs that were originally dominated by missed detection shift more toward displacement and substitution failures. Code and data are open-sourced at https://github.com/ericliang1/physical-adversarial-robustness。