Certified Robustness under Heterogeneous Perturbations via Hybrid Randomized Smoothing

The researchers propose a hybrid randomized smoothing framework that provides certified robustness for joint perturbations of discrete text and continuous images in multimodal models. Existing randomized smoothing certificates cover only a single modality and struggle to handle scenarios where an attacker perturbs multiple inputs simultaneously. The new framework is based on Neyman-Pearson joint worst-case modeling; by analyzing the joint likelihood ranking induced by decomposed discrete and continuous noise, it obtains a closed-form one-dimensional certificate, strictly generalizing image-only Gaussian smoothing and text-only discrete smoothing. The authors state that they have validated the framework on interaction-dependent text-image safety filtering, and state that this is the first model-agnostic Neyman-Pearson certificate for joint perturbations of discrete tokens and continuous images.