GuardPaint: Speculative Safety Decoding for Text-to-Image Generation

The researchers propose GuardPaint, a safety decoding framework for text-to-image generation that enforces protection inside the diffusion trajectory without modifying the base model. Existing defenses mostly filter prompts before generation or classify images after generation; the diffusion process itself lacks protection, and they often can only refuse rather than safely repair. GuardPaint uses a lightweight auditor to monitor intermediate images and locate unsafe regions, triggering targeted inpainting repair only where needed; candidate repairs are generated by a policy-aligned inpainter and screened through a tournament with guardrails, and only edits that simultaneously improve policy compliance while preserving prompt fidelity and perceptual quality are accepted. On five types of jailbreak attacks—SneakPrompt, MMA, PGJ, DACA, and RABell—and on UNet and flow-matching models such as SD 1.5, SDXL, SD 3.5, and FLUX.1-dev, GuardPaint reduced attack success rates and harmful generations, with only slight decreases in image quality, prompt fidelity, and normal behavior.