Skip to content

SafeCap: Improving LVLM Safety with Image Description Reinforcement Learning

SafeCap: Improving LVLM Safety with Image Captioning Reinforcement Learning

arXiv· arXiv:2608.10513· Caoyuan Ma, Wenpu Liu, Weichu Xie, Tian Gu, Shilei Zhao, Lingxi Min, Shuai Dong, Yuqi Xu, Ji Zhao, Ziyue Wang, Wenzheng Chang, Taiqiang Wu, Yongfu Zhu, Wenqi Shao, Yinqiang ZhengThe University of Tokyo; JD.com; Peking University; Wuhan University; Shanghai AI Laboratory; Shanghai Innovation Institute; Shanghai Jiao Tong University; The University of Hong KongIndexed by this site · 20 days ago· Originally publishedDefenses

Quick read

Defenses

Based on the paper PDF (AI-generated); refer to the original

The authors propose SafeCap, which uses GRPO to train an LVLM to first generate safety-related image descriptions and then answer; the authors state that under four model settings, DirectCap safety average scores improve by 3.7–19.0 points.

Threat modelDefense paper. Targeting jailbreak attacks that use visual input to bypass language-side safety alignment: harmful instructions can be embedded in typographic images, implied by objects, or become unsafe only after visual grounding. The defense is training-time alignment: using GRPO on public SPA-VL data to train a policy LVLM to first generate a labeled image caption and then answer, with the caption checked by a frozen text-only LLM for whether it supports a safe and consistent answer.

Questions
The visual pathway of LVLMs bypasses the safety alignment inherited from the language backbone: harmful instructions can be hidden in typographic images, implied by objects, or appear only after visual grounding, and text-only safety training does not cover these failure modes.
Methods
SafeCap uses GRPO on public SPA-VL data to train an LVLM to first output a labeled image caption and then answer; the reward consists of a format gate, an answer reward (utility multiplied by a risk discount), and a caption reward (awarded only if a frozen text-only LLM answers from the caption alone and the safety judgment is consistent).
Experiments and results
On 5 safety benchmarks and 6 visual utility benchmarks, the authors state that under the DirectCap protocol the safety average scores of the four model settings improve by 3.7–19.0 points while utility is largely maintained; with the same backbone and data, it outperforms safety SFT, DPO, and SafeGRPO.
Limitations and future work
The authors note: the reward depends on the capability of the frozen LLM, cannot identify factual errors in the descriptions, and the safety reward is not fully verifiable; experiments use only smaller Qwen3.5 variants, and some supplementary experiments use early-stopped checkpoints.
Experimental setupQwen3.5-2B、Qwen3.5-2B-Base et al. · MM-SafetyBench、MSSBench et al. · 1 − ASR、S-Avg et al.
Evaluated models
Qwen3.5-2BQwen3.5-2B-BaseQwen3.5-4BQwen3.5-4B-Base
Benchmarks
MM-SafetyBenchMSSBenchVLSBenchFigStepMIS-TestMM-VetBLINKMMVPERQAVPCTMMStarSPA-VLSafeTag-VL-3K
Metrics
1 − ASRS-AvgV-AvgMSSBench overall accuracyVLSBench safe rate
AI summarySafeCap targets jailbreak attacks that exploit visual inputs, using image captioning reinforcement learning to improve LVLM safety: the policy model first generates safety-related captions and then answers, and the captions are optimized by whether a frozen LLM can make a safety-aligned decision based on them. On five multimodal safety benchmarks and six visual utility benchmarks, under four model settings the safety average score improves by 3.7-19.0 points, with no decrease in visual utility, and in controlled experiments it surpasses safety SFT, DPO, and SafeGRPO.

SafeCap targets jailbreak attacks that exploit visual inputs, using image captioning reinforcement learning to improve LVLM safety: the policy model first generates safety-related captions and then answers, and the captions are optimized by whether a frozen LLM can make a safety-aligned decision based on them. On five multimodal safety benchmarks and six visual utility benchmarks, under four model settings the safety average score improves by 3.7-19.0 points, with no decrease in visual utility, and in controlled experiments it surpasses safety SFT, DPO, and SafeGRPO.

Paper analysis

6 questions; open one to read the full answer

  1. What problem does this paper attempt to solve?

    To address the problem of visual input bypassing the language-side safety alignment of LVLMs, the authors propose a training-time self-captioning reinforcement learning framework, SafeCap.

    Core problem: the visual pathway of LVLMs can weaken or bypass the safety alignment inherited from the language backbone, causing the model to produce unsafe answers under combined image-text inputs.

    • Problem setting: The authors state that text-only models can rely on language-side refusal strategies, whereas LVLMs must first perceive safety-relevant evidence from the image before deciding to answer, warn, or refuse; harmful instructions can be embedded in typographic images, danger intent can be implied by objects, or the input can become unsafe only after visual grounding.
    • Shortcomings of existing methods: The authors state that inference-time description-mediated defenses (such as ECSO) may lose fine-grained visual details when converting images to text and rely on the downstream language model's own safety capability; training-time methods face a trade-off between improving refusal and maintaining visual utility and avoiding over-refusal.
    • Core observation: The authors argue that if the model first generates a description containing safety-relevant visual evidence and then answers based on it, visual risks can be exposed to the language-side safety mechanism; the key is reward design—a reward that only rewards safety will bias toward refusal, while a reward that only rewards descriptions cannot guarantee the final answer is safe.
    • Paper's approach: Proposes SafeCap, a reinforcement learning framework that trains a policy LVLM to first generate a labeled image description for each image-text pair and then give the final answer, and whether the description helps a frozen text-only LLM reach a safety-consistent conclusion also enters the reward.
    • Threat model (a defense paper; the attack surface addressed):
      • Attacker goal: use visual input to make the LVLM give actionable help to harmful requests (bypassing safety alignment).
      • Attack carriers: harmful instructions in typographic images (FigStep), hidden intent in images with harmless text (HADES), visual flowcharts encoding attack logic, and multi-image combinations producing unsafe intent (MIS).
      • Victim system: LVLMs that inherit safety alignment from a language backbone; the defender is the trainer, who can modify model parameters during training.
      • Success criterion: on safety benchmarks, metrics such as 1 − ASR measure whether the model refuses or avoids actionable harmful information.
  2. What related research exists?

    Related work is divided into four groups: safety risks introduced by visual input, inference-time and caption-mediated defenses, training-time alignment, and caption rewards; this paper is positioned as training the LVLM's own caption-answer pathway.

    Safety risks introduced by visual input

    • MM-SafetyBench and VLSBench (Liu et al. 2024; Hu et al. 2025)—evaluate the risk in seemingly harmless image-text pairs where harmful intent must be inferred from the visual input.
    • FigStep (Gong et al. 2023) converts harmful instructions into incomplete typographic images; image-to-text logic jailbreak (Zou, Li, and Chen 2024) encodes the attack logic into visual flowcharts; HADES (Li et al. 2024) hides safety-critical intent in images; MIS (Ding et al. 2025) extends the risk to multi-image combinations.
    • Pantazopoulos et al. 2024 and Lee et al. 2024—analyze how visual instruction tuning and cross-modal projection themselves may weaken the refusal behavior of language models.

    Inference-time and caption-mediated defenses

    • Inference-time defenses include defense prompts (Wang et al. 2024b; Mo et al. 2024), visual or multimodal guardrails (Oh et al. 2024; Hao et al. 2025), cross-model or reward-guided decoding (Wang et al. 2024a; Ghosal et al. 2025), and representation-level calibration (Zou et al. 2025; Zhang et al. 2025a).
    • ECSO (Gou et al. 2024)—first detects unsafe answers, then converts the visual input into a query-oriented text description, using the text-only pathway to generate safer answers; the authors point out that it depends on the information retained after conversion and on the downstream model's safety capability, and image-to-text may lose details needed for safety decisions.
    • Image Textualization (Pi et al. 2024b) generates inspectable, filterable image captions; MLLM-Protector (Pi et al. 2024a) detoxifies harmful outputs after generation.

    Training-time alignment and caption rewards

    • SPA-VL (Zhang et al. 2024) provides LVLM safety preference data; VLGuard (Zong et al. 2024) studies safety fine-tuning; MM-RLHF (Zhang et al. 2025b) explores multimodal human feedback alignment; Chakraborty et al. 2024 studies whether text safety alignment can transfer to multimodal.
    • CapRL (Xing et al. 2025) converts open-ended caption quality into decoupled VQA signals to reduce reward hacking; the authors state that they are inspired by this, introduce a binary LLM safety-consistency judgment into the caption-mediated reward, and adapt caption-mediated reinforcement learning from perceptual VQA to the safety setting.

    Baselines and benchmarks: the comparison baselines are safety SFT, DPO (Rafailov et al. 2023), and SafeGRPO (Rong et al. 2025, compared on its released SafeTag-VL-3K data); evaluation uses 5 safety benchmarks (MM-SafetyBench, MSSBench, VLSBench, FigStep, MIS-Test) and 6 visual utility benchmarks (MM-Vet, BLINK, MMVP, ERQA, VPCT, MMStar).

    The authors position this paper as follows: unlike methods that wrap a frozen model at inference time, SafeCap trains the LVLM's own caption-answer pathway; unlike SafeGRPO, which relies on additional safety annotations, SafeCap uses only public SPA-VL data.

  3. How does the paper address this problem?

    SafeCap uses GRPO to train an LVLM to first generate a labeled safety-relevant image caption and then answer, with the reward composed of a format gate, an answer reward, and a caption-mediated reward combined after group normalization.

    Overall idea: formulate LVLM safety alignment as a self-captioning problem—the model first outputs a trainable intermediate representation (image caption), then gives the final answer; the caption is designed as a safety-relevant evidence channel, and a frozen text-only LLM checks whether it supports a safe and consistent answer.

    Problem setting

    • Each training sample is an image-text pair (x, q), and the policy model πθ generates a structured output y = ⟨ caption ⟩ c⟨/caption⟩ a, where c is the image caption and a is the final answer; this format corresponds to the DirectCap operating point in the experiments.
    • Design goal: c should expose the visual details that determine whether the request is benign, and a should use this evidence to answer safely; the authors emphasize that this is not simply prepending a lengthy description to every answer.

    Training objective

    • Optimize with GRPO (Shao et al. 2024) on public SPA-VL safety alignment data, without introducing proprietary utility data; sample K = 8 rollouts per prompt.
    • Three signals form the outcome-level advantage: Ai = wtmpr̃i,tmp + wcapr̃i,cap + wansr̃i,ans, i.e., the template, caption, and answer components are each normalized within the group and then weighted and summed, used in the PPO-clipped GRPO loss.
    • The main run does not enable an explicit KL loss or KL reward penalty; the frozen LLM and the judge model are only used to compute rewards and are never updated.

    Three reward components

    • Template reward: a rule-based parser extracts the caption and answer; it requires exactly one complete caption block, non-empty caption and answer, and termination within the length limit; if the gate fails, the remaining rewards are skipped. The authors state that this gate is part of the reward design, preventing rollouts with format degradation or endless repetition from scoring.
    • Answer reward: the judge model scores the final answer with two 0–5 integer scores—a preservation score u(a) (the degree to which legitimate information needs are met) and a risk score h(a) (the degree of harmful content); the reward takes the exponential risk-discount form Rans = u(a)γh(a), γ ∈ (0, 1), where each additional point of risk multiplies the utility contribution by γ. The authors state that this multiplicative form makes high-risk answers quickly lose value while retaining positive reward for safe and useful answers, avoiding the situation under linear rewards where low-information captions or generic refusals instead prevail.
    • Caption reward: the caption c together with the original question is given to a frozen text-only LLM (which cannot see the image) to obtain an answer af; the judge gives the caption a 0–5 coverage score u(c) (concrete coverage of the subject, attributes, actions, spatial layout, scene, and visible text), and then a binary safety-consistency judge compares whether the safety judgments of a and af are consistent, yielding g ∈ {0, 1}; the caption reward is Rcap = g(q, a, af ) · u(c). The authors state that this design prevents captions from scoring merely for being detailed, requiring the caption to support an independently generated text-only answer that reaches the same safety conclusion as the policy answer.

    Normalization and key parameters

    • The three components are normalized separately within the rollout group (subtract the group mean, divide by the group standard deviation, ε = 1e−6) and then combined with weights; the authors state that this follows the decoupled normalization idea of GDPO but does not perform its batch-level whitening, in order to prevent components with different scales from coupling before credit assignment.
    • Key parameters: γ = 0.35, reward weights (wtmp, wcap, wans) = (0.5, 0.5, 1.0); the frozen text-only model is Qwen3-4B, and the judge model is gpt-oss-20b (deterministic decoding); trained for 200 steps, learning rate 5×10−7, prompt and response limit 4096 tokens (Appendix A).

    Inference and diagnostic protocols

    • DirectCap (target operating point): the trained policy first writes a caption and then answers in the same response.
    • Direct (diagnostic): removes the caption requirement, testing whether RL training harms native image-text QA behavior.
    • Prism (diagnostic): gives the generated caption to a frozen text-only LLM to answer, isolating whether the caption itself contains sufficient transferable visual evidence.
  4. What experiments did the paper conduct?

    Evaluate on 5 safety benchmarks and 6 utility benchmarks across 4 Qwen3.5 settings × 3 inference protocols, with controlled comparisons against SFT, DPO, and SafeGRPO, plus reward ablations and a seed robustness check.

    Experimental setup

    • Models tested: four settings, Qwen3.5-2B, Qwen3.5-2B-Base, Qwen3.5-4B, and Qwen3.5-4B-Base, all trained with GRPO on public SPA-VL data for 200 steps.
    • Benchmarks: on the safety side, MM-SafetyBench, MSSBench, VLSBench, FigStep, and MIS-Test; on the utility side, MM-Vet, BLINK, MMVP, ERQA, VPCT, and MMStar; the paper does not report the sample size of each benchmark.
    • Metrics: MM-SafetyBench, FigStep, and MIS-Test report 1 − ASR; MSSBench reports its official overall accuracy; VLSBench reports the safe rate of its multimodal judge; S-Avg and V-Avg are the means of the 5 safety and 6 utility metrics, respectively.
    • Judging method: the judge model is gpt-oss-20b (deterministic decoding); the original FigStep benchmark relies on human evaluation, so the authors use the automatic harmful/safe judging prompt from DAVSP and conduct a stratified-sample human review; the frozen text-only LLM is Qwen3-4B.
    • Protocols: Direct, DirectCap, and Prism; superscript numbers are percentage-point changes relative to the zero-training model under the same protocol.
    • Number of repetitions: the main results are from a single training run; a robustness check is additionally performed with 3 seeds (42, 43, 44) at the 100-step early-stopping checkpoint.

    Main results

    Changes after SafeCap training relative to zero training (according to Table 2, S-Avg / V-Avg, superscripts are percentage-point changes):

    Swipe horizontally to read the table

    ModelProtocolS-AvgV-Avg
    2BDirectCap45.98 (+4.0)51.96 (+6.4)
    2B-BaseDirectCap45.40 (+3.7)46.12 (+7.1)
    4BDirectCap57.06 (+6.6)56.49 (+4.5)
    4B-BaseDirectCap59.39 (+19.0)53.05 (−0.1)
    • The authors state that the gains are most consistent under DirectCap: the 11-benchmark overall average scores of the four settings improve by 5.29, 5.57, 5.48, and 8.57 points, respectively; DirectCap utility also improves for 2B, 2B-Base, and 4B, and the 4B-Base utility average is close to the zero-training level.
    • The authors state that the Direct protocol depends more on the model: the safety average scores of the base-initialized 2B-Base and 4B-Base improve (+8.1, +12.5), but the Direct safety average scores of the instruction-tuned 2B and 4B decrease (−2.1, −2.2).
    • The authors point out a zero-training artifact: Qwen3.5-2B-Base has a relatively high FigStep score under DirectCap; in a 500-sample comparison, 66 of the 81 answers that went from safe to harmful had zero-training answers that merely described a blank template, which the authors regard as a protocol-specific artifact rather than stronger safety behavior.

    Comparison with safety fine-tuning and SafeGRPO

    • SFT/DPO comparison (Qwen3.5-4B-Base, same SPA-VL data and training steps, Appendix F Table 10): the DirectCap S-Avg of SFT and DPO rises only to 43.19 and 41.36, while SafeCap reaches 59.39, and V-Avg stays near the zero-training value (53.05 vs. 53.13); SFT exhibits DirectCap instruction-following failures (malformed caption block format), so the authors did not perform a Prism comparison for SFT/DPO.
    • SafeGRPO comparison (same Qwen3.5-4B-Base initialization, each trained for 200 steps on SafeGRPO's SafeTag-VL-3K data, Table 3): SafeCap's DirectCap S-Avg is 55.06 and V-Avg 51.34, while SafeGRPO's are 41.27 and 50.56; under Base (direct) inference, SafeCap's safety of 46.86 is also higher than SafeGRPO's 44.98. The authors state that SafeCap does not rely on SafeGRPO's additional safety annotations.

    Ablations and diagnostics

    • Reward component ablation (Table 4, percentage-point changes relative to full SafeCap): removing the answer reward lowers Direct and DirectCap safety by 5.31 and 6.49 points, respectively; removing the caption reward lowers safety by 1.08–2.53 points and utility by 1.23–2.41 points; removing decoupled normalization lowers Direct and DirectCap safety by 2.74 and 2.02 points, with the only exception being a +0.42 direct utility.
    • Risk coefficient ablation (Table 5, early-stopping checkpoint): γ = 0.20 (stronger risk penalty) converges faster than γ = 0.35, with DirectCap safety +1.68; the authors state that γ = 0.35 is a balanced choice.
    • Alternative reward forms (Appendix B Table 7, 200 steps): the linear Classic form raises DirectCap safety by +5.13 but lowers utility by −0.85; the Lagrangian form gives DirectCap safety +3.32 and utility −1.04; Rank-2 lowers both safety and utility under both protocols. The refusal diagnostic (Table 8) shows that Classic has a higher refusal rate on safety benchmarks (e.g., FigStep 63.4 vs. 55.2 for the full version), and the authors believe that part of the safety score may come from stronger refusal behavior.

    Other analyses

    • Seed robustness (Appendix A Table 6): the three-seed 100-step 4B-Base DirectCap S-Avg is 50.83 ± 1.14, and V-Avg 54.32 ± 1.14 corresponds to 54.32 ± 1.54; a one-sample t-test against the zero-training DirectCap S-Avg (40.43): improvement of 10.40 ± 1.14 points, t(2) = 15.76, p = 0.0040; utility change +1.19 ± 1.54, p = 0.313, which the authors state is no evidence of utility decline. This test uses the early-stopping checkpoint and is not directly comparable to the main table.
    • Prism generalization (Appendix E Table 9): with the same 4B-Base SafeCap checkpoint, replacing the frozen LLM from Qwen3-4B with Qwen3-14B, which was not seen during training, raises the Prism safety average from 39.62 to 44.34 (+4.72), with all five safety benchmarks improving; the authors state that the safety-relevant evidence in the caption remains usable for a stronger text-only reasoner, but the Prism score is affected by the downstream LLM's capability.
    • Human review (Appendix G): 100 responses were stratified-sampled from all evaluation benchmarks for human review; after discussion the reviewers reached full agreement, consistent with the judge model's labels 100/100.
    • Length diagnostic (Appendix A): after training, the average response length of 4B-Base increases by only about 100 tokens, and the authors believe that the safety gains cannot be mainly explained by longer responses.
  5. What are some points that can be further explored?

    The authors note limitations such as the reward depending on frozen LLM capability, the inability to identify factual errors in descriptions, and the safety reward not being verifiable, and plan to explore description-mediated SFT and larger models.

    Limitations and future directions noted by the authors

    • The effectiveness of the description reward is limited by the capability of the frozen LLM that interprets the descriptions; the frozen LLM cannot see the images, and the reward itself cannot identify factual errors in the descriptions (Appendix D).
    • The answer reward uses a fixed human scoring rubric rather than a fully verifiable safety reward; the authors state that developing more verifiable safety and risk rewards is an open problem for the community (Appendix D).
    • For cost reasons, the experiments use smaller Qwen3.5 variants; the authors state that evaluating larger models and more model families when compute is sufficient is an important next step, and that the method itself does not require Qwen-specific adaptation (Appendix D).
    • The authors state that a promising extension is description-mediated SFT before RL, but supervision data suitable for this intermediate objective is currently limited (Appendix D).
    • Some supplementary experiments use early-stopped checkpoints, and the authors explicitly note that these results are not directly comparable to the main results from full training (Appendix D).

    Scope of experimental coverage

    • The models tested are Qwen3.5-2B, Qwen3.5-2B-Base, Qwen3.5-4B, and Qwen3.5-4B-Base, 4 settings in total; the training data is the public SPA-VL; the comparison with SafeGRPO is conducted on its SafeTag-VL-3K data (Experiments, Appendices A, F).
    • Evaluation covers 5 safety benchmarks and 6 visual utility benchmarks, with three inference protocols (Direct, DirectCap, Prism); the paper does not report the sample size of each benchmark.
    • The main results are from a single training run; the seed robustness check uses 3 seeds and 100-step early-stopped checkpoints (Appendix A).
    • The frozen text-only LLM is Qwen3-4B, with an additional Prism generalization check using Qwen3-14B in Appendix E; the judge model is gpt-oss-20b.
    • The human consistency check uses a stratified sample of 100 responses, which agrees with the judge model labels 100/100 (Appendix G).
  6. Summarize the main content of the paper

    SafeCap turns LVLM safety alignment into a trainable self-captioning pathway; the authors report that under the DirectCap protocol the safety average score improves by 3.7–19.0 points while utility is largely maintained.

    SafeCap is a reinforcement learning framework for LVLM safety alignment, targeting the problem of visual input bypassing language-side safety alignment.

    • Problem: the authors state that harmful instructions can be embedded in typographic images, implied by objects, or appear only after visual grounding, which text-only safety alignment does not cover; existing inference-time caption-mediated defenses lose visual details, while training-time methods must balance refusal and utility.
    • Method: use GRPO on public SPA-VL data to train an LVLM to first generate a labeled image caption and then answer; the reward consists of a format gate, an answer reward (utility multiplied by γ to the power of the risk score, γ = 0.35), and a caption reward (scored only if frozen Qwen3-4B answers from the caption alone and the safety judgment is consistent), with the three components normalized within the group and then weighted.
    • Main results: on 5 safety benchmarks and 6 utility benchmarks, the DirectCap safety average scores of the four model settings (Qwen3.5-2B/2B-Base/4B/4B-Base) improve by 3.7–19.0 points relative to zero training, with 4B-Base improving by 19.0 points and its utility average essentially unchanged (Table 2).
    • Controlled comparisons: with the same backbone, data, and steps, SafeCap's DirectCap S-Avg (59.39) is higher than safety SFT (43.19) and DPO (41.36); on SafeGRPO's SafeTag-VL-3K data, SafeCap DirectCap S-Avg is 55.06 vs. SafeGRPO's 41.27 (Tables 3, 10).
    • Ablations and checks: removing the answer reward lowers DirectCap safety by 6.49 points; the safety improvement in the three-seed early-stopping check is statistically significant (p = 0.0040); human review of 100 responses fully agrees with the judge model.
    • Authors' conclusion: the authors believe that incorporating perception-side evidence into multimodal safety alignment is valuable and complements output-side alignment; they also note that the reward is limited by the frozen LLM's capability, cannot identify factual errors in captions, and that the safety reward is not verifiable, and that the experiments are limited to smaller Qwen3.5 variants.

Related papers

Show 3 more papers
Read originalarxiv.org