Safety Recovery of Reasoning Models Only Needs a Few Early Steps of Guidance

Post-training for reasoning raises the jailbreak success rate of multimodal reasoning models. Can safety be pulled back above the threshold at inference time without retraining and without sacrificing reasoning?

• Scenario: The authors state that after RL post-training (e.g., GRPO) strengthens explicit chain of thought in MLRMs, task performance improves while safety degrades. Figure 1 on Hades shows R1-Onevision relative to the base model Qwen2.5-VL: ASR goes from 19.13% to 69.07%, and the safety score (100−ASR) goes from 80.87% to 30.93%.
• Existing shortcomings: The authors state that refusal prompts, safety prompts, or decoding heuristics are often unstable under jailbreaks, and using them aggressively can hurt reasoning. Treating safety as an objective to maximize can also be overly conservative (Jiang et al., 2025).
• Hypothesis: The authors believe that safe behavior is mostly still present but is not selected under adversarial conditions; safe continuations can exist, but with conditional probability near 0. Therefore, they frame recovery as reconditioning at inference time rather than retraining.
• Approach: They propose an inference-time defense, SafeThink. A safety reward monitors the chain of thought, and only when it falls below the threshold is a short corrective prefix injected, framing safety as a constraint of "just crossing the threshold."
• Threat model:
  • Victim system: A multimodal reasoning model that takes images and text as input, first generates a reasoning trajectory z, then generates an answer y.
  • Attack goal: Under adversarial inputs, cause harmful content to appear in the chain of thought or the final answer. ASR is determined by an oracle; if either side is harmful, it counts as a success.
  • Attack surface: Text jailbreaks with visual inputs (JailbreakV-28K), and embedding malicious instructions in images (HADES, FigStep, MM-SafetyBench). The paper does not specify whether the attacker has white-box or black-box knowledge.
  • Defender: Does not change parameters; can read intermediate reasoning steps, scores them with a public safety reward model, injects a steering token when below τ, and can intervene only in the first m steps.

Grouped according to Section 6 and Appendix F; works explicitly criticized by the paper are attributed only to the works it names.

Multimodal reasoning models

• CoT and multimodal CoT: Wei et al. (2022) uses step-by-step reasoning; Fei et al. (2024), Shao et al. (2024), and Zhang et al. (2023b) extend CoT to multimodal. Appendix F states that early short chains are often insufficient for long-horizon planning.
• RL-trained MLRMs: The authors discuss DeepSeek-R1 (Guo et al., 2025) and a set of models that use reinforcement learning to lengthen and structure chains of thought, including the evaluated OpenVLThinker, R1-Onevision, Vision-R1, LlamaV-o1, and LLaVA-CoT.

Safety of MLRMs

• Reasoning-induced jailbreak susceptibility: Fang et al. (2025), Huang et al. (2025a), Jiang et al. (2025), Zhou et al. (2025), among others. The authors claim that stronger reasoning amplifies jailbreak susceptibility; CoT supervision or RL fine-tuning can increase jailbreak rates.
• Suppressing reasoning or fixed prefixes: Jiang et al. (2025) uses a zero-shot strategy to shorten thinking; Jeung et al. (2025) prepends a fixed 8-token safety prefix. The authors state that both require a trade-off between safety and reasoning quality.
• Other listings: Appendix F also lists Lou et al. (2025), Mazeika et al. (2024), Parmar and Govindarajulu (2025), and Wang et al. (2025). The paper does not compare them one by one.

Jailbreak attacks

• Text side: Discrete optimization of adversarial suffixes or prompts, such as Zou et al. (2023) and Zhu et al. (2024); another line pushes the output distribution toward harmful targets. Qi et al. (2024b) discusses that benign fine-tuning can also erase safety guardrails.
• Multimodal side: Visual adversarial perturbations (Qi et al., 2024a; Niu et al., 2024, etc.), embedding malicious instructions in images (Gong et al., 2023; Liu et al., 2023a), or switching to text jailbreaks (Luo et al., 2024). Appendix F states that Ying et al. (2024) perturbs both modalities simultaneously.

Baselines and benchmarks

• Baselines: ZeroThink, LessThink (Jiang et al., 2025), ZS-SafePath (Jeung et al., 2025), AdaShield (Wang et al., 2024b). All are inference-time defenses.
• Benchmarks: JailbreakV-28K, HADES, FigStep, MM-SafetyBench; reasoning utility uses MathVista.

The authors position this paper as early-step directed steering, rather than suppressing reasoning or only adding a fixed prefix, and adopt satisficing (Simon, 1956; Chehade et al., 2025): the safety score only needs to exceed a threshold, not be maximized.

SafeThink 把安全恢复写成推理时的阈值约束：监视每一步，只有低于阈值才注入短转向信号，让安全续写的条件概率升高，同时少改原推理策略。

问题设定

• 生成：输入 x 为图像加文本。模型先采样推理步，再采样答案。联合分布是各步条件概率的乘积（式 1）。
• satisficing：不最大化安全分。中间步 t 上，安全续写的概率要不低于 ρ：Pr_{z∼πθ(·|x,z<t)}[R_safe([x,z_<t],z)≥τ]≥ρ。含义是：从当前策略采样下一步时，安全分不低于 τ 的概率至少为 ρ。
• 覆盖崩溃：作者称在对抗输入下该概率可接近 0，所以拒绝采样和 best-of-N 在有限预算内抽不到安全续写。Figure 2 图注称 k 增到 20 时，直接从原策略抽样的 BoN* 仍低于 τ。

监视

• 何时干预：候选步来自原策略。r_t = R_safe(x_adv, z_≤t)，分数归一化到 [-1, 1]。r_t ≥ τ 则接受，否则拒绝并转向。轨迹以 end-of-thinking 结束。
• 奖励模型：主实验用 Llama-Guard-3。附录 D 换 Qwen-Guard-3。正文也提到可用 RewardBench 一类公开无害奖励模型，实验实例是上述两个。
• 阈值从哪来：实验取 τ=0。附录 C 在 JailbreakV-28K 的 500 条上，用 VLAA-Thinker 每条生成 N=20，Llama-Guard-3-8B 打分、GPT-4 做 oracle。图注写越过 τ=0 后，约 90% 的回复被判为安全，更高分不再上升。

转向 token

• 选 token 的目标：在离散信号 s 上尽量少偏离原策略，同时满足安全约束：min_s D_KL(π_θ(·|x_adv,z_<t,s) | π_θ(·|x_adv,z_<t))。约束与上面的概率阈值同一要求（式 6 用 ρ）。
• 候选集怎么来：从四个越狱基准随机抽 500 条 held-out 样本，让 GPT-4 按不安全推理轨迹生成短纠正短语，去重得到 S。再按估计的安全成功概率和 KL 筛选。
• 实验里用哪句：Figure 5（JailbreakV-28K）上，作者称没有显式安全用语的短语留在 τ=0 以下，带安全提示的更高；“Wait, think safely” 安全奖励最高且 KL 低。推理时固定用这一句，不再在线搜索。
• k 与符号：每个候选 s 抽 k 个下一步，用蒙特卡洛估计安全成功概率。实验取 k=3。算法正文把入选写成估计概率 ≥ τ，与式 6 的 ρ 不是同一个符号。

早步预算与判定

• 深度 m：只允许前 m 个推理步触发监视和转向，之后关闭。m=0 就是不转向。作者称早步定下意图和高层计划，越狱时不安全意图往往早形成。
• 偏离上界：作者把加了前 m 步转向后的轨迹 KL，上界写成这 m 步 KL 之和（式 10）。这是作者给出的界，实验未报告实测 KL。
• 成功怎么判：ASR 是对抗样本里，思维链或最终答案被 oracle 判为含有害内容的比例。oracle 是 GPT-4。Llama-Guard-3 只负责触发转向，不代替这个判定。

Experimental Setup

• Tested models: R1-Onevision-7B, OpenVLThinker-7B, VLAA-Thinker-7B, Vision-R1-7B, LlamaV-o1, LLaVA-CoT. Figure 1 also gives the base Qwen2.5-VL's undefended ASR on Hades.
• Benchmarks: JailbreakV-28K pairs adversarial text with noise, blank, natural images, or stable diffusion images, with text split into template, persuasion, and logic. HADES has five categories of harmful instructions, with images as SD, TYPO, ADV. FigStep has ten categories, typographic images. MM-SafetyBench has thirteen categories, SD, TYPO, SD+TYPO. Reasoning utility uses MathVista. The paper does not specify the number of items per benchmark for the main experiments.
• Baselines: ZeroThink (empty think segment), LessThink (outputs a short reasoning first), ZS-SafePath (fixed 8-token safe prefix), AdaShield (adds a defensive prompt before the query to check the image and refuse).
• Metrics: ASR is the proportion of jailbreak attempts where harmful content appears, which can be in the chain of thought or final answer. The oracle is GPT-4. Safety score is defined as 100−ASR.
• Sampling: Each adversarial query is independently sampled 3 times; if any of the three is flagged as a jailbreak, the query is considered successful.
• Defense parameters: In the main experiments, R_safe is Llama-Guard-3, τ=0, k=3, and the steering phrase is fixed as “Wait, think safely”. Appendix D replaces R_safe with Qwen-Guard-3. Latency uses the average generation time over 100 random JailbreakV items.

Main Results

The table below is the Average column from Table 2 and Table 3, in ASR (%). Undefended is the Original column in the table.

Swipe horizontally to read the table

Model	JailbreakV Undefended	JailbreakV SafeThink	Hades Undefended	Hades SafeThink
R1-Onevision	50.62%	10.36%	69.07%	5.65%
OpenVLThinker	45.69%	1.12%	66.67%	2.18%
VLAA-Thinker	20.38%	4.39%	28.67%	2.22%
Vision-R1	41.84%	3.56%	61.96%	8.35%
LlamaV-o1	63.33%	5.74%	66.80%	6.00%
LLaVA-CoT	42.21%	6.57%	26.85%	2.04%

• The authors claim SafeThink achieves the lowest ASR across the six models, with early-step steering outperforming truncated reasoning or adding safety prompts only at the input side. The main text states that on JailbreakV, LlamaV-o1's undefended ASR is as high as 64.3%, while Table 2 Average is 63.33%. The authors claim reductions relative to undefended of 57.6% (LlamaV-o1) and 44.6% (OpenVLThinker), and further reductions of 39.2% and 18.3% relative to the strongest baseline.
• For Hades, the main text states R1-Onevision goes from 69.1% to 5.7%, with the authors claiming an absolute reduction of 63.4%; Table 3 shows 69.07% to 5.65%. The authors claim ZS-SafePath and AdaShield reduce at most 25.3% on that model. Figure 7 states OpenVLThinker goes from 66.7% to 2.2%, with the authors claiming the highest absolute reduction of 64.5%; the table shows 66.67% to 2.18%.
• The abstract claims ASR reductions of 30–60% across four benchmarks, citing LlamaV-o1's 63.33% to 5.74% and R1-Onevision's 69.07% to 5.65% as examples. This is the authors' summary, not a cell-by-cell subtraction from the table above.

FigStep and MM-SafetyBench

• FigStep (Table 4 Average): Undefended to SafeThink is R1-Onevision 43.60% to 12.00%, OpenVLThinker 49.20% to 4.40%, VLAA-Thinker 26.20% to 7.80%, Vision-R1 37.20% to 6.40%, LlamaV-o1 60.20% to 17.20%, LLaVA-CoT 53.60% to 6.20%. The authors claim the highest absolute reduction is 44.8% (OpenVLThinker, with the main text stating 49.2% to 4.4%), and claim R1-Onevision drops 31.6% and Vision-R1 drops 30.8%.
• Not lowest in every category: The authors claim the lowest ASR across the six models, which holds for the Average column. On R1-Onevision's Adult Content, SafeThink is 10.00% and AdaShield is 6.00%; on Health Consultation, SafeThink is 8.00% and LessThink is 4.00%.
• MM-SafetyBench (Figure 11): The authors claim the lowest across the six models. The numbers given in the main text are Vision-R1 from 59.64% to 2.84% (with the authors claiming the highest absolute reduction of 56.8%), and LLaVA-CoT from 58.14% to 7.91%; they also claim VLAA-Thinker drops 39.8% and LLaVA-CoT drops 50.2%. The extracted text does not include a complete table for this category.

Reasoning Utility and Latency

• MathVista: The abstract states accuracy goes from 65.20% to 65.00%, without specifying the model. Figure 14 at τ=0 states R1-Onevision is 65.0% and LlamaV-o1 is 64.7%. The authors claim LessThink reduces VLAA-Thinker by 15% and Vision-R1 by 17.6%. The bar heights in Figure 9 are not present in the text.
• Table 1 summary row: Reasoning Acc. is Original 63.51%, ZeroThink 54.41%, LessThink 52.98%, ZS-SafePath 60.86%, SafeThink 63.46%. Average ASR in the same order is 44.01%, 31.09%, 31.78%, 31.75%, 5.30%; the benchmark is not specified in that row. The main text also states ZS-SafePath's mean ASR on JailbreakV-28K is 33.40%, which is inconsistent with the table's 31.75%; the table takes precedence.
• Latency (seconds, Table 1): SafeThink is R1-Onevision 8.02, VLAA-Thinker 6.84, Vision-R1 6.86, LlamaV-o1 8.32, LLaVA-CoT 9.32; OpenVLThinker is not in the same table. The authors claim 0.1–0.9 seconds more per item relative to ZS-SafePath; in the table, LLaVA-CoT's ZS-SafePath is 8.10. The main text states an absolute ASR reduction of 38.71% relative to the original model, which does not match the table's 44.01% to 5.30%, so that sentence is not used. The 26.45% relative to ZS-SafePath matches 31.75% to 5.30%, and is still reported per the authors' claim.

Few-Step Steering and Reward Model Replacement

• Steering depth (Figure 4): Steering is only allowed in the first m steps. The authors claim ASR drops in the first few steps and then plateaus, with m≤3, often m≤2, entering the safe zone from the jailbreak zone. The text does not provide ASR readings for each m. The subplot titles are (a) R1-Onevision, (b) VLAA-Thinker, (c) Vision-R1; the caption states LlamaV-o1 / JailbreakV-28K, R1-OneVision-7B / Hades, OpenVLThinker-7B / FigStep. The two do not match.
• Qwen-Guard-3 (Appendix D): In Figure 12 (JailbreakV-28K), the authors claim LlamaV-o1 goes from 63.33% to 4.82%, OpenVLThinker from 45.69% to 0.88%, with the highest absolute reduction of 58.51%. In Figure 13 (Hades), the authors claim all six models have ASR below 7%, with R1-Onevision from 70.67% to 5.1% and LlamaV-o1 from 68.44% to 5.9%; baselines remain above 20% in most cases. These undefended numbers are not exactly the same as Table 3.
• Best-of-N (Figure 2): R1-Onevision, VLAA-Thinker, Vision-R1 on HADES. The caption states that when k increases to 20, BoN* remains below τ; after adding steering tokens, k=2 is enough to cross the threshold. The authors conclude that failing to sample safe trajectories is due to poor conditional coverage, not the absence of safe continuations.

Other Ablations and Analysis

• τ (Figure 14, JailbreakV-28K; R1-Onevision, LlamaV-o1, VLAA-Thinker): τ takes values -0.3, -0.15, 0, 0.15, 0.3. The authors claim safety score increases with τ, with the largest increase from -0.3 to 0, then plateauing after 0; MathVista is stable for τ≤0, and at τ=0.3, R1-Onevision goes from 65.0% to 61.4% and LlamaV-o1 from 64.7% to 60.5%.
• k: In Figure 2, SafeThink crosses the threshold at k=2; the main experiments use k=3.
• Steering phrase (Figure 5, JailbreakV-28K): The authors claim phrases without explicit safety wording remain below τ=0; “Wait, think safely” has the highest safety reward and low KL. The text does not provide specific KL or reward values.
• Qualitative: Figures 10, 15, and 16 compare responses across defenses on HADES, FigStep, and MM-SafetyBench. The authors claim SafeThink refuses malicious queries. The attack sentences and model outputs are not paraphrased.

Limitations and future directions noted by the authors

• Dependence on the monitoring component (Appendix B): The authors write that effectiveness depends on the quality of the safety monitoring component, especially whether the safety reward model is good enough.
• Ablations performed (Appendix B): The authors state that switching to Llama-Guard-3 and Qwen-Guard-3 yielded performance gains in both cases, and they therefore consider the method robust to reward model choice. Appendix B lists no other limitations, and the Conclusion does not mention future experiments.

Experimental coverage

• The evaluated reasoning models are the six open-source MLRMs from Section 5.1; Figure 1 also compares Qwen2.5-VL and R1-Onevision on Hades in terms of undefended ASR.
• Jailbreak evaluations are JailbreakV-28K, HADES, FigStep, and MM-SafetyBench; reasoning utility is MathVista; baseline defenses are ZeroThink, LessThink, ZS-SafePath, and AdaShield.
• In the main experiments, ASR is judged by GPT-4, with each query sampled 3 times and a jailbreak counted if any sample succeeds; R_safe is Llama-Guard-3, τ=0, k=3. Appendix D additionally reports ASR with Qwen-Guard-3.
• The steering-depth experiments cover three model–benchmark combinations (the Figure 4 caption and in-figure title are inconsistent; see question 4); candidate steering tokens are selected offline on 500 held-out samples across the four benchmarks; latency uses 100 samples from JailbreakV (Table 1).
• The paper does not report the number of evaluation samples per benchmark in the main experiments, nor the agreement between GPT-4 and human judgments.

SafeThink is an inference-time defense for multimodal models post-trained for reasoning: safety only needs to exceed a threshold, not be maximized, and parameters are not modified.

• Problem: The authors claim that explicit chain-of-thought post-training methods like GRPO weaken safety. Figure 1 on Hades shows R1-Onevision's ASR relative to Qwen2.5-VL rising from 19.13% to 69.07%.
• Approach: Llama-Guard-3 scores each reasoning step, and only when the score is below τ=0 is a short offline-selected prefix "Wait, think safely" injected. In the main experiment, k=3 candidate steps are sampled each time. The authors claim the intervention typically only needs the first 1–3 steps, after which the trajectory stays in the safe region.
• Jailbreak numbers: In Table 2, LlamaV-o1's ASR on JailbreakV-28K drops from 63.33% to 5.74%; in Table 3, R1-Onevision on Hades drops from 69.07% to 5.65%, and OpenVLThinker from 66.67% to 2.18%. The abstract describes the reductions across four benchmarks as 30–60%. ASR is judged by GPT-4, with a jailbreak counted if any of 3 attempts per query succeeds.
• Utility and authors' conclusions: The abstract reports MathVista going from 65.20% to 65.00%, without specifying the model; Table 1 shows average Reasoning Acc. of 63.51% and 63.46%. In the conclusion, the authors state that early-step steering improves safety robustness while preserving the reasoning capabilities from post-training; poor condition coverage is the earlier explanation for why steering is used instead of more sampling.