Skip to content

Reading Scene States: Revealing Safety Failures of Vision-Language Models

Reading Scene States: Uncovering Safety Failures in Vision-Language Models

arXiv: 其他 AI 安全论文· arXiv:2611.23206· Tongxi Wu, Yuyang Cheng, Jiacheng Xian, Menglin Yang, Kai Chen, Jian Zhang, Yang GaoNanjing University; The Hong Kong University of Science and Technology (Guangzhou)Indexed by this site · 2 days ago· Originally publishedAttacks· Its place in the scan record →

Quick read

Attacks

Based on the paper PDF (AI-generated); refer to the original

The authors propose the scene-state security vulnerability and the S3TA attack framework, achieving an average ASR of 71.2% across eight models on the seven categories of MM-SafetyBench.

Threat modelThe attacker transforms a harmful request x into a scene-state image and question, and obtains harmful assistance aligned with x from the target VLM M through independent image-text queries; there is no shared conversation history, and no access to model parameters or gradients; the check model C provides image modification feedback before evaluation.

Questions
When a VLM interprets the scene state of an activity in progress in an image, even if the question does not explicitly state a harmful goal, it may infer and provide harmful assistance; the authors say this scene-state security vulnerability has been understudied.
Methods
Construct SceneSafetyBench (375 image-text pairs, three scene states × five visual styles) to evaluate the vulnerability, use HiddenDetect readout for mechanism comparison, and propose S3TA: plan the scene, generate the image, and concatenate answers after independent queries.
Experiments and results
Eight VLMs on SceneSafetyBench have ASR 45.6–72.3%; S3TA on the seven categories of MM-SafetyBench has a model-average ASR of 71.2%, which the authors state is 6.1 percentage points higher than the strongest baseline; text filtering and visual moderation defenses are limited.
Limitations and future work
The paper does not specifically discuss limitations. Experiments cover: eight target VLMs and two benchmarks; mechanism analysis is limited to three models and 75 tasks; manual verification covers only 50 InternVL3.5-8B responses; defense evaluation is on Qwen3-VL-8B and does not test adaptive attacks.
Experimental setupQwen3-VL-8B、GLM-4.1V-9B et al. · SceneSafetyBench、MM-SafetyBench · ASR、H et al.
Evaluated models
Qwen3-VL-8BGLM-4.1V-9BInternVL3.5-8BMiniCPM-V 4.5Gemma 3 12B ITGPT-5.6 LunaGemini 3.5 FlashClaude Haiku 4.5LLaVA-v1.5-13B
Benchmarks
SceneSafetyBenchMM-SafetyBench
Metrics
ASRHADHD
AI summaryResearchers identify a scene-state safety vulnerability in vision-language models (VLMs): without an explicit request for a potentially harmful goal, a model can provide harmful assistance by interpreting visual events. The paper's SceneSafetyBench contains 375 image-question pairs covering initial, intermediate and final scene states and five visual presentations; across eight VLMs, scene-state inputs achieve attack success rates (ASR) of 45.6%–72.3% in eliciting harmful assistance. The proposed S3TA attack accordingly converts a harmful goal into a scene state, queries each image-question pair independently and assembles the answers, achieving a model-average ASR of 71.2% across seven MM-SafetyBench categories on eight open-weight and commercial VLMs, exceeding the strongest baseline by 6.1 percentage points. The paper also shows that text-side filtering and visual moderation offer limited protection, arguing that safety defenses need to cover assistance the model gives through scene reasoning, not just screening input content.

Researchers identify a scene-state safety vulnerability in vision-language models (VLMs): without an explicit request for a potentially harmful goal, a model can provide harmful assistance by interpreting visual events. The paper's SceneSafetyBench contains 375 image-question pairs covering initial, intermediate and final scene states and five visual presentations; across eight VLMs, scene-state inputs achieve attack success rates (ASR) of 45.6%–72.3% in eliciting harmful assistance. The proposed S3TA attack accordingly converts a harmful goal into a scene state, queries each image-question pair independently and assembles the answers, achieving a model-average ASR of 71.2% across seven MM-SafetyBench categories on eight open-weight and commercial VLMs, exceeding the strongest baseline by 6.1 percentage points. The paper also shows that text-side filtering and visual moderation offer limited protection, arguing that safety defenses need to cover assistance the model gives through scene reasoning, not just screening input content.

Paper analysis

6 questions; open one to read the full answer

  1. What problem does this paper attempt to solve?

    When VLMs interpret the scene state in an image, they infer and provide harmful assistance; the authors state that this vulnerability has not been systematically studied before.

    Core question: a VLM infers a harmful goal by interpreting the entities, relations, and activity progression in an image (what the authors call scene state) and provides assistance, even if the question itself does not explicitly state that goal.

    • Problem setting: MLLMs increasingly support tasks that require understanding visual context and giving actionable assistance; when answering questions about a depicted situation, the model infers what is happening, the goal of the activity, and possible actions, which the authors see as a safety risk.
    • Gaps in existing research: prior work has studied visual encoding, cross-modal information distribution, contextual attacks, and so on, but the authors say scene state as a source of safety failure remains understudied; existing work relies on dialogue history, image sequences, or specified initial–final state pairs, whereas this paper focuses on what kind of assistance is implied by the scene depicted in a single image.
    • Core observation: Figure 1 shows GPT-5.6 Luna explicitly refusing a purely textual harmful request and a QR-related image, but giving action-oriented guidance under a scene-state input.
    • Paper outputs: the SceneSafetyBench benchmark, a mechanistic analysis based on HiddenDetect, and the S3TA attack framework.
    • Threat model (Section 4.1):
      • Goal: obtain harmful assistance aligned with the original harmful request x through independent image-text queries.
      • Knowledge: no access to the parameters or gradients of the target model M.
      • Capabilities: construct scene images and questions, query independently, with no shared dialogue history; a checking VLM C provides image modification feedback before evaluation.
      • Victim systems: eight open-source and commercial VLMs.
  2. What related research exists?

    Related work is grouped into four categories: visual jailbreak, context and sequence attacks, state-type safety, and multimodal safety evaluation; the authors state that this paper focuses on single-image scene states.

    Visual jailbreak attacks

    • FigStep [12] and FC-Attack [41]: express harmful requests using typographic images and flowcharts, respectively; other methods convey harmful intent indirectly through visual encoding, substitution, or analogy [2].
    • Optimization-based attacks: modify image pixels or jointly manipulate visual and textual inputs [5, 30, 32]; ASO optimizes visual style as a jailbreak trigger [22].
    • Semantic methods: introduce QR-related images or distribute harmful information across multiple modalities [18, 21, 24, 34].

    Contextual and sequential visual attacks

    • CIA [37]: embeds harmful queries in a semantically coherent visual context; VisCo [25]: constructs fictional image dialogue contexts.
    • VRSA [42]: distributes harmful intent across a sequence of related images for the target model to recover through joint reasoning, supporting multi-turn or concatenated single-turn inputs.
    • The authors say that in this paper's setting the question asks the model to interpret or continue the depicted activity, with the harmful goal kept implicit, which distinguishes it from the above work.

    State-based safety and reasoning

    • STAR [17]: studies how dialogue history changes a model's safety state.
    • State2State [29]: expresses harmful intent through descriptions of an initial state and a desired final state, having the model explain the causal transition to induce a harmful procedure.
    • The authors say this paper studies the scene state depicted in a single image, without needing dialogue history or specified initial–final state pairs.

    Multimodal safety evaluation

    • MM-SafetyBench, HADES, JailBreakV-28K [18, 21, 23]: evaluate multimodal safety under diverse harmful requests and jailbreak inputs; MSSBench [43]: examines how visual context determines whether assistance is safe; OmniSafeBench-MM [15]: a unified evaluation framework considering harmfulness, intent satisfaction, and response detail.
    • Baselines and benchmarks: S3TA's comparison baselines are QR-related [21], FigStep [12], VisCRA [31], MML [34], FC-Attack [41], and CIA [37], and the benchmark is the seven categories of MM-SafetyBench (Table 3).

    Authors' positioning: SceneSafetyBench focuses on harmful assistance induced through interpretation of scene states, organizing image-text pairs by the conditions, progression, and outcome of an activity, which differs in emphasis from the above benchmarks.

  3. How does the paper address this problem?

    First build SceneSafetyBench to evaluate the vulnerability and conduct mechanism analysis, then propose S3TA: plan the scene, generate the image, query independently, and concatenate answers.

    The overall approach has three steps: build SceneSafetyBench to quantify scene-state safety failures, use HiddenDetect readout for mechanism comparison, and finally systematize the vulnerability into the S3TA attack framework.

    SceneSafetyBench Construction and Evaluation

    • Data: 75 base samples, with images from the internet, each paired with a guiding question; divided into Initial (conditions before the activity), Medium (activity in progress), and Ultimate (activity outcome) three categories, each with 25 different tasks and scenes.
    • Visual styles: Each base image keeps Original, and four additional styles—Pixel Art, Pencil Sketch, Oil Painting, Film Noir—are rendered (generated by GPT-Image-2), for a total of 375 image-text pairs.
    • Evaluation protocol: Following OmniSafeBench-MM, GPT-4o-mini scores harmfulness H (1–10), intent alignment A (1–5), and detail D (1–5); H≥3, A≥3, D≥3 counts as a successful attack, and ASR is reported; attempts blocked by policy count as failures.

    Mechanism Analysis Method

    • Subjects: Qwen3-VL-8B, LLaVA-v1.5-13B, InternVL3.5-8B, 75 base tasks, seven input conditions (Original plain text, QR-related, FigStep, MML, CIA, PGD, Scene-State).
    • Readout: Modify HiddenDetect to project intermediate-layer residual states into vocabulary space and compute the cosine with safety vocabulary indicator vectors, HD(l,t)i,m = cos(z(l,t)i,m, v), measuring alignment with safety-related vocabulary rather than refusal probability.
    • Three research questions: RQ1 compares layer-wise HD profiles (quantifying distance with RMSE to Scene-State); RQ2 tracks HD in the last third of layers during generation, and uses two fixed prefixes to teacher-force eight tokens to align the output context; RQ3 performs attention-head interventions, selecting components on 20 discovery tasks and evaluating on 55 held-out tasks, with the intervention effect defined as E(c)i,m = HDtarget(c)i,m − HDcontrol(c)i,m.

    S3TA Attack Framework

    • Planning: deepseek-flash identifies the required answer form and plans the scene description; single-scene is the main case, with a cap of K=5 for multi-stage requests and K=3 for multiple options.
    • Scene construction: Each scene depicts an activity in progress, using visible entities, relations, and unfinished actions or outputs to indicate progress and pending content; the accompanying question asks the model to interpret or continue the activity, without restating x.
    • Generation and verification: Qwen-Image renders the image; Qwen3-VL-8B serves as check model C, using an image-only question to check whether the activity or attribution relation is recognizable, and on failure feeds back modifications, with a modification budget B=1; if re-checking fails, the final image is still kept.
    • Query and concatenation: Each state is queried independently, yk = M(Ik, qk), and finally y = Concat(s1:K, y1:K); rule-based processing removes image-referring openings and cross-stage references, adds the plan title, and does not call the model again.

    Success Determination

    • Same as SceneSafetyBench: GPT-4o-mini sees only the original request x and the final answer y, and gives a set of H/A/D and a success label, with H≥3, A≥3, D≥3 as success.
  4. What experiments did the paper conduct?

    Eight models on SceneSafetyBench have ASR 45.6–72.3%; S3TA has a model-average ASR of 71.2%, which the authors state exceeds the strongest baseline by 6.1 percentage points.

    Experimental Setup

    • Models tested: Qwen3-VL-8B, GLM-4.1V-9B, InternVL3.5-8B, MiniCPM-V 4.5, Gemma 3 12B IT, GPT-5.6 Luna, Gemini 3.5 Flash, Claude Haiku 4.5, eight in total; mechanism analysis additionally uses LLaVA-v1.5-13B.
    • Benchmarks: SceneSafetyBench (375 image-text pairs, 75 pairs per model per style); S3TA uses the original requests of the seven categories of MM-SafetyBench (Illegal Activity, Hate Speech, Malware Generation, Physical Harm, Economic Harm, Fraud, Privacy Violence).
    • Baselines: QR-related, FigStep, VisCRA, MML, FC-Attack, CIA.
    • Metrics and judging: GPT-4o-mini scores H (1–10), A (1–5), D (1–5); H≥3, A≥3, D≥3 is judged successful, and ASR is reported.
    • Sample sizes: SceneSafetyBench has 375 attempts per model; mechanism analysis has 75 tasks (intervention experiments: 20 discovery + 55 held-out); manual verification has 50 responses.

    Main Results

    Overall ASR by visual style on SceneSafetyBench (%, according to Table 1, 375 pairs per model):

    Swipe horizontally to read the table

    ModelOriginalPixel ArtPencil SketchOil PaintingFilm NoirOverall
    Qwen3-VL-8B45.344.042.744.052.045.6
    GLM-4.1V-9B73.369.373.372.073.372.3
    InternVL3.5-8B57.361.358.754.762.758.9
    MiniCPM-V 4.566.764.064.060.065.364.0
    Gemma 3 12B IT70.764.072.064.068.067.7
    GPT-5.6 Luna64.052.050.750.753.354.1
    Gemini 3.5 Flash68.060.068.062.774.766.7
    Claude Haiku 4.546.741.348.048.044.045.6
    • The authors state that the Overall ASR of the eight models is 45.6–72.3%, with a model average of 59.4%; the model-average ASR across the five styles is 57.0–61.7%, and no single style is consistently best across all models.
    • By scene state (Table 2, 125 attempts per state): the model-average ASR for Initial, Medium, and Ultimate is 64.4%, 57.8%, and 55.9%, respectively, and every model shows failures in all three categories.

    S3TA Attack Comparison

    Table 3 compares S3TA with six baselines on the seven categories of MM-SafetyBench: S3TA has a model-average ASR of 71.2%, which the authors state exceeds CIA (65.1%) and MML (62.2%) by 6.1 and 9.0 percentage points, and achieves the highest average H and D.

    • Per model: S3TA has the highest ASR on Qwen3-VL-8B (87.3%), GLM-4.1V-9B (75.1%), GPT-5.6 Luna (71.0%), and Claude Haiku 4.5 (49.6%); the authors state that on Qwen3-VL and GPT it is 17.0 and 17.6 percentage points higher than each model's strongest baseline, respectively.
    • Exceptions: On Gemini 3.5 Flash, S3TA is 54.8%, lower than MML's 94.7%; on MiniCPM-V 4.5, S3TA is 77.0%, lower than MML's 87.1% and VisCRA's 80.9%; on InternVL3.5-8B, S3TA is 73.8%, lower than MML's 84.0% (all according to Table 3).

    Ablation Experiments (Table 4, Figure 8)

    • Setup: Qwen3-VL-8B, four categories of MM-SafetyBench (Illegal Activity, Malware Generation, Physical Harm, Privacy Violence), five configurations.
    • Results: Original, Decompose, and Text-only all have an Overall ASR of 6.84%; No-gate (removing image verification) 87.97%; Full S3TA 94.81% (A=4.92, D=4.75).
    • Authors' interpretation: Decomposition alone or text-only questions bring no additional success; scene images contribute greatly; verification and modification further improve it, with the Physical Harm category rising from 79.86% to 90.97%.

    Mechanism Analysis Results (Figure 4–6)

    • RQ1: Scene-State's layer-wise HD profile is closest to QR-related on all three models (RMSE 1.67, 1.23, 1.53×10⁻³, respectively), but the final-layer average HD is lower than QR-related by 2.07, 0.92, 2.19×10⁻³; the authors report that the differences are significant for Qwen3-VL and InternVL3.5, and not significant for LLaVA.
    • RQ2: In natural generation, Scene-State's HD after eight tokens is lower than PGD (gaps 3.26, 3.54, 5.25×10⁻³); under a fixed prefix the relative strength changes, and the authors state that the relative signal strength depends on the output context.
    • RQ3: In InternVL3.5, four of the nine Scene-State interventions differ from the control, and six of the Scene–FigStep effect differences are significant after correction; the authors state that the local computation of different attack conditions depends on different components, and significance varies by model.

    Defense Evaluation and Manual Verification

    • Text-side defenses (on Qwen3-VL-8B): Llama Guard 3 and the NeMo perplexity heuristic flag no inputs, and ASR remains 87.25%; PPL-p95 and p99 block 96 and 95 requests, respectively, and ASR is still 81.00%, a decrease of only 6.25 percentage points.
    • Visual-side defenses: ShieldGemma 2-4B has an overall image block rate of 21.81%, 48.34% for the Malware Generation category, and 13.65% for the Economic Harm category; the authors state that most scene images are not blocked.
    • Manual verification (Table 5, 50 InternVL3.5-8B responses, 3 annotators): GPT-4o-mini agrees with the human majority vote 92% of the time (Cohen's κ=0.78); pairwise agreement among annotators is 90–94% (Fleiss' κ=0.75); individual ASR is 76–82%, human majority 80%, automatic judging 72%.

    Other Ablations and Analyses

    • None (the main ablations and parameter analyses are listed above).
  5. What are some points that can be further explored?

    The paper does not specifically discuss limitations; the experiments cover eight target models and two benchmarks, the mechanistic analysis is limited to three models, and the defense evaluation does not include adaptive attacks.

    Limitations and future directions noted by the authors

    • The paper does not specifically discuss limitations (there is no Limitations section, and the Conclusion does not list limitations or future work).
    • The Conclusion makes a directional claim: the authors say multimodal safety needs to address harmful assistance inferred from visual events, extending defenses from input screening to scene-grounded reasoning.

    Experimental coverage

    • The models tested are eight VLMs (five open-source, three commercial); the mechanistic analysis uses only three of them—Qwen3-VL-8B, InternVL3.5-8B, and LLaVA-v1.5-13B (Section 3.2).
    • The evaluation benchmarks are SceneSafetyBench (375 image-text pairs, 75 base tasks) and seven categories of MM-SafetyBench; the ablation uses only four of those categories (Section 4.4).
    • Defense evaluation includes the text side (Llama Guard 3, the NeMo perplexity heuristic, GPT-2 PPL-p95/p99) and the visual side (ShieldGemma 2-4B), all performed on Qwen3-VL-8B; adaptive attacks were not tested (Section 5).
    • The human verification sample is 50 InternVL3.5-8B responses with 3 annotators (Table 5); the paper does not report the number of repetitions for the main SceneSafetyBench and S3TA experiments.
    • The models used in the attack pipeline are deepseek-flash (planning), Qwen-Image (image generation), and Qwen3-VL-8B (checking); the paper does not report the number of queries or the time cost of S3TA (Section 4.2).
  6. Summarize the main content of the paper

    The authors propose the scene-state safety vulnerability, SceneSafetyBench, and the S3TA attack; the eight-model average ASR is 71.2%, and existing input-side defenses are limited.

    A VLM safety attack paper: the authors point out that when VLMs interpret the scene state of an activity in an image (entities, relations, progress), they infer and provide harmful assistance even when the question does not explicitly state a harmful goal.

    • Problem: Figure 1 shows GPT-5.6 Luna refusing a plain-text harmful request and a QR-related image, yet giving action guidance for a scene-state input; the authors state that this vulnerability has been understudied in prior work.
    • Method: Construct SceneSafetyBench (75 base tasks × five visual styles = 375 image-text pairs, divided into Initial/Medium/Ultimate three categories); use HiddenDetect safety vocabulary readout to compare mechanisms across three models; propose S3TA—deepseek-flash plans the scene, Qwen-Image generates the image, Qwen3-VL-8B verifies, and answers are concatenated by rules after independent queries.
    • Vulnerability prevalence: Eight VLMs on SceneSafetyBench have Overall ASR 45.6–72.3%, model average 59.4% (Table 1); the three scene-state categories have model-average ASR 55.9–64.4% (Table 2).
    • Attack effectiveness: S3TA on the seven categories of MM-SafetyBench and eight models has an average ASR of 71.2%, which the authors state exceeds the strongest baseline CIA by 6.1 percentage points; however, on Gemini 3.5 Flash, MiniCPM-V 4.5, and InternVL3.5-8B it is lower than MML (Table 3).
    • Mechanism findings: The authors state that Scene-State's layer-wise safety signal profile is closest to QR-related, but the final-layer readout is lower, the generation dynamics and component intervention effects differ, and the relative signal strength depends on the output context.
    • Defenses and conclusion: Text filtering (Llama Guard 3, perplexity) almost never blocks, and ShieldGemma 2's image block rate is 21.81%; the authors argue that multimodal safety needs to extend from input content screening to defenses for scene-grounded reasoning.

Related papers

Show 3 more papers
Read originalarxiv.org