Skip to content

CuRSE-Bench: A benchmark for probing cultural blind spots in text-to-image safety evaluation

CuRSE-Bench: A Benchmark for Probing Cultural Blind Spots in Text-to-Image Safety Evaluation

arXiv: Dangerous Capabilities & Safety Evals· arXiv:2611.20821· Minsuk Jang, Minseok Son, KiHyun Seol, Yeongeun Shim, Changick KimKorea Advanced Institute of Science and TechnologyIndexed by this site · 2 days ago· Originally publishedEvaluation· Its place in the scan record →

Translation in progress — untranslated text is omitted.

Quick read

Evaluation and benchmarks

Based on the paper PDF (AI-generated); refer to the original

AI summaryThe researchers propose CuRSE-Bench to probe cultural blind spots in text-to-image safety evaluation. Existing evaluations typically assign a single culture-agnostic harmful label to a prompt or image, so content that is harmful only in a specific culture may be passed as harmless. The authors propose an agentic framework, CultureCraft, that constructs prompts that appear harmless under generic scoring criteria but carry implicitly harmful meanings in the target culture, and on this basis build a benchmark containing 10,709 culture-specific prompts and 2,648 culture-generic controls, covering six regions and ten harm axes, with each prompt traceable to at least two independent sources. Six regional persona judges and paired culture-blind judges scored all prompts; the results show that culture-blind judges underestimate culture-specific implicitly harmful content and are comparable to persona judges on the control set; each persona judge mainly identifies implicitly harmful content from its own region, and this selective cultural blind spot also exists in deployed moderation systems.

Questions
T2I safety evaluation gives each prompt or image a culture-blind harm label, so content harmful only within a specific culture is judged benign. The authors argue that the blind spot between culture-conditioned review and culture-blind review itself needs to be measured.
Methods
Proposes CultureCraft: starting from harm vectors documented in dual sources, it rewrites generic harmful seeds into culture-specific everyday-scenario prompts through a drafting–verification loop, constructs CuRSE-Bench, and scores with six regional persona reviewers and one culture-blind reviewer paired on the same scoring task.
Experiments and results
The authors report: culture-blind review underestimates culture-specific harm by 0.82 points (0–10 scale), with a difference of −0.02 on the control set; the gap is concentrated on the POL, HAT and REL axes; the culture-specific set bypasses the built-in filters of five generation pipelines at an average of 96.3%, higher than the compared jailbreak attacks (0.6–55.2%).
Limitations and future work
The authors note: only six regions are covered (Africa, Central Asia, and Oceania are not included); all prompts are in English; each region is verified by only two student researchers; the reviewers are persona-conditioned LLMs rather than cultural truth. Regions are locale aggregates, and no verifier is culturally inside all locales within an aggregate.
Read originalarxiv.org