Study evaluates harmful content generation and review gaps in five open-source text-to-image models(original, opens in a new tab)
评测文生图模型的有害内容生成能力与审核检测缺口。
Translation in progress Some entries still include text in the original language.
Page 9 Newer contentBack to latest
评测文生图模型的有害内容生成能力与审核检测缺口。
Proposes Perturbot perturbation training to break modality shortcuts in VLA.
After California's AI transparency law took effect, the Brennan Center retested six chatbots' ability to identify AI-generated images, and the results showed no improvement. The tests covered ChatGPT, Claude, Gemini, Grok, Meta AI, and Perplexity, using images generated by Gemini and ChatGPT for verification. Only Gemini read the source metadata embedded in the images and identified the creation tool; the other chatbots did not evaluate metadata and relied only on visual judgment. For an image generated by Gemini of postal workers sorting election mail, four chatbots misjudged it as a real photo, with ChatGPT and Perplexity noticing the visible watermark in the lower right corner but interpreting it as a social media or platform mark. The researchers noted that the law requires companies to provide free tools for users to verify whether images are AI-generated, but does not require that tool to be embedded within chatbots, and recommended integrating verification capabilities directly into chatbots.
《卫报》测试了 ChatGPT、Grok 和 Gemini 的图像编辑功能,发现三款聊天机器人都会按要求移除穆斯林女性照片中的头巾。此前一个月,一名法国极右翼政客曾用 AI 抹去一名真实穆斯林女性的头巾。该问题已持续数月,加剧了人们对 AI 工具被轻易用于脱去女性和儿童衣物的担忧。
提出任务感知的跨域蒸馏与剪枝方法以压缩音频深度伪造检测器。
The researchers propose TAPDreamer, an adversarial patch attack against world action models that constructs fixed local perturbations using only a public encoder and transfers across tasks and action architectures without querying the target policy.
提出 TRACER,从权重差识别扩散模型被擦除概念。
Proposes an audit-aware unlearning framework AVCE, erasing diffusion model concepts from both text and visual pathways.
Proposes Geodesic Unlearning, which edits weights along geodesics to improve robustness to missing modalities.
提出 FADE,对文生视频模型做帧感知多概念擦除。
提出 OmniConfess,冻结候选并按通道重打分以缓解全模态幻觉。
MBZUAI 团队提出校准重合成方法,以生成器反演保真度认证真实图像并防御对抗扰动。
Why it was selected论文给出可复现的校准流程与开源代码,并量化了生成器迭代对事后可验证内容的侵蚀速度。
主张多模态安全评测应衡量可控性,并用 SAE 建立四轴画像。
提出融合定位拒答与能力保留的安全微调,补齐视觉定位输出的拒答缺口。
NVIDIA 研究团队评测 ReAct 工具调用对多模态模型拒答有害请求的削弱。
该 arXiv 论文讨论具身智能体的越狱护栏基准。
The Arizona Court of Appeals vacated Gabriel Paul Horcasitas's 10-and-a-half-year manslaughter sentence for the 2021 shooting of Christopher Pelkey, ruling that an AI-generated victim statement video played at the sentencing hearing was unreliable. The three-judge panel found that the video did not record real events but instead presented the victim's sister's imagination as the victim's own thoughts and statements. Pelkey's family used voice recordings, videos, and photos to reconstruct his likeness, allowing an AI version of Pelkey to speak at the May 2025 sentencing hearing, after nine relatives and friends had already testified. Horcasitas's defense attorney argued that Judge Todd Lang's admission of the AI evidence violated due process and that the defendant had no meaningful opportunity to rebut the video's content; the prosecution said the judge did not sentence based on it. The victim's sister, Stacey Wales, expressed disappointment that the video was cited as a reason for the reversal, saying its only purpose was to show the judge her brother's humanity. The court upheld the conviction and remanded the case for resentencing.
vLLM's scale-out multimodal transport treats caller-supplied features as trusted rendering results without binding validation against the current model's rendering contract. Five trust boundary flaws can let authenticated requests cause shared engine crashes, cross-request encoder cache poisoning or disclosure, and loss of transport integrity. Report submitted by Patch the Planet (a Trail of Bits and OpenAI collaboration), discovered using GPT-5.5-Cyber. The GitHub advisory marks affected versions as < 0.30.0 and the fixed version as 0.30.0.
Why it was selectedThe announcement itemizes the trigger points and impact of five forged fields, along with a fix patch; teams deploying vLLM multimodal inference can use this to investigate similar trust boundaries.
vLLM's GLMGA video sampler constructs an index list based on request-controlled sampling parameters, then deduplicates and reads the video. Small videos combined with overly large parameters can also consume disproportionate CPU time and memory, delaying other media requests and causing partial denial of service. The advisory does not report memory corruption, data leakage, or code execution; video options can be filtered at the gateway, and untrusted clients can be restricted from selecting this sampler. The GitHub advisory marks affected versions as >= 0.23.0rc2, < 0.30.0, with the fixed version being 0.30.0.
Why it was selectedThe announcement gives the cause and fix recommendations for the denial-of-service in the vLLM video sampling interface, and teams deploying multimodal inference services can use this to check request-level parameter boundaries.
vLLM's Qwen2-VL and Qwen3-VL video samplers do not enforce the existing num_frames cap on request-level max_frames, which can lead to memory exhaustion. In the advisory's CPU-only test, decoded frames increased from 60 to 900 and peak RSS increased from 2,271 MiB to 13,629 MiB; these values correspond only to that test. Paths such as /tokenize do not require vLLM API-key authentication; the Rust frontend is unaffected because it rejects the relevant media options. The GitHub advisory marks affected versions as >= 0.24.0, < 0.30.0 and the fixed version as 0.30.0.
Why it was selectedThe report provides measured data on unauthenticated requests exhausting vLLM memory and the affected version range; teams deploying Qwen-VL services can use this to check versions and request parameters.
Girard Sharp and other law firms announced a class action lawsuit on behalf of Jane Doe against xAI in the U.S. District Court for the Northern District of California, alleging that Grok's training and image generation involved child sexual abuse material and harmed victims' rights. The plaintiffs also questioned xAI's safety investments and post-incident access restrictions, seeking damages and injunctions to stop and destroy the relevant material. These are all plaintiff allegations and requests for relief, not court findings; this item is based on the law firm announcement and does not present unadjudicated liability claims as judicial conclusions.
Why it was selected诉讼指控 xAI 用含 CSAM 的数据训练 Grok 并生成新图像,呈现了训练数据来源与图像生成护栏缺失的完整指控链条。
分析文体改写是否改变多模态声明核验的判决与置信度。
提出 LiBRA,在潜空间双向优化以规避图像水印双侧检测。
提出条件特征抑制防御,检出对抗补丁后抑制 VLA 内部特征。
Scientific Reports 发表研究,考察牙科影像场景下的视觉提示注入攻击及其防御。研究涉及视觉提示注入这一攻击形式,并探讨相应防御手段。
提出 imMRAG 攻击,从多模态 RAG 私有图像库提取内容。
用 MMPIBench 评测多模态提示注入对 Agent 框架的影响。
Why it was selected论文用统一的跨框架测试台定位注入攻击在感知、规划、工具调用各阶段被拦下的位置,并给出模型与框架各自的作用。
审计视觉语言模型在敏感图像上从拒答转向国家立场重构的行为。
Scale AI proposes SteerDuplex, a full-duplex spoken dialogue model fine-tuned from Moshi that can adjust tone, persona, speaking rate, and voice style according to user instructions. The accompanying SteerBench benchmark contains 390 voice prompts and 1,067 human-written audio and text scoring rubrics; supervised training improves the average audio control pass rate by 44.5 percentage points over the strongest open-source baseline, and the Audio MultiChallenge task average pass rate improves by 7 percentage points. Two-stage reinforcement learning raises the clean-source interruption response rate from 72.5% to 82.5% and reduces synthetic pause false triggers from 26.5% to 9%, but reward probing finds the model engages in reward hacking through incomplete answers.
提出 SPAR 免训练注意力干预,削弱多模态模型的结构偏置。
因果审计视觉工具调用,分离观察贡献与动作诱发捷径。
比较多模态大模型上三类免训练不确定性估计。
提出 Mind2Web-Injection 评测网页 Agent 视觉注入护栏。
提出 RECAST,本地改写图表与语音并做可逆数值映射以保护远程推理隐私。
提出MAMJ,在元层面交替优化多模态越狱的策略提示与攻击者权重。
提出 HACA,将图文越狱拆为原子策略并自动组合以诱导有害输出。
提出 SPARED,用对抗编辑数据训练基于推理的 AI 生成图像检测器。
构建Mandela-Bench,评测多模态模型是否凭记忆接受篡改图像。
提出 DURA,用扩散模型生成自然对抗补丁操控 VLA 机器人动作。
提出抗攻击多语言视觉指令遵循基准 MM-IFEval-Pro。