Translation in progress Some entries still include text in the original language.

Research library

Conference research

AI safety research, organized by conference, year and research area.

Total 45 papers

Jailbreaks and attacks

EvilHarmony: Stealthy Adversarial Attacks against Black-box Speech Recognition Systems
Non-AI trackXuejing Yuan, Jiangshan Zhang, Feng Guo et al.Conference page
Fuzz-Testing Meets LLM-Based Agents: An Automated and Efficient Framework for Jailbreaking Text-To-Image Generation Models
Yingkai Dong, Xiangtao Meng, Ning Yu et al.Conference page
Modifier Unlocked: Jailbreaking Text-to-Image Models Through Prompts
Shuofeng Liu, Mengyao Ma, Minhui Xue et al.Conference page
UnMarker: A Universal Attack on Defensive Image Watermarking
Andre Kassis, Urs HengartnerConference page

Prompt injection

DataSentinel: A Game-Theoretic Detection of Prompt Injection Attacks
Yupei Liu, Yuqi Jia, Jinyuan Jia et al.Conference page
Fun-tuning: Characterizing the Vulnerability of Proprietary LLMs to Optimization-based Prompt Injection Attacks via the Fine-Tuning Interface
Andrey Labunets, Nishit V. Pandya, Ashish Hooda et al.Conference page

Defenses and guardrails

Adversarial Robust ViT-based Automatic Modulation Recognition in Practical Deep Learning-based Wireless Systems
Non-AI trackGen Li, Chun-Chih Lin, Xiaonan Zhang et al.Conference page
EveGuard: Defeating Vibration-based Side-Channel Eavesdropping with Audio Adversarial Perturbations
Non-AI trackJung-Woo Chang, Ke Sun, David Xia et al.Conference page
Exploring Parent-Child Perspectives on Safety in Generative AI: Concerns, Mitigation Strategies, and Design Implications
Non-AI trackYaman Yu, Tanusree Sharma, Melinda Hu et al.Conference page
Fight Fire with Fire: Combating Adversarial Patch Attacks using Pattern-randomized Defensive Patches
Jianan Feng, Jiachun Li, Changqing Miao et al.Conference page
GuardAIn: Protecting Emerging Generative AI Workloads on Heterogeneous NPU
Non-AI trackAritra Dhar, Clément Thorens, Lara Magdalena Lazier et al.Conference page
On the Conflict between Robustness and Learning in Collaborative Machine Learning
Mathilde Raynal, Carmela TroncosoConference page
Query Provenance Analysis: Efficient and Robust Defense against Query-based Black-box Attacks
Shaofei Li, Ziqi Zhang, Haomin Jia et al.Conference page
SoK: Watermarking for AI-Generated Content
Xuandong Zhao, Sam Gunn, Miranda Christ et al.Conference page
Supporting Human Raters with the Detection of Harmful Content using Large Language Models
Non-AI trackKurt Thomas, Patrick Kelley, David Tao et al.Conference page
TSQP: Safeguarding Real-Time Inference for Quantization Neural Networks on Edge Devices
Yu Sun, Gaojian Xiong, Jianhua Liu et al.Conference page
Verifiable Boosted Tree Ensembles
Stefano Calzavara, Lorenzo Cazzaro, Claudio Lucchese et al.Conference page
Watermarking Language Models for Many Adaptive Users
Aloni Cohen, Alexander Hoover, Gabe SchoenbachConference page

Alignment

Alleviating the Fear of Losing Alignment in LLM Fine-tuning
Kang Yang, Guanhong Tao, Xun Chen et al.Conference page

Reward hacking

Preference Poisoning Attacks on Reward Model Learning
Junlin Wu, Jiongxiao Wang, Chaowei Xiao et al.Conference page

Backdoors and poisoning

Architectural Neural Backdoors from First Principles
Harry Langford, Ilia Shumailov, Yiren Zhao et al.Conference page
BAIT: Large Language Model Backdoor Scanning by Inverting Attack Target
Guangyu Shen, Siyuan Cheng, Zhuo Zhang et al.Conference page
Not All Edges are Equally Robust: Evaluating the Robustness of Ranking-Based Federated Learning
Zirui Gong, Yanjun Zhang, Leo Yu Zhang et al.Conference page
PEFTGuard: Detecting Backdoor Attacks Against Parameter-Efficient Fine-Tuning
Zhen Sun, Tianshuo Cong, Yule Liu et al.Conference page
Practical Poisoning Attacks with Limited Byzantine Clients in Clustered Federated Learning
Viet Vo, Mengyao Ma, Guangdong Bai et al.Conference page
Secure Transfer Learning: Training Clean Model Against Backdoor in Pre-Trained Encoder and Downstream Dataset
Yechao Zhang, Yuxuan Zhou, Tianyu Li et al.Conference page

Privacy and data breaches

CipherSteal: Stealing Input Data from TEE-Shielded Neural Networks with Ciphertext Side Channels
Non-AI trackYuanyuan Yuan, Zhibo Liu, Sen Deng et al.Conference page
Codebreaker: Dynamic Extraction Attacks on Code Language Models
Non-AI trackChangzhou Han, Zehang Deng, Wanlun Ma et al.Conference page
Comet: Accelerating Private Inference for Large Language Model by Predicting Activation Sparsity
Non-AI trackGuang Yan, Yuhui Zhang, Zimu Guo et al.Conference page
Edge Unlearning is Not "on Edge"! An Adaptive Exact Unlearning System on Resource-Constrained Devices
Xiaoyu Xia, Ziqi Wang, Ruoxi Sun et al.Conference page
GRID: Protecting Training Graph from Link Stealing Attacks on GNN Models
Jiadong Lou, Xu Yuan, Rui Zhang et al.Conference page
HARMONYCLOAK: Making Music Unlearnable for Generative AI
Syed Irfan Ali Meerza, Lichao Sun, Jian LiuConference page
Machine Learning with Privacy for Protected Attributes
Saeed Mahloujifar, Chuan Guo, Edward Suh et al.Conference page
On the Effectiveness of Prompt Stealing Attacks on In-The-Wild Prompts
Yicong Tan, Xinyue Shen, Yun Shen et al.Conference page
Prevalence Overshadows Concerns? Understanding Chinese Users' Privacy Awareness and Expectations Towards LLM-based Healthcare Consultation
Non-AI trackZhihuang Liu, Ling Hu, Tongqing Zhou et al.Conference page
Prompt Inversion Attack against Collaborative Inference of Large Language Models
Wenjie Qu, Yuguang Zhou, Yongji Wu et al.Conference page
Rigging the Foundation: Manipulating Pre-training for Advanced Membership Inference Attacks
Zihao Wang, Rui Zhu, Zhikun Zhang et al.Conference page
SoK: Dataset Copyright Auditing in Machine Learning Systems
Linkang Du, Xuanru Zhou, Min Chen et al.Conference page
Towards Reliable Verification of Unauthorized Data Usage in Personalized Text-to-Image Diffusion Models
Boheng Li, Yanhao Wei, Yankai Fu et al.Conference page
Understanding Users' Security and Privacy Concerns and Attitudes Towards Conversational AI Platforms
Mutahar Ali, Arjun Arunasalam, Habiba FarrukhConference page

Supply chain security

My Model is Malware to You: Transforming AI Models into Malware by Abusing TensorFlow APIs
Non-AI trackRuofan Zhu, Ganhao Chen, Wenbo Shen et al.Conference page
On the (In)Security of LLM App Stores
Xinyi Hou, Yanjie Zhao, Haoyu WangConference page

Dangerous Capabilities

GPTracker: A Large-Scale Measurement of Misused GPTs
Xinyue Shen, Yun Shen, Michael Backes et al.Conference page

AI and cyber offense and defense

Make a Feint to the East While Attacking in the West: Blinding LLM-Based Code Auditors with Flashboom Attacks
Non-AI trackXiao Li, Yue Li, Hao Wu et al.Conference page
SV-TrustEval-C: Evaluating Structure and Semantic Reasoning in Large Language Models for Source Code Vulnerability Analysis
Non-AI trackYansong Li, Paula Branco, Alexander M. Hoole et al.Conference page