TrustLDM: A Trustworthiness Evaluation Benchmark for Language Diffusion Models

The researchers propose the TrustLDM benchmark, which systematically evaluates the trustworthiness of language diffusion models (LDMs) in terms of safety, privacy and fairness, covering different LDM architectures and multiple types of static post-hoc context. Experiments show that LDMs generally have good trustworthiness when they receive only the user prompt, but after the masked response is appended with malicious post-hoc context, alignment behavior degrades markedly; longer context does not necessarily have a stronger effect, and decoding order and generation length also affect evaluation results. The authors also propose the TrustLDM-Auto automatic evaluation framework, which exploits the decoding flexibility of LDMs to systematically locate vulnerable configurations, and find that all tested models have notable trustworthiness weaknesses across all dimensions. Code is open-sourced at https://github.com/PKU-ML/TrustLDM。