PrivacyBench: A Conversational Benchmark for Evaluating the Privacy of Personalized AI

The researchers propose PrivacyBench, which uses a social-context dataset containing embedded secrets and multi-turn dialogue evaluation to measure the secret-protection capability of personalized AI. Tests on RAG (retrieval-augmented generation) assistants show that they leak user secrets in up to 26.56% of interactions; adding privacy-aware prompts reduces the leak rate to 5.12%, but this is only a partial mitigation. The authors point out that the retrieval mechanism still accesses sensitive data indiscriminately, placing the entire protection burden on the generator and creating a single point of failure; they therefore conclude that current architectures are unsuitable for large-scale deployment and require privacy-intrinsic structural defenses.