Planting and Mitigating Memorized Content in Predictive-Text Language Models

The researchers tested a series of privacy protection techniques for mitigating language models' unintended memorization of sensitive user text, with experiments varying both model scale and adversarial conditions. Results show that, except for L2 regularization, heuristic mitigation methods without formal privacy guarantees were largely unable to prevent memorization in their experiments, possibly because such methods make overly strong assumptions about the features defining sensitive or private text; whereas differential privacy training, which provides provable privacy guarantees, reliably prevented memorization in the experiments, at the cost of additional computational overhead and some loss in model performance. Language models are widely used for automatic text completion in products, and the risk that large models memorize private training data and that adversaries may extract and leak it is the background of this research.