Recovering private text in federated learning of language models
Recovering Private Text in Federated Learning of Language Models
Translation in progress — untranslated text is omitted.
Researchers at Princeton University propose FILM, a gradient inversion attack against federated learning of language models; the authors claim to be the first to demonstrate the feasibility of recovering private text from large training batches of up to 128 sentences. The attack assumes an eavesdropper intercepts the gradients transmitted between clients and the server, first extracting from the word embedding gradients the set of words that appeared in the batch, then using beam search to reconstruct sentences with the help of the pretrained model's prior and memorization during training, and finally reordering the words by perplexity and gradient norm. Experiments on GPT-2 with WikiText-103 and Enron Email show that even with large batches, single sentences can still be reconstructed with high fidelity, and the iterative attack can recover about one third of the sentences in a 16-sentence batch. On the defense side, both gradient pruning and DPSGD cause significant utility loss; even with a pruning ratio as high as 0.9999, the attack can still recover more than 90% of tokens; the authors propose freezing the word embeddings when fine-tuning a public pretrained model on private text, which can effectively block the attack with little utility loss.
- In federated learning, gradients uploaded by clients may leak private data; there has been a large body of work on image recovery, but text recovery had previously succeeded only on small batches. The authors focus on the risk of text leakage in federated training of language models.
- FILM has three steps: extract the bag of words from the nonzero rows of the word-embedding gradients, reconstruct sentences via beam search using the bag of words plus language model probabilities, then perform phrase-level and word-level reranking with a scoring function combining perplexity and gradient norm; it can iteratively recover multiple sentences. For defense, it proposes freezing word embeddings during training.
- On GPT-2 base with WikiText-103 and Enron Email, at b=1 each metric is close to 1, and at b=128 the average ROUGE-L is about 0.29; at b=16, 100 iterations can recover about 34% of sentences. Gradient pruning and DPSGD incur large utility loss; starting from a pretrained model and freezing the word embeddings can completely prevent bag-of-words recovery while perplexity remains almost unchanged.
- The authors note: not validated on other NLP tasks; did not try batches above 128, with ablation only up to b=16; how to select good sentences in multi-sentence recovery is unresolved; freezing the word embeddings harms utility when training from scratch. Experiments use only GPT-2 base and two datasets, with 20 batches per condition.
The paper advances text gradient inversion in federated learning to large batches of up to 128 sentences, and its defense idea of freezing word embeddings offers a low-loss option for trainers in privacy-sensitive scenarios.