Information leakage in embedding models

Researchers conduct a privacy leakage study on text embedding models, proposing three types of attacks: embedding inversion, sensitive attribute inference, and membership inference. The attacks can recover 50%–70% of input words on popular sentence embeddings, with F1 of 0.5–0.7; author identity can be inferred using 10–50 labeled sentences per author, a 1.5–3× improvement over prior stylometry methods; and a membership inference advantage of about 30% is achieved on low-frequency training data. The study evaluates on models including Word2Vec, FastText, GloVe, BERT, and ALBERT, and proposes an adversarial training defense that reduces inversion attacks and attribute inference attacks by about 30% and 80%, respectively, at the cost of a small utility loss.