Black-box privacy attacks on shared representations in multi-task learning
Black-Box Privacy Attacks on Shared Representations in Multitask Learning
Translation in progress — untranslated text is omitted.
Researchers propose a black-box task inference attack against shared representations in multi-task learning: an attacker can determine whether a certain task participated in training using only the embedding vectors obtained by querying the shared representation. The attack exploits dependencies among embeddings of the same task and does not require a shadow model or labeled reference data. In various multi-task learning scenarios in the vision and language domains, experiments show that even with only new samples of the target task rather than training data, a black-box attacker can successfully infer whether that task was in the training set. The authors also provide a theoretical analysis for a simplified learning setting, showing a strict separation between attackers with training samples and those with only new samples.
- The shared representation of multi-task learning (MTL) is supposed to be the minimal unit of information that parties need to share; the authors study whether it leaks which tasks participated in training, and pose a task-level (rather than sample-level) inference problem.
- Proposes a black-box task-inference threat model (strong and weak adversaries), and designs two attacks that require no shadow model: a coordinate variance attack and a pairwise inner product attack (including whitening); additionally uses a tracing attack based on Gaussian mean estimation for theoretical analysis.
- Under both MTL settings—personalization and multiple learning problems—on CelebA, FEMNIST and Stack Overflow, the strong and weak adversaries both achieve non-trivial AUC; the strong adversary reaches AUC 1.000 on Stack Overflow personalization, and the weak adversary's gap to the strong adversary becomes smaller in the multiple-learning-problem setting.
- The authors state that user-level differential privacy is a potential defense but was not experimentally validated; the attack still succeeds under gradient clipping and normalization. Experiments are limited to two MTL usages, three datasets, and ResNet/BERT architectures, and do not evaluate adaptive attacks or actual defense effectiveness.