LOGAN: Membership Inference Attacks Against Generative Models

1. 01What problem does this paper attempt to solve?

   Studies whether generative models leak training set membership information, proposing white-box and black-box membership inference attacks.

   Core question: given a data record and access to a generative model, can the attacker determine whether that record was used to train the model. The authors describe this as research on membership inference attacks against generative models, which is harder than the analogous attack on discriminative models because generative models do not output prediction confidence that can be exploited.

   • Scenario and motivation: MLaaS platforms let users query models trained on private data; in applications such as synthetic medical images and face generation, membership itself is sensitive (e.g., revealing health conditions or criminal records), and can also be used by regulators for forensics and by platforms for privacy assessment.
   • Shortcomings of existing methods: Membership inference by Shokri et al. relies on prediction vectors output by the model, and generative models have no such signal; the authors state that detecting overfitting in generative models is itself regarded as an important research problem in machine learning.
   • Core observation: If a generative model overfits, the discriminator of a GAN trained on its samples can learn statistical differences between training and non-training samples, even if invisible to the human eye.
   • Paper contributions: The authors propose a white-box attack (directly reusing the target discriminator) and a GAN-based black-box attack (including both no auxiliary knowledge and limited auxiliary knowledge variants), evaluated on DCGAN, DCGAN+VAE, BEGAN and three datasets LFW, CIFAR-10, DR, and examine defenses.
   • Threat model:
     • Goal: infer whether a single known record is in the target generative model's training set.
     • Knowledge: in both settings the attacker knows the training set size; in black-box the attacker does not know the target model's parameters, hyperparameters, architecture or training data.
     • Capability: white-box can access the target GAN's generator and discriminator parameters (e.g., data leak or model deployed to mobile devices); black-box can only query the target model to obtain generated samples.
     • Victim system: generative models trained on sensitive image data (GAN-type; the black-box attack places no restriction on the target model type).

   Collapse this answer

2. 02What related research exists?

   Related work falls into three categories: membership inference on discriminative models, other privacy attacks, and defenses; this paper extends membership inference to generative models.

   Membership inference on discriminative models

   • Shokri et al. [57]—for supervised models, uses shadow models for each class to learn the difference between member/non-member outputs. The paper notes that it requires prediction vectors and some dataset information, whereas this paper targets generative models and does not rely on prediction vectors.
   • Yeom et al. [70] and Long et al. [37]—analyze the relationship between membership inference and model inversion, overfitting; [70] assumes the attacker knows the training distribution and colludes with the training algorithm.
   • Membership inference on genomic and location data [24, 5, 50]—infer whether an individual's data exists in an aggregated dataset.

   Other privacy attacks

   • Model inversion [19, 18]—infer sensitive input features from model outputs; the paper notes it cannot generalize to inputs unseen during training.
   • Hitaj et al. [23]—white-box attack on collaborative learning using GANs; the paper notes it cannot scale to black-box and was only evaluated on MNIST and AT&T's 400 grayscale faces, whereas this paper uses datasets of 13,233/60,000/88,702 images.
   • Inference attacks in distributed learning [2, 41], memorization and extraction of training data [58, 11], model stealing [61], etc.

   Defenses

   • Differentially private training [1, 56] and PATE [46, 48]; Rahman et al. [53] state that [1] partially mitigates the attack of [57].
   • Beaulieu et al. [6]—trains a differentially private GAN discriminator using the noisy gradient descent of [1] to generate clinical data; Nasr et al. [43]—uses adversarial regularization to defend against membership inference on discriminative models.

   Baselines and benchmarks: The paper's comparison baseline is random guessing (10% for LFW/CIFAR-10, 20% for DR); it also evaluates two unsuccessful strawman methods (Euclidean distance attack, shadow model attack, Appendix A). The datasets are LFW, CIFAR-10, Diabetic Retinopathy.

   The authors position this paper as a general framework extending membership inference to generative models: it does not rely on prediction vectors, and the black-box attack requires only generated samples from the target model.

   Collapse this answer

3. 03How does the paper address this problem?

   Detecting overfitting of the target generative model with a GAN discriminator: white-box reuses the target discriminator; black-box trains a GAN on generated samples.

   Overall approach: train a GAN to learn to distinguish "real samples in the target model's training set" from "other samples," and use the discriminator's sensitivity to distribution differences to detect overfitting. Attack accuracy is defined as the proportion of records correctly inferred as training set members.

   Threat model and setup

   • The attacker holds a candidate set X = {x1, ..., xm+n}, where n is the training set size (known to the attacker) and m is the number of non-member samples; the attack performs membership determination on the samples in X.
   • White-box: attacker Awb can access the parameters of the target GAN's generator Gtarget and discriminator Dtarget. Black-box: attacker Abb can only query the target model to obtain generated samples, and does not know its parameters, architecture, or training data.

   White-box attack

   • Awb copies Dtarget to obtain Dwb, feeds all of X into Dwb, and obtains a probability vector p = [Dwb(x1), ..., Dwb(xm+n)].
   • Sort p in descending order and take the n samples with the highest probabilities as the training set prediction (Fig. 3). If the target is overfitted, training samples have higher confidence.
   • The attacker does not need to train any model, only needs internal access to the target model.

   Black-box attack (no auxiliary knowledge)

   • Abb locally trains a GAN (Gbb, Dbb) using samples generated by Gtarget, i.e., treats the target model as a "real data" source and reconstructs a local copy of the target.
   • After training, it switches to the white-box procedure: feed X into Dbb, sort, and take the top n. The target model can be any generative model (not limited to GAN).
   • The attacker model is trained using soft and noisy labels (real sample labels replaced with random numbers in [0.7, 1.2], fake samples with [0.0, 0.3]) and occasionally flips labels to stabilize GAN training.

   Black-box attack (limited auxiliary knowledge)

   • Discriminative setting: when the attacker knows part of the test set (20%) or part of the training+test set (30%), it directly trains a binary classification discriminator: test samples are labeled fake, target-generated samples (and known training samples) are labeled real.
   • Generative setting: when the attacker knows 20% of the training set, or 30% of the training+test set, it injects the auxiliary samples into the training of the attacker GAN's discriminator Dbb: known training samples and target-generated samples are labeled real, Gbb-generated samples (and known test samples) are labeled fake.
   • Auxiliary knowledge is introduced only after 1000 steps of attacker model training; the paper reports that introducing it too early yields a weaker discriminator (Fig. 8).

   Success determination

   • The metric is membership inference accuracy (the proportion of records correctly identified as members), compared with a random guessing baseline (10% for LFW/CIFAR-10, 20% for DR); some results are reported as "improvement over random guessing."

   Collapse this answer

4. 04What experiments did the paper conduct?

   The white-box attack achieves 100% accuracy against DCGAN/DCGAN+VAE; the black-box attack without auxiliary knowledge achieves 22–40%, rising to at most 81% with auxiliary knowledge.

   Experimental setup

   • Target models: DCGAN, DCGAN+VAE, BEGAN (BEGAN excluded from black-box experiments due to low white-box accuracy); attacker model fixed as DCGAN.
   • Datasets: LFW (13,233 faces), CIFAR-10 (60,000 32x32 images), Diabetic Retinopathy (88,702 retinal images, taking moderate-to-severe samples with labels 2–4, accounting for 19.7%).
   • Training set construction: LFW and CIFAR-10 randomly take 10% as the training set; LFW additionally takes the ten most frequent classes (accounting for 12.2%) as a more homogeneous training set.
   • Metric: membership inference accuracy; random guessing baseline is 10% for LFW/CIFAR-10 and 20% for DR.
   • Procedure: white-box measures accuracy at successive epochs of target model training; black-box fixes the target model (epoch 400) and measures accuracy at successive steps of attacker model training; mini-batch is 32.
   • Environment: PyTorch, NVIDIA Titan X GPU; the paper does not report multiple repetitions or statistical significance tests.

   Main results

   According to Table 1 (random 10% training set; DR is the label 2–4 subset; target model DCGAN):

   Swipe horizontally to read the table

   Attack	LFW	CIFAR-10	DR
   White-box	100%	100%	95%
   Black-box (no auxiliary knowledge)	40%	37%	22%
   Black-box (limited auxiliary knowledge)	60%	58%	81%
   Random guessing	10%	10%	20%

   • The authors state that the white-box attack achieves perfect inference on DCGAN and DCGAN+VAE; BEGAN is much more resistant, with a white-box upper bound of 62% (LFW random 10%, Fig. 5b), and on CIFAR-10 only 9 percentage points above random (Fig. 5c).
   • The black-box attack's success rate when target and attacker architectures differ (DCGAN+VAE target, DCGAN attacker) is close to when architectures are the same; the authors state the attacker does not need to know the target architecture.
   • On DR, white-box approaches 100% at 350 epochs (Fig. 11a); black-box without auxiliary knowledge performs poorly, and with 30% auxiliary knowledge exceeds 80% after 35K steps (Fig. 11b).

   Effect of auxiliary knowledge

   • Discriminative method fails: knowing only 20% of the test set, the attack on LFW and CIFAR-10 is no different from random guessing; knowing 30% of the training+test set gives LFW 50%, CIFAR-10 33% (Fig. 7), not better than the generative black-box results without auxiliary knowledge.
   • Generative method (20% training set knowledge): LFW rises from 40% to nearly 60%, CIFAR-10 shows no improvement; the authors state this setting does not generalize (Fig. 9a).
   • Generative method (30% training+test set knowledge): LFW rises from 40% to 60%, CIFAR-10 rises from 37% to 58% (Fig. 9b); the authors state a small amount of auxiliary knowledge can improve the attack.

   Training set size and prediction ranking

   • As the training set proportion increases, the black-box attack's improvement over random decreases: 40–55% improvement at 10%, negligible improvement at 90% (Fig. 12); the white-box attack is unaffected by training set size.
   • The authors give two possible explanations: more training data makes the model fit better and not leak, or a small number of training records is itself hard to capture; they consider the latter more likely, because the improvement magnitude is similar on both datasets and white-box is unaffected.
   • Prediction ranking has structure: the top 20% of guesses have the highest accuracy, decreasing as candidates increase; the authors state that when the attacker does not know the training set size, the ranking can be used to set a threshold.

   Defense evaluation

   • Weight Normalization: white-box attack improves 88%/46% (top 10/top 500 classes), almost the same as 89%/52% without defense, and often causes training instability (Fig. 13).
   • Dropout (p=0.5): improvement drops to 70%/23%, but noticeably slows training, requiring more epochs to obtain usable samples (Fig. 13).
   • Differentially private GAN (δ=10⁻⁴): at ε=1.5 the attack is no better than random, at ε=28.3 accuracy reaches 85%; the authors note that under acceptable privacy levels (ε<10) sample quality is very poor (Fig. 14).

   Other ablations and analyses

   • Euclidean distance strawman attack (DCGAN target, LFW/CIFAR-10 random 10%) is no better than random guessing (Fig. 15).
   • Shadow model attack (DCGAN shadow model trained with 10% training set knowledge, LFW) accuracy is about 18%, with no improvement during training (Fig. 16).
   • Attack cost: black-box without auxiliary knowledge best result requires 50,000 steps, about 42 minutes, 32×50,000 queries; estimated at $2,352 by Google MLaaS pricing; the auxiliary knowledge version requires 15,000 steps, about 13 minutes, $672 (Section 6.4).

   Collapse this answer

5. 05What are some points that can be further explored?

   The authors state that existing defenses are either ineffective or harm sample quality, that new defenses need to be studied, and that they plan to extend to datasets such as location data.

   Limitations and future directions noted by the authors

   • Insufficient defenses: Weight Normalization, Dropout and differential privacy mechanisms are "only effective to some extent" and require longer training, bring training instability and/or worse generated sample quality; the authors state this motivates future research on defenses against information leakage in generative models (Section 7).
   • Practical tension of differential privacy: under acceptable privacy budgets (ε<10) sample quality is very poor (Section 6.3).
   • Future directions: the authors plan to apply the attack to other privacy-sensitive datasets, including location data (Section 7).
   • The authors state that the black-box attack success rate is lower than white-box (Section 5.4 summary), and that the generative auxiliary attack with 20% training set knowledge shows no improvement on CIFAR-10 and "does not generalize" (Section 5.5), but do not list it as future work.

   Scope of experiments

   • Target models tested are DCGAN, DCGAN+VAE, BEGAN, 3 in total; BEGAN was not used in black-box experiments (excluded due to low white-box accuracy).
   • The attacker model is fixed as DCGAN; shadow model experiments also use DCGAN.
   • Datasets are LFW, CIFAR-10, Diabetic Retinopathy, 3 image datasets in total; training sets are random 10% subsets or the LFW top ten classes.
   • Defense evaluation covers three: Weight Normalization, Dropout and differentially private GAN; the paper notes that defenses for discriminative models (limiting top-k, coarsening prediction vectors) do not apply to generative models because generative models do not output prediction vectors.
   • The paper does not report the number of experimental repetitions, statistical significance tests, or an upper limit on the total number of queries to the target model in black-box attacks (only an estimate of the number of queries needed to reach the best results).

   Collapse this answer

6. 06Summarize the main content of the paper

   Membership inference attack against generative models: the GAN discriminator detects overfitting, reaching 100% in the white-box setting and up to 81% in the black-box setting with auxiliary knowledge.

   An attack paper on privacy leakage in generative models, which the authors describe as a membership inference attack study against generative models: determining whether a given record was used to train a generative model.

   • Problem: Synthetic medical and facial data from generative models is increasingly widespread; if membership can be inferred, sensitive information is directly leaked; generative models have no prediction confidence, making attacks harder than against discriminative models.
   • Method: In the white-box setting, directly reuse the target GAN's discriminator to score and rank candidate samples; in the black-box setting, locally train a GAN on samples generated by the target model and use its discriminator to detect overfitting; when 20–30% auxiliary knowledge of the training/test sets is available, inject known samples into discriminator training.
   • Main results (Table 1, DCGAN target, random 10% training set): white-box achieves 100% on LFW and CIFAR-10, and 95% on DR; black-box without auxiliary knowledge achieves 40%/37%/22%; with 30% auxiliary knowledge this rises to 60%/58%/81% (random baseline 10%/10%/20%).
   • Differences across models: BEGAN is highly resistant, with a white-box ceiling of 62%, only 9 percentage points above random on CIFAR-10; the authors state that models with better generalization are more resistant to membership inference.
   • Defenses: Weight Normalization is essentially ineffective, Dropout is partially effective but slows training, and differential privacy blocks the attack at ε=1.5 but yields poor sample quality when ε<10; the authors state that new defenses targeting generative models are needed.
   • Cost: The black-box attack takes about 42 minutes and an estimated $2,352 (without auxiliary knowledge), which the authors consider low cost.

   The authors' conclusion: multiple generative models exhibit privacy leakage; the attack is cheap, requires no information about the target model, and generalizes reasonably well; the white-box attack can also serve as an empirical measure of overfitting and generalization quality in generative models.

   Collapse this answer

Related papers