Skip to content

Robust Membership Encoding for Deep Learning: Inference Attacks and Copyright Protection

Robust Membership Encoding: Inference Attacks and Copyright Protection for Deep Learning

arXiv· arXiv:1909.12982· Congzheng Song, Reza ShokriCornell University; National University of SingaporeIndexed by this site · 23 days ago· Originally publishedAttacks

Quick read

Attacks

Based on the paper PDF (AI-generated); refer to the original

The authors propose membership encoding: during training, the membership of a subset of the training set is encoded into the model; in the white-box setting on the Texas dataset, encoding 20% of the data achieves precision 0.97, with almost no drop in test accuracy.

Threat modelMLaaS/algorithm market setting: a third-party attacker provides the training code, which can access customer data but cannot exfiltrate it, and whose only output is the model; during training, the 1-bit membership of a selected subset is encoded into the model according to a key, and after the model is released it is decoded via white-box (access to parameters) or black-box (prediction API). In the watermark scenario, the model owner trains locally and the service provider holds the key to verify ownership.

Questions
In MLaaS and algorithm markets, third-party training code may secretly embed sensitive information about the training data into the model for later extraction; at the same time, model owners need robust watermarks to protect copyright. Existing embedding methods have small capacity, modify many parameters, and are fragile to model modifications.
Methods
During training, a subset of the training set is selected, and a membership discriminator is added and jointly optimized with the main model, encoding the 1-bit membership of the subset, via synthetic data generated from a key seed, into the model's hidden activations or output log probabilities; at inference, the same key is used to reconstruct the discriminator for decoding.
Experiments and results
Evaluated on Purchase, Texas, MNIST, and CIFAR10 with MLP/CNN/VGG/ResNet: white-box encoding with 20% training data achieves precision 0.83–0.97, with test accuracy roughly on par with the baseline; black-box is slightly lower. Encoding is robust to input masking, pruning, and fine-tuning; against adversarial pruning, the encoding is still retained when 70% of parameters are pruned.
Limitations and future work
The paper does not specifically discuss limitations. The experiments cover 4 datasets and 4 model architectures, with an encoding ratio of 20% of the training set (increased to 50% in some experiments); no defenses or repetition counts are reported.
Experimental setupMLP (Purchase)、MLP (Texas) et al. · Purchase、Texas Hospital et al. · test accuracy、encoding precision et al.
Evaluated models
MLP (Purchase)MLP (Texas)MLP (MNIST)CNN (MNIST)VGG (CIFAR10)ResNet (CIFAR10)
Benchmarks
PurchaseTexas HospitalMNISTCIFAR10STL-10
Metrics
test accuracyencoding precisionencoding recallAUC
AI summaryThe paper proposes a membership encoding method for deep neural networks that can encode membership information of a subset of the training data into the model. The method can be used for robust watermarking for model copyright protection, and also for analyzing the risk of hidden data-embedding privacy attacks. The authors state that the encoded model can determine whether heavily modified data points belong to the training set, and remains robust to model compression and fine-tuning, while the drop in prediction accuracy caused by encoding a relatively large portion of the training set is negligible.

The paper proposes a membership encoding method for deep neural networks that can encode membership information of a subset of the training data into the model. The method can be used for robust watermarking for model copyright protection, and also for analyzing the risk of hidden data-embedding privacy attacks. The authors state that the encoded model can determine whether heavily modified data points belong to the training set, and remains robust to model compression and fine-tuning, while the drop in prediction accuracy caused by encoding a relatively large portion of the training set is negligible.

Paper analysis

6 questions; open one to read the full answer

  1. What problem does this paper attempt to solve?

    In the MLaaS setting, robustly encode the membership of a subset of the training set into the model, for privacy attack risk analysis and copyright watermarking.

    Core question: how, when training a deep neural network, to secretly and robustly encode the membership of a subset of the training data (1 bit: whether it was used for training) into the model, such that only the key holder can decode it.

    • Setting and importance: With the rise of MLaaS and algorithm markets, data holders use third-party training code to train models. The authors argue this raises two concerns: model copyright needs protection; and malicious training code may embed secret information about the training data into the model, which can be extracted after deployment (covert channel).
    • Shortcomings of existing methods: The authors state that existing embedding algorithms exploit the network's redundant capacity, relying on overfitting or directly writing parameters to embed secrets, requiring about 4,000 poisoning data points and modifying a large number of parameters, with poor scalability and fragility to model modifications (pruning, etc.).
    • Core observation: Deep networks have the ability to memorize random data; the model can be made to learn a distinguishable representation (hidden layer or output) for a target subset without harming main-task accuracy.
    • Proposed in the paper: the membership encoding algorithm—during training, a membership discriminator is jointly optimized to encode the membership of a selected subset into the model according to a key; it can be used for watermarking to verify ownership, and also for analyzing the model's susceptibility to covert channel attacks.
    • Threat model:
      • Attacker goal: encode the membership of a selected training subset (e.g., records of some sensitive attribute) into the model, decode it after the model is released, and perform membership inference or reconstruction attacks.
      • Knowledge and capability: the attacker provides the training code, which can access customer data during training but cannot exfiltrate it (platform security isolation); the only output is the model; the attacker holds a secret seed s.
      • Victim system and access: the model owned and released by the customer; two release modes, white-box (access to parameters) or black-box (prediction API only).
      • Watermark scenario: the model owner trains locally, and a third-party service provider holds the key s to verify ownership; an attacker wants to forge a watermark to pass verification, and the authors consider this unlikely to succeed without knowing s.
  2. What related research exists?

    Related work falls into four groups: membership inference attacks, memorization and data embedding in ML, model watermarking, and other privacy attacks against ML.

    Membership inference attacks

    • Shokri et al. [28]—black-box membership inference, training a discriminator with shadow models, with the attack working best when the model overfits; Yeom et al. [34] theoretically link membership inference to generalization/overfitting.
    • Subsequent work extends attacks to differentially private models [25], generative models and language models [15, 31], and collaborative learning [23, 24]; Long et al. [22] show that well-generalizing models also leak membership information, but the paper notes that they first need to identify a small number of susceptible records.
    • Paper's positioning: membership encoding makes well-generalizing models leak membership information in both white-box and black-box settings, and covers a large proportion of the training set rather than a small number of records.

    Memorization and data embedding in ML

    • Zhang et al. [35]—deep networks can achieve perfect training performance on random data (memorization ability).
    • Song et al. [30]—a malicious training algorithm encodes training data into the model for later extraction; the paper notes that its white-box attack directly writes parameters, and its black-box attack encodes one lossy CIFAR10 image requiring 4,096 random data points, so it can only encode a small number of inputs, whereas this paper encodes in hidden activations or outputs.
    • Carlini et al. [6]—generative sequence models unintentionally memorize training text, and specific inputs can be extracted when the input format is known.

    Model watermarking

    • A body of work embeds watermarks into model parameters [32], prediction outputs on trigger sets [1], or intermediate activations [7, 26], embedding an owner-specific secret binary string.
    • Paper's positioning: this paper embeds the membership of a training subset, imposing a stronger constraint on the verifier—the correct encoded training records must be known to claim ownership.

    Other privacy attacks against ML

    • Attribute inference [3, 23], model inversion (Fredrikson et al.), inferring class representatives with GANs in collaborative learning [17], stealing hyperparameters [33].
    • Paper's positioning: unlike the above attacks that infer attributes of a class or subgroup, membership encoding targets the exact membership of a subset of the training set.

    Baselines and benchmarks: The experimental baseline is a normally trained model without membership encoding (baseline accuracy); the benchmarks are Purchase, Texas Hospital, MNIST, and CIFAR10, with STL-10 used in transfer experiments.

  3. How does the paper address this problem?

    During training, a membership discriminator is added and optimized alternately with the main model, pushing the target subset's representation toward the synthetic data distribution generated from the key, achieving decodable membership encoding.

    Overall idea: while training the classification model fθ, also train a membership discriminator dϕ, so that the model learns, for the target subset DC-m, a representation similar to the synthetic reference data DA-m and distinguishable from the rest of the training data; at inference time, the secret seed is used to regenerate the synthetic data and rebuild the discriminator for decoding. The method is shown in Algorithm 1 and Figure 1.

    Problem setting and notation

    • The customer's training set DC is split into the subset DC-m encoded as member and the rest DC-nm; the attacker additionally generates two sets of synthetic data with different distributions, DA-m and DA-nm; Dm = DC-m ∪ DA-m, Dnm = DC-nm ∪ DA-nm.
    • Encoding objective: jointly minimize the main-task loss and the encoding loss, i.e., minθ Lce and minθ,ϕ Lenc, where Lenc(h,z) = z · log dϕ(h) + (1−z)·log(1−dϕ(h)), h is the model representation and z is the membership label, meaning the discriminator is trained to distinguish member from nonmember representations.

    Synthetic data generation (GenSyntheticData)

    • A secret integer s is used as the pseudorandom number seed, ensuring that the encoding and inference stages generate exactly the same data.
    • Two mean vectors are sampled from a uniform distribution, then n points are sampled from each of two multivariate Gaussians with unit covariance, forming naturally separated DA-m and DA-nm.

    Encoding stage

    • Discriminator input: for white-box, hidden-layer activations (in the experiments, tanh is applied to the selected activations); for black-box, the log of the output probabilities hθ = log(fθ); the authors state that using [0,1] probabilities directly harms main-task accuracy, whereas log values are unconstrained and allow more flexible encoding.
    • Alternating optimization: in each iteration, first update θ and ϕ on mini-batches of Dm ∪ Dnm to optimize Lenc (k steps), then optimize Lce on mini-batches of DC; the authors state that the two objectives do not conflict this way.

    Inference stage

    • Use the seed s to regenerate DA-m and DA-nm, query the target model to obtain representations, and train a new discriminator dϕ′ (experiments use a one-layer MLP with 128 hidden units, trained for 80 epochs).
    • For any record x, the membership probability is dϕ′(hθ(x)), used for privacy attacks or watermark verification.

    Key parameters and variants

    • By default, 20% of the training set is encoded, with 1,000 synthetic data points (500 member, 500 nonmember); for white-box, 50% of hidden-layer units are randomly selected for encoding (20% for CIFAR10).
    • Against pruning: first prune the p% smallest parameters, then attach a new discriminator and fine-tune with customer data and synthetic data, with gradients of pruned parameters set to zero to maintain sparsity.
    • Hybrid attack: combined with the high-capacity embedding of [30], using membership encoding to eliminate collisions from compressing non-sensitive attributes; the paper does not give experimental numbers for this combination.
  4. What experiments did the paper conduct?

    Evaluate white-box/black-box encoding precision and main-task accuracy on 4 datasets and 4 architectures, and test robustness to input masking, pruning, and fine-tuning.

    Experimental Setup

    • Models tested: two-layer MLP for Purchase and Texas; MLP and CNN for MNIST; 9-layer VGG and 32-layer ResNet for CIFAR10.
    • Datasets: Purchase (150,000 training / 30,000 test, 600-dimensional binary features, 5 classes), Texas Hospital (50,000 / 10,000, 6,170-dimensional, 10 classes), MNIST (50,000 / 20,000), CIFAR10 (50,000 / 10,000); transfer experiments use STL-10 (5,000 training / 800 test).
    • Baseline: a model trained normally without membership encoding (baseline accuracy).
    • Metrics: test accuracy; encoding precision, recall, AUC (evaluated on the selected training data and test data).
    • Default settings: encode 20% of the training data; 1,000 synthetic data points; inference discriminator trained for 80 epochs; the paper does not report the number of repetitions or variance.

    Main Results

    White-box encoding (Table 2, encoding 20% of the training data):

    Swipe horizontally to read the table

    DatasetModelBaseline accTest accEnc preEnc recEnc AUC
    PurchaseMLP98.50%98.12%0.850.960.93
    TexasMLP97.85%97.82%0.970.750.98
    MNISTMLP97.80%97.75%0.830.880.93
    MNISTCNN99.03%99.04%0.870.940.95
    CIFAR10VGG90.16%89.67%0.920.980.98
    CIFAR10ResNet92.10%91.85%0.830.980.96

    Black-box encoding (Table 4, encoding 20%): MNIST MLP test accuracy 95.38%, precision 0.70, recall 1.00, AUC 0.95; MNIST CNN 97.94%, 0.85, 1.00, 0.98; CIFAR10 VGG 85.92%, 0.93, 0.70, 0.94.

    • The authors state that the encoding models' main-task accuracy is essentially on par with the baseline, while leaking 20% of the training data with relatively high precision/recall/AUC.
    • Figure 2 (CIFAR10 white-box) shows Lce and Lenc converging simultaneously, with the classification loss for members and nonmembers converging to the same point; the authors state that encoding did not cause overfitting to the member set; in Figure 4 (black-box), nonmember loss is lower than member loss, and the authors state that black-box encoding biases member outputs.
    • In the PCA and T-SNE projections in Figures 5 and 6 (CIFAR10 VGG), member (DA-m, DC-m) and nonmember (DA-nm, DC-nm) clusters are separated, and DC-m and DC-nm each still form 10 class clusters in T-SNE.

    Capacity and Encoding Quantity

    • Model capacity (Figure 3, Purchase MLP and MNIST CNN, encoding 20%): test accuracy is unaffected by parameter count; the authors state that encoding performance rises with capacity, rising slightly on Purchase and more on CIFAR10 (note: the main text here says CIFAR10, while the figure caption says Purchase and MNIST).
    • Encoding quantity (Table 3): on Purchase, |DC-m| increases from 30K to 55K, precision drops from 0.85 to 0.64, and AUC drops from 0.94 to 0.50; on CIFAR10 VGG, it increases from 10K to 35K (50% of the training set), precision drops from 0.92 to 0.83, and AUC drops from 0.98 to 0.77, with test accuracy on par with the baseline. The authors state that the faster decline on Purchase may be because MLP capacity and representational flexibility are inferior to VGG.

    Robustness: Input Masking, Pruning, Fine-tuning

    • Input masking (Table 5, CIFAR10 VGG, mask width w=4/8/12/16): with white-box center masking at w=8, precision is 0.88 and recall is 0.67; at w=16, recall drops to 0.12. Black-box recall drops more (center masking at w=8 gives recall 0.20). Table 6 (16 8×8 regions) shows precision stable at 0.87–0.91, with lower recall when the masked region is near the center (minimum 0.67).
    • Pruning (Figure 7, CIFAR10 VGG/ResNet): with ordinary pruning at 30%, precision is about 0.70; as the pruning ratio increases, both encoding and test accuracy decline; adversarial pruning (VGG, fine-tuned for 20 epochs) leaves test accuracy almost lossless when 70% of parameters are pruned, with a small decline in encoding performance.
    • Transfer fine-tuning (Figure 8, CIFAR10 → STL-10, 60 epochs): encoding performance for both models remains stable across fine-tuning epochs; the authors state that encoding is robust to cross-task fine-tuning.

    Other Ablations and Analyses

    • Black-box vs white-box (MNIST CNN): precision 0.85 vs 0.87, test accuracy 97.94% vs 99.04% (Tables 2, 4).
    • Masking location: center masking retains higher precision than boundary masking (Table 5, e.g., w=8 white-box 0.88 vs 0.77).
    • Negative results: on Purchase, AUC drops to 0.50 when encoding 55K items (Table 3); in black-box masking with w≥12, recall drops to 0.01–0.02 (Table 5); the paper gives no explanation.
  5. What are some points that can be further explored?

    The paper does not specifically discuss limitations; the experiments cover 4 datasets and 4 architectures, and no defenses or repetition counts are reported.

    Limitations and future directions noted by the authors

    • The paper does not specifically discuss limitations; there is no Limitations section, and the conclusion (Section 8) and discussion do not explicitly list shortcomings or future work.

    Scope of experiments

    • The models tested are 4 architectures in total: MLP, CNN, VGG, and ResNet, all classification models trained by the paper itself (Section 5.2).
    • The datasets are Purchase, Texas Hospital, MNIST, and CIFAR10; the transfer experiments additionally use STL-10 (Sections 5.1 and 6).
    • The default encoding ratio is 20% of the training set; in the encoding-count experiments, this increases to 55K on Purchase and 35K on CIFAR10 (50% of the training set) (Table 3).
    • Robustness evaluations include input masking (CIFAR10, w=4–16), parameter pruning (10%–90%), adversarial pruning, and cross-task fine-tuning (Section 6).
    • The paper does not report the number of experimental repetitions or confidence intervals, and does not evaluate defense methods against encoding; the mixed attack (Section 3.1) is only a design, with no experimental numbers in the paper.
  6. Summarize the main content of the paper

    Proposes membership encoding, which robustly encodes the membership of a training subset into the model, with white-box precision up to 0.97 and almost no drop in main-task accuracy.

    An attack paper on training-data information embedding in the MLaaS and algorithm market setting, whose encoding mechanism can also be used for copyright watermarking.

    • Problem: Third-party training code can, during training, secretly embed the membership of a subset of the training data into the model, for extraction by the key holder after release; existing embedding methods have small capacity, make large modifications, and are fragile to model modifications.
    • Method: During training, a membership discriminator is added and optimized alternately with the main model, pushing the representation of the target subset toward the synthetic data distribution generated from a secret seed; white-box encoding is on hidden activations (tanh), black-box encoding on output log-probabilities; at inference, the same seed is used to rebuild the discriminator for decoding.
    • Main results (Table 2, encoding 20% of the training data): white-box precision 0.83–0.97, 0.97 on Texas with test accuracy 97.82% (baseline 97.85%); CIFAR10 ResNet AUC 0.96, test accuracy 91.85% (a drop of 0.25 percentage points).
    • Black-box (Table 4): MNIST CNN precision 0.85, recall 1.00; CIFAR10 VGG precision 0.93 but recall 0.70.
    • Robustness: under central 8×8 masking on CIFAR10, white-box precision 0.88 (Table 5); against pruning, the encoding is largely preserved when 70% of parameters are pruned (Figure 7); encoding performance is stable after 60 epochs of cross-task fine-tuning (Figure 8).
    • Authors' conclusion: Deep networks can simultaneously learn to leak training-data membership information without affecting the main task; this mechanism can be used for watermarking to verify ownership, and also for analyzing the model's susceptibility to covert channel attacks.

Related papers

Show 3 more papers
Both use: MNIST
Read originalarxiv.org