StolenEncoder: A stealing attack against pretrained encoders in self-supervised learning
StolenEncoder: Stealing Pre-trained Encoders in Self-supervised Learning
The authors propose StolenEncoder, which steals pretrained image encoders by querying the EaaS; the authors state that SA/TA is at least 93% on three real encoders, and that all three defenses are insufficient.
Attacker's goal: steal the target encoder deployed as EaaS, preserving its functionality (Goal I) with few queries (Goal II). Knowledge: possesses a small-scale unlabeled surrogate dataset (which may be from a different distribution than the pretraining data), and does not need to know the target encoder's architecture or pretraining algorithm. Capability: can query the EaaS API to obtain feature vectors for input images. Victim system: a contrastive-learning-pretrained image encoder deployed as a paid EaaS.
- Self-supervised pretrained image encoders are being deployed as EaaS, and encoders need to be kept confidential due to training cost and misuse risk. The authors state that existing model stealing attacks mostly target classifiers, and that directly extending them to encoders does not work well.
- StolenEncoder queries the EaaS with a small number of unlabeled surrogate images to obtain feature vectors, minimizing the weighted sum of L1 (original-image feature distance) and L2 (approximating augmented-image features with original-image features), with the number of queries equal to the surrogate dataset size.
- On STL10/Food101/CIFAR10 self-trained encoders, SA/TA is no lower than 0.98; on the three real encoders Google ImageNet, OpenAI CLIP, and Clarifai, SA/TA is at least 93%. The authors state that the three defenses top-k features, feature rounding, and feature poisoning are all insufficient.
- The authors state that future work includes stealing the target encoder's hyperparameters (such as architecture) and developing new defense mechanisms. The experiments are limited to image encoders and four image classification downstream tasks; the Clarifai experiments use 5,000 surrogate images; the paper does not report variance over multiple repetitions.
- Evaluated models
- ResNet18ResNet34VGG19_bnVGG19DenseNet121MobileNetV2ShuffleNetV2x1ImageNet encoder (Google)CLIP encoder (OpenAI)Clarifai's General Embedding encoder
- Benchmarks
- STL10Food101CIFAR10MNISTFashionMNISTSVHNGTSRBImageNet (surrogate dataset)
- Metrics
- TASASA/TA#Queries
The paper proposes StolenEncoder, a stealing attack against pretrained image encoders in self-supervised learning, which the authors claim is the first attack of its kind. The attacker needs only a small number of unlabeled images as a substitute dataset, without knowing the pretraining data distribution, queries the EaaS API to obtain feature vectors, and then uses SGD to train a stolen encoder with similar functionality; the number of queries need only equal the size of the substitute dataset. On Google's ImageNet encoder, OpenAI's CLIP encoder, and Clarifai's paid EaaS, downstream classifiers trained on the stolen encoders achieved accuracy at least 93% of the target encoder's; stealing CLIP used only 113,000 STL10 images (less than 0.03% of its 400 million image-text pairs) and 53.9 hours on a single GPU, whereas the original encoder was trained for 432 hours on 592 V100 GPUs; stealing the Clarifai encoder cost 16 US dollars, whereas normally calling the API to train and test a downstream classifier costs 931.6 US dollars. The authors have reported the vulnerability to Clarifai.
Why it was selectedThe paper quantifies the stealing cost and functionality retention, showing that deploying an encoder only as a paid API is insufficient to protect confidentiality, and that three types of perturbation defenses also failed to mitigate it.
Paper analysis
What problem does this paper attempt to solve?
How to steal a confidential pretrained image encoder by querying the EaaS API, with few queries and low cost.
Core question: Can an attacker, using only queries to an EaaS API and a small number of unlabeled images, steal a confidential self-supervised pretrained image encoder so that it functions comparably to the target encoder?
- Problem setting: Self-supervised learning has given rise to the Encoder as a Service (EaaS) model, in which providers (such as Google, OpenAI, Clarifai) deploy pretrained encoders and charge per query. The authors state that encoders need to be kept confidential because of the high cost of training data and compute and because they could be misused (e.g., to generate deepfakes), and that deploying them only as a cloud service is insufficient to protect their confidentiality.
- Shortcomings of existing methods: The authors state that most model-stealing attacks target classifiers; classifiers return labels or confidence vectors whereas encoders return feature vectors, and loss functions used for classifiers are unsuitable for measuring feature vectors; the paper's experiments show that extending classifier-stealing attacks to encoders yields suboptimal results (in Table 8, the SA of StolenEncoder-w/o-aug is clearly lower).
- Core observation: A contrastive-learning-pretrained target encoder produces similar feature vectors for an image and its augmented version, so the feature vector of the original image can be used to approximate the feature vector of the augmented image, saving queries on augmented images.
- What the paper proposes: The authors state that they propose StolenEncoder, formalize it as an optimization problem and solve it with SGD; they evaluate it on a self-trained encoder and three real encoders (Google ImageNet, OpenAI CLIP, Clarifai) and explore three defenses.
- Threat model:
- Goals: Goal I maintains the target encoder's functionality (comparable downstream classifier accuracy); Goal II uses few queries (low economic cost).
- Knowledge: Holds a small unlabeled surrogate dataset, in three cases (a subset of the pretraining data, same distribution but non-overlapping, different distribution); does not need to know the target encoder's architecture (ResNet34 is chosen when unknown) or the contrastive learning algorithm used for pretraining.
- Capabilities: Can query the EaaS API to obtain feature vectors for input images.
- Victim system: A contrastive-learning-pretrained image encoder deployed as a paid EaaS.
What related research exists?
Related work falls into four categories: stealing classifiers, stealing hyperparameters and training data, defenses against classifier stealing, and other security issues of pretrained encoders.
Stealing classifiers
- Tramèr et al. [50]—steal logistic regression, decision trees, SVMs and simple neural networks by querying a target classifier deployed on MLaaS. Orekondy et al. [43]—use a reinforcement-learning-based method to reduce the number of queries for stealing a classifier. Chandrasekaran et al. [13]—explore the connection between active learning and classifier stealing attacks. Zhu et al. [59]—steal classifiers based on hardware side channels, assuming the attacker's process runs on the same machine as the target classifier.
- Concurrent work [16]—proposes an attack that steals encoder parameters, making the stolen encoder and the target encoder output similar feature vectors on a surrogate dataset; the paper states that this attack is a special case of StolenEncoder when λ=0, and that it uses cosine similarity as the distance metric, and the paper states that Table 8 and Table 6 show these two points are suboptimal, respectively.
- Krishna et al. [34] and Zanella et al. [55]—steal language models fine-tuned from publicly available pretrained language encoders (such as BERT); the paper states that the difference is that StolenEncoder steals the confidential pretrained encoder itself, rather than a model fine-tuned from a public encoder.
Stealing hyperparameters and training data
- Wang and Gong [51]—black-box stealing of hyperparameters of a trained model; Oh et al. [42]—steal neural network architecture and optimizer hyperparameters via queries; Yan et al. [53]—steal network architecture based on CPU cache side channels. He et al. [22]—steal the training graph structure by querying a graph neural network. Jia et al. [30]—assume an untrusted training library embeds training data or hyperparameters in the model for later stealing.
Defenses against classifier stealing
- Detecting anomalous queries: Juuti et al. [31] (PRADA), Zhang et al. [58] (SEAT). The paper states that such defenses are difficult to apply to encoder stealing: encoders inherently accept queries for different downstream tasks, making "anomalous" hard to define; and attackers can register multiple accounts to spread out queries.
- Perturbing confidence vectors: Kariyappa and Qureshi [33], Orekondy et al. [43, 44], Tramèr et al. [50]. This paper generalizes three of them to feature vectors (top-k features, feature rounding, feature poisoning) and evaluates them. Watermarking [5, 26, 39, 57] and fingerprinting [9, 40] are used for post-hoc verification of model ownership and do not prevent stealing.
Other security issues of pretrained encoders
- Data poisoning [37], backdoor attacks [11, 28], membership inference attacks [12, 23, 38], etc.
Baselines and benchmarks: The compared methods are Pre-training-encoder (self-training with SimCLR on the surrogate dataset), StolenEncoder-w/o-aug (using only L1, equivalent to generalizing classifier stealing attacks), StolenEncoder-query-aug (the high-query variant using L2′), and knowledge distillation [24]; the pretraining datasets are STL10, Food101, CIFAR10, and the downstream datasets are MNIST, FashionMNIST, SVHN, GTSRB. The authors position this paper as the first attack that steals self-supervised pretrained encoders (as stated by the authors).
How does the paper address this problem?
Stealing is formalized as an optimization problem minimizing L1+λ·L2, using original-image features to approximate augmented-image features, with the number of queries equal to the surrogate dataset size.
Overall approach: encoder stealing is formalized as an optimization problem such that the stolen encoder outputs feature vectors similar to the target encoder's on surrogate images and their augmented versions, solved with standard SGD to obtain the stolen encoder StolenEncoder.
Problem Formalization (Goal I)
- L1 term: L1 = 1/(|D|) ∑x ∈ D d(ft(x), fs(x)), i.e., the average distance between the target encoder's and the stolen encoder's feature vectors on the surrogate dataset images (ℓ2 distance by default). A small L1 indicates the two encoders' outputs are similar.
- L2′ term: because the surrogate dataset is small, using only L1 leads to suboptimal functionality; for each image, a data augmentation combination A (RandomHorizontalFlip, ColorJitter, RandomGrayScale by default) generates an augmented image, and the two encoders are required to output similar features on the augmented images as well, i.e., L2′ is the average feature distance on augmented images.
- Joint objective: min L′ = L1 + λ·L2′, where λ is a balancing hyperparameter (default 20).
Reducing Query Cost (Goal II)
- Key observation: a contrastive-learning-pretrained target encoder produces similar feature vectors for an image and its augmented version, i.e., f_t(A(x)) ≈ f_t(x).
- Approximation: define L2 = 1/(|D|) ∑x ∈ D d(ft(x), fs(A(x))), using the cached original-image features f_t(x) to approximate augmented-image features, replacing L2′.
- Query count: the final objective min L = L1 + λ·L2 requires only |D| total queries (the surrogate dataset size); directly optimizing L2′ would require (e+1)·|D| queries (e being the number of training epochs).
Solution Algorithm (Algorithm 1)
- Query the EaaS API once for each image in the surrogate dataset and cache the feature vectors.
- Randomly initialize the stolen encoder's parameters.
- For each epoch and each mini-batch, compute L1 and L2 and update the parameters with gradient descent; augmented images are regenerated randomly each epoch.
- Default parameters: λ=20, e=100, lr=0.001, batch size 64, distance metric ℓ2; stolen encoder architecture ResNet34 (assuming the attacker does not know the target architecture).
Functionality Evaluation Method
- Downstream classifiers: train a fully connected network on top of the target/stolen encoder (two hidden layers of 512/256 neurons, ReLU, Softmax output), Adam with initial learning rate 0.0001, batch size 256, trained for 500 epochs.
- Metrics: TA (test accuracy of the target encoder's downstream classifier), SA (test accuracy of the stolen encoder's downstream classifier), #Queries (number of queries needed for stealing, equal to the surrogate dataset size).
Defense Evaluation Design
- Three classifier-stealing defenses are extended to feature vectors: top-k features (features whose absolute value is not in the top-k are set to 0), feature rounding (features keep m decimal places), feature poisoning (solving a max-perturbation optimization problem so that the perturbed features become "data poisoning" for stolen training, with the perturbation's ℓ_p norm constrained by ε).
- Strong-defender assumption: in feature poisoning, the defender is assumed to be able to use exactly the same parameter settings as the attacker and to train a stolen encoder with its own pretraining dataset as the surrogate in order to construct the perturbation.
What experiments did the paper conduct?
On self-trained encoders SA/TA≥0.98, on three real encoders SA/TA≥93%, and the authors consider all three defenses insufficient.
Experimental Setup
- Target encoders: three self-trained (ResNet18 pretrained with SimCLR on STL10/Food101/CIFAR10, 1,000 epochs, batch 256, Adam, lr 0.001; MoCo and 7 architecture variants were also tested); three real: Google's ImageNet encoder (SimCLR, ResNet50, 2048 dimensions), OpenAI's CLIP encoder (modified ResNet50, 1024 dimensions), Clarifai General Embedding (paid EaaS, $3.2/1,000 queries, technical details undisclosed).
- Surrogate dataset: by default sampled from ImageNet (a different distribution from the pretraining data), with size 5% of the pretraining dataset (5,250/4,545/2,500 images for STL10/Food101/CIFAR10 respectively); the real-encoder experiments instead used STL10 (64,058 images for the ImageNet encoder, all 113,000 images for CLIP, 5,000 images for Clarifai).
- Downstream datasets: MNIST, FashionMNIST, SVHN, GTSRB (GTSRB by default, which the paper calls the most challenging one); images uniformly resized to 32×32 (224×224 for the real-encoder experiments).
- Baselines and variants: Pre-training-encoder, StolenEncoder-w/o-aug, StolenEncoder-query-aug, knowledge distillation (temperature 1).
- Metrics: TA, SA, SA/TA, #Queries; the paper does not report multiple repetitions or variance, and experiments were run on 1 Quadro-RTX-6000 GPU.
Main Results
According to Table 2 (default setting, downstream datasets row by row):
Swipe horizontally to read the table
Pretraining dataset Downstream dataset TA (%) SA (%) SA/TA×100% STL10 MNIST 97.63 97.74 100 STL10 SVHN 56.97 73.50 129 STL10 GTSRB 74.97 74.57 99 Food101 SVHN 62.22 77.50 125 Food101 GTSRB 79.69 78.12 98 CIFAR10 SVHN 59.63 77.04 129 CIFAR10 GTSRB 80.67 79.43 98 (Table 2 has 12 rows in total; the three FashionMNIST rows are omitted here, whose SA/TA is 100–101; in the complete table all SA/TA values are no lower than 0.98.)
- Authors' interpretation: the authors state that StolenEncoder preserves the functionality of the target encoder; the ratio exceeding 1 on SVHN is conjectured by the authors to be because SVHN is more similar to the surrogate dataset (ImageNet) than to the three pretraining datasets.
- Resource comparison: according to Table 3, stolen training takes 0.12–0.28 hours, whereas pretraining the target encoder from scratch requires 12.50–26.27 hours; the authors state that stealing is two orders of magnitude more efficient.
Real Encoder Case Studies (Section 6, Table 9, Table 10)
- Functionality: SA/TA on the three real encoders is at least 93%; the lowest are CLIP+GTSRB (TA 80.46%, SA 74.50%, ratio 93) and Clarifai+GTSRB (TA 61.00%, SA 58.16%, ratio 95).
- Resources: according to Table 10, pretraining the CLIP target encoder used 592 V100s for 432 hours (255,744 GPU hours) and 400 million image-text pairs; stealing used only 1 Quadro-RTX-6000 for 53.9 hours and 113,000 STL10 images (less than 0.03% of the pretraining data volume).
- Economic cost: training and testing four downstream classifiers with the Clarifai EaaS cost $931.6, whereas stealing that encoder with 5,000 queries cost only $16.
Defense Evaluation (Section 7, Tables 11–13, downstream dataset GTSRB)
- top-k features: as k decreases from 512 to 1, SA and TA both decrease (e.g., on STL10 with k=10, TA 48.67%, SA 63.44%); the authors state that reducing k lowers SA while lowering TA to the same or a greater extent, so the defense is insufficient.
- feature rounding: for m=1,2,3, SA is close to or even higher than TA (e.g., on Food101 with m=1, TA 70.13%, SA 78.73%); the authors state that this defense is ineffective.
- feature poisoning: as ε increases, SA and TA decrease in tandem (ℓ2 norm, on CIFAR10 with ε=10, TA 66.91%, SA 57.56%); the authors state that it lowers stolen-encoder functionality by harming target-encoder functionality, and is still insufficient.
Other Ablations and Analyses
- Effect of λ (Figure 2): SA first rises and then saturates as λ increases; the authors state that a relatively large value of λ (e.g., 20) suffices.
- Surrogate size (Figure 3): SA first rises and then saturates; the authors state that at 3% SA is already close to TA.
- Surrogate distribution (Table 4): SA is similar under the three distribution settings (e.g., on CIFAR10, 78.81/78.84/79.43).
- Target architecture (Table 5): with the stolen encoder fixed as ResNet34, SA is close to or higher than TA for 7 target architectures; the authors state that if the stolen architecture has weaker representational capacity than the target (e.g., MobileNetV2), SA is lower than TA.
- Distance metric (Table 6): SA with ℓ2 is higher than with ℓ1, and ℓ1 is higher than cosine (e.g., on CIFAR10, 79.43/74.52/61.07).
- Variant comparison (Table 8, CIFAR10+GTSRB, 5%): StolenEncoder SA 79.17% (2,500 queries), Pre-training-encoder 71.40% (0 queries), StolenEncoder-w/o-aug 51.50%, StolenEncoder-query-aug 79.11% (252,500 queries); knowledge distillation SA only 0.0954. Data augmentation combinations (Table 14): SA is high for all 16 combinations (71.47–79.94), and 51.50 without augmentation.
What are some points that can be further explored?
The authors state that future work could steal encoder hyperparameters and develop new defenses; the experiments are limited to image encoders and classification downstream tasks.
Limitations and Future Directions Noted by the Authors
- Stealing hyperparameters: Conclusion and Future Work (Section 9) states that future work includes developing attacks that steal the target encoder's hyperparameters (e.g., architecture).
- New defense mechanisms: Section 9 states that future work includes developing new defense mechanisms to mitigate StolenEncoder; the contributions list also states that the experimental results highlight the need for new defense mechanisms.
- Practical relevance of stealing public encoders: Section 6 acknowledges that stealing the ImageNet and CLIP encoders is "not practically relevant" because both are public; the authors cite GPT-3 being available only via API as an example, stating that image encoders may become private in the future.
- The paper has no dedicated Limitations section; the above are the shortcomings and directions explicitly written by the authors.
Scope of Experimental Coverage
- Subjects tested: 3 self-trained image encoders (ResNet18, SimCLR/MoCo) plus 3 real encoders (ImageNet, CLIP, Clarifai); downstream tasks are 4 image classification datasets.
- Defense evaluation: covers only the three defenses extended from classifier-stealing defenses (top-k features, feature rounding, feature poisoning), evaluated on self-trained encoders and GTSRB; anomalous-query-detection defenses were not tested, and the paper states they are difficult to apply to the encoder setting (Section 8).
- Query scale: in the self-trained experiments the surrogate is 5% of the pretraining dataset (2,500–5,250 images); the Clarifai experiment uses 5,000 images.
- Training configuration: target encoders are pretrained for 1,000 epochs and stolen training runs for 100 epochs; the paper does not report means or variances over multiple repeated runs.
- Disclosure: the paper states that Clarifai was notified that its EaaS is at risk of being stolen (Section 6).
Summarize the main content of the paper
StolenEncoder can steal an image encoder by querying the EaaS with a small number of unlabeled images, and the authors state that all three existing defenses extended to this setting are insufficient.
A paper on model stealing attacks: targeting self-supervised pretrained image encoders deployed as EaaS, the authors propose StolenEncoder, which the authors state is the first such attack.
- Problem addressed: encoders need to be kept confidential due to high training cost and misuse risk, but the authors state that deploying them only as a cloud service is insufficient to protect confidentiality; existing stealing attacks mostly target classifiers, and directly extending them to encoders yields suboptimal results.
- Method highlights: the attacker queries the EaaS with a small number of unlabeled surrogate images (which may have a different distribution from the pretraining data) to obtain feature vectors, and trains a stolen encoder by minimizing the weighted sum of L1 (original-image feature distance) and L2 (approximating augmented-image features with original-image features); the total number of queries equals the surrogate dataset size.
- Main results:
- On self-trained encoders (STL10/Food101/CIFAR10), SA/TA is no lower than 0.98, and even exceeds 1 on SVHN (Table 2).
- On three real encoders (Google ImageNet, OpenAI CLIP, Clarifai), SA/TA is at least 93% (Table 9).
- Stealing CLIP used only 113,000 images (less than 0.03% of the pretraining data volume) and 1 GPU for 53.9 hours, whereas the original training used 592 V100s for 432 hours; stealing Clarifai cost $16, whereas normally training and testing downstream classifiers with its API requires $931.6.
- Under the default setting, the knowledge distillation baseline SA is only 0.0954, versus 0.7943 for StolenEncoder.
- Defenses: top-k features and feature poisoning lower SA while lowering TA to a similar or greater extent, and feature rounding is essentially ineffective; the authors state that all three defenses are insufficient.
- Authors' conclusion: the authors state that stealing a pretrained image encoder requires far less data and compute than pretraining from scratch, and that future work needs to develop attacks that steal encoder hyperparameters and new defense mechanisms.