Don't forget to sign the gradient: GradSigns proposes a watermarking framework for DNNs

1. 01What problem does this paper attempt to solve?

   How to embed a high-capacity ownership watermark into a DNN that resists removal attacks and can be remotely verified via an API.

   Core question: how to protect the intellectual property (IP) of DNNs, so that the model vendor can prove ownership of a stolen model.

   • Scenario and motivation: Training top-tier DNNs requires data, manpower, and compute, and the model is the vendor's IP; an attacker who purchases a license may resell the model at a low price or deploy it beyond the permitted scope, which the authors argue requires the vendor to be able to verify and prove ownership.
   • Shortcomings of existing methods: The authors state that existing watermarking methods can embed only very limited information and are vulnerable to watermark removal attacks; most black-box methods can embed only one bit of information.
   • Core observation: Over-parameterization gives the non-convex optimization of DNNs multiple optimal solutions with comparable performance, each corresponding to a different decision boundary; GradSigns exploits this to find a solution that simultaneously carries signature information.
   • Proposed approach: Proposes GradSigns, which embeds a signature into the gradient of the cross-entropy loss with respect to the model input, has negligible impact on model performance, supports remote verification through the prediction API, and can embed multi-bit information.
   • Threat model (§2.2):
     • Attacker's goal: Profit from model M without paying royalties, remove the vendor's signature or hinder verification, while not excessively degrading model performance.
     • Attacker's knowledge: Has the compute and experience to train models, knows all existing watermarking techniques, and can design adaptive attacks against the deployed method.
     • Attacker's capability: Has far less data than the vendor's dataset D; can perform general attacks such as parameter pruning, model fine-tuning, and query invalidation-modification.
     • Victim system: Model M trained by the vendor on private dataset D; during verification, the stolen model provides outputs only through an API or encrypted software (black-box setting).

   Collapse this answer

2. 02What related research exists?

   Related work falls into two categories, white-box and black-box watermarking; black-box methods mostly rely on specially crafted trigger sets, and the authors state that they can embed only one bit of information.

   White-box watermarking

   • Uchida et al. (2017) — The paper states that this is the first work to focus on DNN watermarking; it embeds the watermark into the parameters of the model layers, and verification requires access to the model's internal details.

   Black-box watermarking (trigger-set based)

   • DeepSigns (Rouhani et al., 2018) — Can be used for both white-box and black-box; the black-box version fine-tunes the model with constructed images made from original data plus random labels, selects as the watermark key set the samples that the unmarked model misclassifies and the fine-tuned model classifies correctly, and determines ownership based on the model's accuracy on the key set.
   • Zhang et al. (2018), Adi et al. (2018), Guo & Potkonjak (2018) — Use specially crafted samples (unrelated images, overlaid content or Gaussian noise, abstract images) as the watermark key set and assign them random labels, training them together with the original training set; the authors point out that these methods all rely on the model's anomalous behavior on specially crafted inputs for discrimination, and that the contribution of Guo and Adi lies in using cryptography to establish reliable verification rather than a new watermarking technique, so §4 does not evaluate them.
   • AFS (Le Merrer et al., 2019) — Uses true and false adversarial examples as the watermark key set.
   • EXP (Namba & Sakuma, 2019) — Designed against parameter pruning, it uses an exponentially weighted custom activation function so that the watermark is carried by parameters with large absolute values; the authors also point out that their proposed query-invalidation-modification attack is effective against Zhang and DeepSigns.

   Comparison baselines and benchmarks

   • Baseline methods: DS (black-box version of DeepSigns), EXP, AFS, Zhang (three variants: Content/Noise/Unrelated), compared with GradSigns under pruning + fine-tuning attacks (Figure 2); white-box methods (Uchida, white-box DeepSigns) do not participate in the comparison because their assumptions are stronger.
   • Benchmarks: three image classification datasets: CIFAR-10, SVHN, YTF.

   Positioning of this paper

   The authors state that GradSigns is the first watermarking method to embed a signature using the gradients of model components; unlike contemporary black-box methods that can embed only one bit, GradSigns can embed multi-bit information, and its watermark key set is taken directly from the training data without modification or relabeling, making the query-invalidation-modification attack inapplicable.

   Collapse this answer

3. 03How does the paper address this problem?

   During training, a regularization term biases an N-bit signature into the input gradients; black-box verification uses zeroth-order gradient estimation plus a BER threshold decision.

   Overall idea: exploiting the property that over-parameterized DNNs have multiple solutions with comparable performance, an embedding regularization term is added to the training objective to encode the signature into the expected gradient of the cross-entropy loss with respect to the input; during verification, this gradient is extracted and combined with the embedding key to recover the signature. GradSigns has two stages, embedding and extraction (Figure 1).

   Preparation before embedding (§3.1)

   • Signature and carrier: Generate an N-bit vector b as the watermark; randomly select a set of input neurons C (carrier set); only the input gradients of carrier nodes participate in embedding.
   • Embedding key: Generate a matrix K (N×|C|, values in [-1,1]) that maps the expected gradients of the carrier nodes to an N-bit binary vector; randomly select a target class T and use images of class T to compute the gradients.
   • Generation method: b and K can be generated with a random number generator, or by hashing a message containing the owner's identity (§4.4, used for anti-forgery).

   Embedding objective

   • Total loss: J(x|θ, y) = J_{cross−entropy}(x|θ, y) + λ J_Embedding(x|θ, y), i.e., an embedding regularization term is added to the task loss, with λ a trade-off hyperparameter.
   • Definition of successful embedding: For each watermark bit j, the sign of the weighted sum of the carrier nodes' expected gradient G with the j-th row of K must equal b_j, i.e., each watermark bit corresponds to a half-space constraint (Equations 2 and 3 in the paper).
   • Form of the regularization term: Each watermark bit is treated as a binary classification task for a single-layer perceptron whose parameters are fixed to the j-th row of K, and G is trained with the binary cross-entropy loss (Equation 4 in the paper).

   Black-box extraction (§3.2)

   • Zeroth-order estimation: Backpropagation is unavailable in the black-box setting, so the carrier node gradients are estimated by difference quotients: Ĝ_c(x) ≈ (J(x + h e_c) − J(x))/h, with h set to 0.0001 in the experiments; each input requires |C|+1 loss evaluations, and the estimation error is on the order of O(|C|·h²).
   • Decision: The expected gradient is multiplied by K to recover the watermark; if the BER of the extracted watermark is below a threshold, the model is judged to belong to the vendor; the authors note that more query-efficient gradient estimation methods could be used instead.

   Reliability threshold (§4.2)

   • Null hypothesis test: The probability that the expected gradient of an unmarked model falls in any given half-space is 1/2; setting τ = 3×10⁻³ and solving the binomial tail probability yields the tolerated number of bit errors η.
   • Threshold results: 16-, 32- and 64-bit signatures require at least 14, 25 and 44 matching bits, respectively, to reliably claim ownership.

   Key parameter choices (§4)

   • λ: The smallest value in (0.0, 1.0] that achieves successful embedding; too large harms performance, too small causes embedding failure.
   • Carrier set size: The smallest value that achieves successful embedding; a larger carrier set improves BESR but increases the query cost of black-box verification.

   Collapse this answer

4. 04What experiments did the paper conduct?

   Measures fidelity, reliability, robustness against generic and adaptive attacks, resistance to forgery, efficiency and capacity on three image benchmarks.

   Experimental setup

   • Models tested: ResNet20 for CIFAR-10; Deep-ID for YTF; a CNN with 6 convolutional layers plus 2 fully connected layers for SVHN (Appendix C, Table 8).
   • Datasets: CIFAR-10 (50,000 training / 10,000 test, 10 classes); SVHN (over 100k digit images); YTF (1200 people, 100 images each, 120,000 images in total, 75% for training).
   • Watermark configuration: 16, 32, and 64-bit signatures are embedded on each benchmark; the target classes are respectively the airplane class in CIFAR-10, digit 1 in SVHN, and the individual with ID 1 in YTF; carrier set sizes are given in Table 6 (e.g., 384 for CIFAR-10 64-bit).
   • Baselines: DS, EXP, AFS, Zhang (Content/Noise/Unrelated); AFS, DS, and EXP each embed 30 key samples, with parameters per the original papers; Zhang's key sample configuration is given in Table 7.
   • Metrics: test accuracy, watermark accuracy, BER (threshold τ=3×10⁻³, 64-bit tolerates about 20 bit errors), BESR.
   • Attack settings: pruning rates of 0%–90% in 10 levels, fine-tuning for 10 epochs after pruning (early stopping, learning rate 0.0005), attacker samples per class ranging from 256 to 2048; the lowest watermark accuracy among models whose test accuracy is not lower than 90% of the baseline is reported.

   Main results

   Fidelity (Table 2, test accuracy):

   Swipe horizontally to read the table

   Dataset	Baseline	GS-16	GS-32	GS-64
   CIFAR-10	91.2%	90.1%	90.4%	90.5%
   SVHN	96.1%	95.5%	95.7%	95.3%
   YTF	99.6%	99.6%	99.6%	98.6%

   • The authors state that GradSigns satisfies fidelity, with performance drops averaging less than 1% on all benchmarks; the CIFAR-10 baseline uses the validation accuracy reported in the original paper, and the SVHN/YTF baselines use the same training settings as the watermarked models.
   • Capacity (§4.6): the authors state that contemporary black-box methods can embed only one bit of information, whereas GradSigns successfully embeds 16, 32, and 64 bits on all three benchmarks.

   Robustness: pruning+fine-tuning and query invalidation

   • Pruning+fine-tuning (Figure 2): On three benchmarks and four attacker data scales, the authors state that only the GradSigns watermark always survives and remains verifiable under the "pruning+fine-tuning" attack; the figure compares DS, EXP, AFS, and three variants of Zhang (the specific curve values are unreadable in the converted text).
   • Query invalidation-modification (§4.3.2, Appendix A): The authors state that this attack is effective against Zhang and DeepSigns but not applicable to GradSigns, because its watermark key set is taken directly from the training data, with no modification and no relabeling.

   Robustness: adaptive attacks (only 64-bit models reported)

   • Model quantization (Table 3): After quantizing parameters to 8-bit fixed-point integers, the BER on CIFAR-10/YTF/SVHN is 2/64, 1/64, and 0/64 respectively, with performance changes of -1.42%, -1.73%, and -0.6%, all below the tolerable threshold.
   • Adversarial fine-tuning (Table 4): Fine-tuning for 5 epochs with FGSM adversarial examples mixed with the original data gives BERs of 5/64, 0/64, and 0/64 respectively (the authors state the worst-case BER is less than 8%), with performance changes of -2.58%, +0.23%, and -0.22%.
   • Input noise addition (Figure 3): Adding zero-mean Gaussian noise (σ from 0.001 to 0.1), the authors state that as long as the noise is not so large as to severely degrade performance, the watermark remains verifiable.
   • Score rounding (Figure 4): The authors state that the rounding attack can break extraction with h=0.0001, but after increasing the estimation step size h, extraction succeeds even when predicted probabilities retain only one decimal place.
   • Score perturbation (Appendix B, Figure 5): 50 verification attempts per benchmark with h=0.1; the authors state that the watermark extraction accuracy is always above the threshold and the attack fails.

   Trustworthiness: forgery attack

   • Setup (Table 5): The attacker forges their own 32-bit signature into the SVHN model already embedded with a 32-bit watermark, retraining for 80 epochs, with 3 trials per data scale (λ respectively smaller than, equal to, and larger than the original embedding value), reporting the best BESR.
   • Results: With 256/512/1024/2048 samples per class, the BESR is 0.57/0.60/0.62/0.68, with performance overhead of -1.57%/-1.15%/-4.43%/-3.37%; only with 4096 samples per class (about 40k, comparable to the original dataset) does the BESR reach 1.0, with overhead -1.09%.
   • Authors' interpretation: The threat model assumes the attacker does not have data at this scale, and with such data they could train their own model; moreover, the forged model's performance is lower than the stolen model's, so the authors consider the forgery attack inefficient.

   Other ablations and analyses

   • Efficiency (Table 6): For CIFAR-10/SVHN, the minimum sample size is 50 and extraction queries per bit are < 500; for YTF, the minimum sample size is 20 and queries per bit are < 100; the sample sizes were verified as sufficient over 20 trials.
   • Carrier set size (§4): Experiments show that a larger carrier set improves BESR but increases black-box verification cost, so the smallest value that allows successful embedding is chosen.
   • Choice of λ (§4): The smallest λ that allows successful embedding is chosen; too large harms performance and too small causes embedding to fail.

   Collapse this answer

5. 05What are some points that can be further explored?

   The paper does not specifically discuss limitations; the experiments cover three image benchmarks and watermarks of up to 64 bits, and some attacks are tested only on the 64-bit model.

   Limitations and future directions noted by the authors

   • The paper does not specifically discuss limitations and has no Limitations or Future Work section.
   • Method-level caveats mentioned by the authors (§4, §4.5): too large a λ harms the performance of the watermarked model, while too small a λ causes embedding to fail; a larger carrier set increases black-box verification cost; extracting more watermark bits requires more queries, and the authors note that random-direction gradient estimation can reduce verification queries (§4.5).

   Scope of experiments

   • Models and tasks tested: ResNet20 (CIFAR-10), Deep-ID (YTF), and a 6-convolutional-layer CNN (SVHN), 3 models in total, all image classification tasks (§4, Appendix C).
   • Watermark size: 16, 32, and 64-bit signatures are tested on each benchmark (§4).
   • Attack evaluation: General attacks (pruning+fine-tuning, query invalidation-modification) and adaptive attacks (quantization, adversarial fine-tuning, input noise addition, score rounding, score perturbation); the adaptive attack section reports only results for the 64-bit model, and the authors state that results on the other benchmarks are consistent (§4.3.3).
   • Forgery attack: Conducted only on the SVHN 32-bit watermarked model, with 3 trials per data scale (§4.4).
   • Comparison baselines: DS, EXP, AFS, and three variants of Zhang, compared only under the pruning+fine-tuning attack (Figure 2); white-box methods are not included in the experiments (§4.3).
   • Not reported in the paper: training and extraction time, and the measured cost of watermark extraction on a real deployed API.

   Collapse this answer

6. 06Summarize the main content of the paper

   GradSigns embeds a multi-bit signature into the input gradients, with small performance loss on three benchmarks, and the authors state that it resists universal and adaptive removal attacks.

   A paper on DNN intellectual property protection that proposes the black-box watermarking framework GradSigns.

   • Problem: Training DNNs is costly, and the model is the vendor's IP, but buyers may resell or misuse it; the authors state that existing watermarking methods have limited embedding capacity and are easily broken by removal attacks.
   • Method: Exploiting the property that over-parameterized networks have multiple equivalent solutions, a regularization term is added to the training loss to encode an N-bit signature into the expected gradient of the cross-entropy loss with respect to the input; for black-box verification, the watermark is extracted via the API using zeroth-order gradient estimation, and ownership is determined by a BER threshold under a null hypothesis test (τ=3×10⁻³).
   • Fidelity: Embedding 16–64 bits on CIFAR-10/SVHN/YTF, test accuracy drops by an average of less than 1% relative to the baseline (Table 2, e.g., CIFAR-10 baseline 91.2%, GS-16 90.1%).
   • Robustness: The authors report that the watermark remains verifiable under pruning+fine-tuning (Figure 2, comparison methods fail), 8-bit quantization (BER at most 2/64, Table 3), adversarial fine-tuning (BER at most 5/64, Table 4), input noise addition, score rounding (extractable after increasing the estimation step size), and score perturbation (all 50 trials above the threshold).
   • Anti-forgery: On the SVHN 32-bit model, when the attacker has at most 2048 samples per class, the forged BESR is at most 0.68, and forgery succeeds only when the data scale is comparable to the original dataset (Table 5).
   • Authors' conclusion: GradSigns satisfies the six requirements in Table 1, and the authors consider it to provide strong proof of ownership; experiments are limited to three image classification benchmarks, the watermark is at most 64 bits, and black-box extraction requires hundreds of queries per bit.

   Collapse this answer

Related papers