Skip to content

Pareto-Secure Machine Learning (PSML): Fingerprinting and Security Protection for Inference Serving Systems

Pareto-Secure Machine Learning (PSML): Fingerprinting and Securing Inference Serving Systems

arXiv· arXiv:2307.01292· Debopam Sanyal, Jui-Tse Hung, Manav Agrawal, Prahlad Jasti, Shahab Nikkhoo, Somesh Jha, Tianhao Wang, Sibin Mohan, Alexey TumanovGeorgia Institute of Technology; University of California, Riverside; University of Wisconsin–Madison; University of Virginia; George Washington UniversityIndexed by this site · 23 days ago· Originally publishedDefenses

Quick read

Attacks

Based on the paper PDF (AI-generated); refer to the original

The authors propose a fingerprinting algorithm that adapts model extraction attacks to model-less inference serving systems; on CIFAR-10, extraction accuracy within 4000 queries is close to the single model setting, and they propose a noise-adding defense.

Threat modelEnd-to-end black-box attack: the attacker poses as an ordinary client and queries the model-less inference serving system through the cross-hair interface; does not know the accuracy, latency, or number of any model in the model library; the goal is to extract the functionality of the highest-accuracy model in the library within a given latency budget (without stealing weights or training data); information leakage comes from the accuracy, latency, and prediction results returned by each query.

Questions
Existing black-box model extraction attacks assume the attacker can repeatedly specify the same victim model, whereas modern model-less inference serving systems hide models behind an abstraction layer, so the attacker cannot determine which model they are interacting with, making old attacks hard to apply directly.
Methods
First use a binary-search-style fingerprinting algorithm to probe the accuracy and latency of each model on the model library's Pareto frontier, then lock onto the victim model according to the latency budget and continuously trigger it, thereby reusing the single-model extraction attack (MixMatch); the defense adds Laplace noise to the accuracy and latency specifications of queries, perturbing the fingerprinting process.
Experiments and results
On CIFAR-10 within a 4000 query budget, the fingerprint attack's extraction accuracy and fidelity at all three model scales are close to the single model setting (within 1%); the noise-adding defense reduces accuracy by at most 9.8% and fidelity by at most 4.8% on medium and small models, but is limited on large models, and the more noise, the lower the system goodput.
Limitations and future work
The authors point out that the defense provides weaker protection for large models and give a fidelity lower bound proof (Appendix B); the experiments are conducted only on CIFAR-10 and SVHN image classification model libraries, with one attack (MixMatch), under a 4000 query budget, and the defense is tested at only four ϵ values.
Experimental setupDenseNet-161、ResNet-101 et al. · CIFAR-10、SVHN · accuracy、fidelity et al.
Evaluated models
DenseNet-161ResNet-101MobileNetV3-small
Benchmarks
CIFAR-10SVHN
Metrics
accuracyfidelitygoodput
AI summaryThe paper proposes for inference serving systems an attack method that first performs fingerprinting and then carries out model extraction, and gives a corresponding noise-adding defense. The authors state that existing black-box extraction attacks assume the same model can be selected repeatedly, whereas the abstraction layers provided by modern inference serving systems break this assumption; their fingerprinting algorithm enables an attacker to reliably trigger a specified model, with extraction fidelity and accuracy within 1% of directly attacking a single specified model, and compared with a naive attack, accuracy improves by up to 14.6% and fidelity by up to 7.7%. The defense blocks fingerprinting by adding noise to specified performance metrics, reducing attack accuracy and fidelity by up to 9.8% and 4.8% respectively on medium-scale model extraction, and while maintaining over 80% goodput, forms a configurable trade-off between defense strength and throughput; the authors state it has been implemented in a real system and is planned to be open-sourced.

The paper proposes for inference serving systems an attack method that first performs fingerprinting and then carries out model extraction, and gives a corresponding noise-adding defense. The authors state that existing black-box extraction attacks assume the same model can be selected repeatedly, whereas the abstraction layers provided by modern inference serving systems break this assumption; their fingerprinting algorithm enables an attacker to reliably trigger a specified model, with extraction fidelity and accuracy within 1% of directly attacking a single specified model, and compared with a naive attack, accuracy improves by up to 14.6% and fidelity by up to 7.7%. The defense blocks fingerprinting by adding noise to specified performance metrics, reducing attack accuracy and fidelity by up to 9.8% and 4.8% respectively on medium-scale model extraction, and while maintaining over 80% goodput, forms a configurable trade-off between defense strength and throughput; the authors state it has been implemented in a real system and is planned to be open-sourced.

Paper analysis

6 questions; open one to read the full answer

  1. What problem does this paper attempt to solve?

    Adapts black-box extraction attacks that rely on repeatedly specifying the same victim model to model-less inference serving systems, and provides a corresponding defense.

    Core problem: modern model-less inference serving systems break the black-box model extraction attack assumption that "the same victim model can be repeatedly specified," so existing attacks cannot be applied directly.

    • Problem setting: inference serving systems (e.g., INFaaS, Clockwork) hide a model zoo behind a model-less interface; users submit only accuracy and latency requirements, and the system selects the model. The authors consider the models in the model zoo to be high-value intellectual property facing extraction threats.
    • Shortcomings of existing methods: the authors state that existing black-box extraction attacks assume the attacker can route all queries to the same explicitly specified victim model, and this assumption is outdated for model-less systems; the paper's naive experiment (Table 1) shows that applying MixMatch directly to the model zoo drops accuracy from 92.00 to 81.04 and fidelity from 87.13 to 79.84.
    • Core observation: the system selects models only from the Pareto frontier, and each query leaks the accuracy and latency of the selected model, which the attacker can use to progressively reconstruct the entire frontier and lock onto the target.
    • Paper's contributions: propose a query-efficient fingerprinting algorithm that lets the attacker consistently trigger the victim model; propose a defense that adds Laplace noise to query accuracy/latency specifications; implement both on PSML, a real system based on Clockwork. The authors state this is the first work they know of to demonstrate the feasibility of black-box attacks on inference serving systems that do not require users to explicitly specify a model.
    • Threat model:
      • Goal: within a given latency budget L, extract the functionality of the highest-accuracy model in the model zoo that meets the budget (without stealing weights or training data).
      • Knowledge: black-box; does not know the accuracy or latency of any model, nor the number of models in the model zoo.
      • Capability: poses as an ordinary client and submits queries with accuracy/latency specifications through the cross-hair interface; total query budget 4000.
      • Victim system: a model-less, latency-aware, real-time inference serving system (the paper implements it based on Clockwork).
  2. What related research exists?

    Related work falls into three categories: inference serving systems, black-box model extraction attacks, and defenses against extraction; the authors claim this paper is the first attack study targeting model-less systems.

    Inference Serving Systems

    • TensorFlow Serving, Clipper, Amazon Sagemaker, NVIDIA Triton—early or commercial inference serving systems, all requiring users to explicitly specify the model.
    • InferLine, INFaaS—INFaaS is a model-less system where users only give latency and accuracy bounds and the system navigates the model library to select a model; this paper's system model follows INFaaS's interface.
    • Clockwork, iGniter—exploit the predictability of inference latency to provide deterministic service; this paper builds PSML on Clockwork, but the paper notes that Clockwork still requires the client to specify the model.

    Black-Box Model Extraction Attacks

    • Tramèr et al., Papernot et al., Knockoff Nets, Copycat CNN, etc.—replicate the victim model's functionality by querying it and training an extraction model on its outputs; the paper collectively calls these single-model attacks because they assume all outputs come from the same victim model.
    • MixMatch extraction attack (Jagielski et al., 2020)—uses semi-supervised learning with unlabeled task data to improve extraction, with the attacker only obtaining output labels; this paper uses it directly as a black-box attack module.
    • Muse (Lehmkuhl et al., 2021)—the paper notes that although it targets inference serving systems, it assumes the attacker can repeatedly access the victim model in the model library.

    Defenses Against Model Extraction

    • Output perturbation (e.g., Prediction Poisoning, etc.)—the paper considers this infeasible because legitimate clients would also receive perturbed outputs.
    • Watermarking (e.g., Entangled Watermarks, DAWN)—the paper notes that it requires post-hoc analysis and that the model owner must be able to obtain the extracted model.
    • Malicious client detection (e.g., PRADA, Pal et al.)—the paper notes that it assumes small l2 distance between adversarial queries and a mix of natural and synthetic queries, an assumption that does not hold under the MixMatch attack.

    Baselines and Benchmarks: the attack baselines are the single-model setting (all queries answered by the victim model) and the naive mzoo no-FP setting (direct extraction without fingerprinting); the benchmarks are image classification model libraries trained on CIFAR-10 and SVHN.

    Authors' positioning: the authors state that to their knowledge, this paper is the first work to demonstrate the feasibility of black-box attacks on inference serving systems that do not require users to explicitly specify the model, and propose a corresponding noise defense.

  3. How does the paper address this problem?

    A binary-search-style fingerprinting algorithm reconstructs the Pareto frontier to lock onto and continuously trigger the victim model, and a defense that adds Laplace noise to query specifications disrupts fingerprinting.

    The overall approach has two steps: first use a fingerprinting algorithm to probe the accuracy and latency of each model on the model library's Pareto frontier, enabling the attacker to reliably route queries to the victim model and thereby adapt a single-model extraction attack (MixMatch) to a model-less system; the defender, in turn, adds noise to the accuracy/latency specifications of queries, distorting the fingerprinting results.

    Problem Setting and Formalization

    • Pareto frontier: the set of models in the model library for which no other model has both higher accuracy and lower latency; the system selects models only from the frontier; the frontier is the traversal path of the fingerprinting algorithm.
    • Feasible set: F = {(ai, li) ∈ P ∣ (ai ≥ acc) ∧ (li ≤ lat)}, i.e., the set of frontier models satisfying the query's accuracy lower bound acc and latency upper bound lat; the system picks one model from it to serve according to a policy (the paper uses random selection).
    • Granularity and bounds: the system has minimum distinguishable granularities accg and latg for accuracy and latency (e.g., accuracy 0.1%, latency 1 millisecond), and latency has an upper bound lup; when no accuracy specification is provided, it defaults to 0.

    Fingerprinting Algorithm (Algorithm 1)

    • Binary search: starting from (areq, lreq) = (0.5, ∞), binary search for the accuracy amax of the most accurate model on the frontier, then binary search its latency lmax; then narrow the bounds to find the next-best models one by one until the entire frontier is reconstructed.
    • Query complexity: worst case is O(n(log(1/accg) + log(lup/latg))), where n is the number of frontier models; based on random simulation experiments the authors state the average query complexity is linear O(n) (Figure 2a), and remains linear under different accg and latg (Figure 2b, 2c).
    • Locking onto the victim model: the attacker selects in the frontier matrix the row k with the largest latency below the budget L, and uses its (ak, lk) as the specification for all subsequent queries, thereby continuously triggering the victim model.

    Noise Defense (Algorithm 2)

    • Where noise is added: Laplace noise is added to the accuracy and latency specifications of each query, with noise strength controlled by a single parameter ϵ (smaller ϵ means more noise); the system determines the feasible set from the perturbed specifications but never serves a model whose latency exceeds the original specification.
    • Design rationale: the authors state that adding noise to model profile values is infeasible because the system cannot distinguish the fingerprinting phase from the label-collection phase, and the attacker could still use the noised specifications to reliably trigger the victim model; adding noise directly to each query instead makes the query process after fingerprinting also uncertain.
    • Cost and measurement: noise changes the feasible set, causing some queries to violate accuracy or latency specifications; the paper uses goodput (the proportion of queries satisfying both accuracy and latency specifications) to measure system performance loss.

    Attack Procedure and Success Criteria

    • Attack procedure: fingerprinting (consuming part of the query budget) → selecting the victim model specification according to budget L → querying with the remaining budget and training a MixMatch extraction model (ResNet-50) on the returned labels for 1024 epochs.
    • Success criteria: the extraction model's accuracy (task accuracy) and fidelity (proportion of outputs consistent with the victim model) on the test set, compared with the single model setting and the no-fingerprinting setting; defense effectiveness is measured by the accuracy/fidelity drop and the goodput trade-off.
  4. What experiments did the paper conduct?

    Tested on CIFAR-10/SVHN model zoos with a 4000-query budget: fingerprinting attack accuracy is close to the single-model setting, and the noise defense is effective for small and medium models but sacrifices goodput.

    Experimental Setup

    • Victim models: 12 models each in the CIFAR-10 and SVHN model libraries (ResNet, WideResNet, DenseNet, MobileNet families, Figure 4, Figure 9); divided into three tiers by attacker latency budget: large model DenseNet-161 (21 ms), medium model ResNet-101 (13 ms), small model MobileNetV3-small (5 ms).
    • Extraction model: ResNet-50 (attacker's initial model), trained with MixMatch for 1024 epochs.
    • Datasets: CIFAR-10 and SVHN (SVHN results in Appendix A); models were not trained to full accuracy in order to construct a realistic Pareto frontier.
    • Baselines: single model (all 4000 queries answered by the victim model) and mzoo no-FP (direct attack on the model library without fingerprinting).
    • Metrics: accuracy, fidelity (proportion of outputs on the test set consistent with the victim model), goodput; defense sensitivity ∆facc = 1, ∆flat = 18.271 (CIFAR-10).
    • Budget and repetitions: total query budget 4000 (including fingerprinting and labeling); each defense experiment setting was run 3 trials and averaged (Table 4).

    Main Results

    CIFAR-10, 4000 query budget, MixMatch trained for 1024 epochs (according to Table 2, Table 3):

    Swipe horizontally to read the table

    SettingSmall model accMedium model accLarge model accSmall model fidMedium model fidLarge model fid
    single model67.2988.9892.0064.6378.4887.13
    mzoo no-FP60.0872.9681.0456.5470.3079.84
    mzoo FP66.2987.5291.3564.278.0187.13
    Defense ϵ=100068.2187.2791.2463.2978.1686.74
    Defense ϵ=10066.8387.1790.7761.1377.7686.51
    Defense ϵ=5065.1985.0590.8160.9477.0086.85
    Defense ϵ=1062.9877.7090.8858.6673.1986.74
    • The authors state that the fingerprint attack's accuracy and fidelity at all three scales are close to the single model setting (within 1%), and attribute this to fingerprinting consuming part of the query budget, with the single model setting still slightly better.
    • The authors state that with the defense at ϵ=10, the medium model setting's accuracy drops by at most 9.8% and fidelity by at most 4.8% (relative to the fingerprint attack without defense), and that smaller ϵ gives stronger protection.
    • Exception: in the large model setting the defense is weak, with accuracy still 90.88 at ϵ=10; the authors explain that in the large model scenario, even with added noise, service is still often provided by a high-accuracy model, so labeling quality degrades only limitedly.

    Fingerprint Query Overhead

    • What was measured: the query cost of the fingerprinting algorithm on a real system. Results: fingerprinting the CIFAR-10 model library used 411 queries (Section 7.2.1); without defense the number of fingerprint queries is constant (the algorithm is deterministic).
    • Changes under defense (Table 4, medium model): at ϵ=1000 fingerprinting averages 901.33 queries, dropping to 406.00 at ϵ=10; at ϵ=50 failed queries reach 1125.67. The authors state that noise makes fingerprint query complexity decrease as ϵ decreases, but failed queries increase.

    Trade-off Between Defense and Goodput

    • What was measured: the relationship between goodput and attack success rate for the medium model (CIFAR-10) under different ϵ (Figure 8). Results: goodput decreases as ϵ decreases; the authors state that at ϵ=50 goodput is above 0.8 while accuracy and fidelity drop noticeably, making it the best trade-off among the tested ϵ values in the medium model setting.
    • Authors' interpretation: goodput ≥ 80% is regarded by the paper as acceptable; system designers can configure ϵ to balance protection and performance.

    Large Model Fidelity Lower Bound (Appendix B)

    • What was measured: theoretical analysis of the lower bound on extraction model fidelity under defense. Results: the authors prove that when both the victim model and the extraction model have accuracy ≥ 0.9, the fidelity lower bound is 0.8.
    • Authors' interpretation: attacking high-accuracy (large) models still yields relatively high fidelity even with defense; obtaining a noticeable fidelity drop requires targeting low-accuracy (small) models.

    Other Ablations and Analyses

    • Effect of query budget (Figure 5, CIFAR-10 large model): accuracy and fidelity generally rise as the query budget increases; in the no-FP setting accuracy is instead higher at 2000 queries, which the authors explain as more labeled points not implying higher labeling quality.
    • Trigger probability distribution (Figure 7, medium model): under noFP the victim model ResNet-101 is rarely triggered, under FP it is triggered with high probability, and after the noise-adding defense the probability mass is spread to other models in the model library.
    • SVHN results (Appendix A, Table 5, 6): trends are consistent with CIFAR-10, e.g., large model single model accuracy 95.66, no-FP 87.61, FP 95.69.
    • Effect of granularity (Figure 2b, 2c): the authors state that fingerprint query complexity remains O(n) under different accg and latg.
  5. What are some points that can be further explored?

    The authors point out that the defense provides weaker protection for large models and give a theoretical explanation; the experiments are limited to two image datasets, one attack, and four ϵ values.

    Limitations and Future Directions Noted by the Authors

    • Weaker protection for large models (Section 7.3.1): the authors state that at the tested ϵ values the defense is more effective on medium and small models and limited on large models, because in the large model scenario service is still often provided by a high-accuracy model even with added noise; ϵ can be configured as the system designer needs.
    • Fidelity lower bound (Appendix B): the authors prove that when attacking high-accuracy models, fidelity remains ≥ 0.8 even with the defense in effect, so obtaining a noticeable protective effect requires targeting low-accuracy models.
    • Open-source plan (Sections 6, 8): the authors plan to open-source the PSML system; it had not been released at the time of writing.

    Scope of Experiments

    • The tested model libraries are 12 image classification models each on CIFAR-10 and SVHN (ResNet, WideResNet, DenseNet, MobileNet families), and the models were not trained to full accuracy.
    • The attack experiments use only one extraction attack, MixMatch; the paper states the defense is effective against any single-model attack, but the experiments validate only MixMatch.
    • The query budget is fixed at 4000; the defense experiments test four values ϵ = 1000, 100, 50, 10, with 3 trials averaged per setting.
    • The system model assumes the system selects models only from the Pareto frontier and that the model selection policy is uniform random; the paper mentions a lowest-cost policy as an alternative but does not experiment with it.
    • The paper does not report measured results of the fingerprinting algorithm on larger model libraries (e.g., hundreds of models), only a complexity analysis via random simulation (Figure 2).
  6. Summarize the main content of the paper

    Model extraction attacks and noise defenses against model-less inference serving systems: the fingerprinting algorithm reconstructs the Pareto frontier, and the noise-adding defense is effective on small and medium models.

    A paper on attacks and defenses for reasoning serving system security, targeting the model-less model library serving setting.

    • Problem: black-box model extraction attacks assume the same victim model can be repeatedly specified, but modern systems such as INFaaS and Clockwork hide models behind an abstraction layer, and the authors state that this assumption is outdated; a naive direct attack on the model library drops accuracy from 92.00 to 81.04 (Table 1).
    • Method: a binary-search-style fingerprinting algorithm is proposed that uses the accuracy and latency leaked by each query to recover the Pareto frontier (411 queries measured on the CIFAR-10 model library), enabling the attacker to continuously trigger the victim model and reuse MixMatch extraction; the defense adds Laplace noise to the accuracy/latency specification of each query (ϵ controls strength), perturbing the fingerprint while never violating the latency specification.
    • Result one: on CIFAR-10 with a 4,000-query budget, the fingerprinting attack reaches accuracy of 66.29/87.52/91.35 on small/medium/large models, within 1% of the single-model setting (67.29/88.98/92.00) (Table 2).
    • Result two: at ϵ=10 the defense reduces medium-model accuracy by at most 9.8% and fidelity by at most 4.8%; but in the large-model setting the defense is weak, with accuracy still 90.88 at ϵ=10 (Table 2, 3).
    • Result three: there is a trade-off between protection and system performance; at ϵ=50 the medium-model setting has goodput above 0.8 and attack success rate decreases, and the authors call it the best compromise among the measured values (Figure 8).
    • Authors' conclusion: the authors state that this is, to their knowledge, the first work to demonstrate the feasibility of black-box extraction on a model-less inference serving system, that the noise defense can provide configurable protection while maintaining acceptable goodput, and that they plan to open-source the PSML system.

Related papers

Show 3 more papers
Read originalarxiv.org