Covert Attacks Against Machine Learning Training in Passively Secure MPC

1. 01What problem does this paper attempt to solve?

   Can an active adversary carry out an effective attack in passively secure MPC training with almost no detection?

   Core question: Is a passively secure MPC machine learning training protocol sufficiently secure against an active adversary? The authors argue that the view that active cheating is easy to detect and therefore passive security is sufficient does not hold in privacy-preserving machine learning (PPML) training.

   • Setting and importance: MPC lets data owners train a model on combined data while protecting data privacy, but actively secure protocols incur greater computation and communication overhead, so many MPC ML studies and implementation frameworks consider only passive adversaries (Introduction).
   • Shortcomings of existing defenses: A common defense is that an active adversary's attack would cause an obvious deviation (e.g., a drop in test accuracy), and once detected would bring reputational loss. The authors state that this defense does not hold: an active adversary can achieve various malicious behaviors without being detected.
   • Threat model:
     • Goal: Break model integrity (poisoning, fairness) or privacy (membership inference, training data reconstruction) while remaining stealthy.
     • Knowledge: The adversary cannot read the secret-shared model parameters and input data; the additive error ϵ must be chosen independently of the secret-shared x, y.
     • Capability: Active adversary, focusing on additive attacks—injecting a chosen additive error ϵ into the secret-shared multiplication x·y; the authors state such attacks apply to almost all passive protocols.
     • Victim systems: Passively secure MPC training protocols, training logistic regression, SVMs and neural networks, in an outsourced MPC setting.
   • Paper contributions: The authors propose low-level attacks on comparisons and various activation functions (Section 3), combine them into three high-level strategies—gradient zeroing, gradient shifting, gradient scaling (Section 4)—and instantiate them as poisoning, membership inference amplification, training data reconstruction and fairness attacks (Section 5).

   Collapse this answer

2. 02What related research exists?

   Related work covers passively secure MPC ML protocols, adversarial machine learning (poisoning and membership inference), and active attacks on MPC.

   Passively secure MPC ML protocols and frameworks

   • Mohassel and Zhang (SecureML), Wagh et al. (SecureNN, Falcon), Keller and Sun et al. — a large body of MPC ML training and inference work considers only passive adversaries; implementation frameworks such as Sharemind, PySyft, CrypTen and CryptGPU also defend only against passive adversaries (Introduction).
   • Keller and Sun [4], Knott et al. [16], Tan et al. [17] — directly approximate the exponential and division in sigmoid and softmax, and are among the targets of this paper's low-level attacks.

   Adversarial machine learning: poisoning

   • Biggio et al., Jagielski et al. — availability poisoning, which degrades model accuracy; the authors state such attacks are easy to detect.
   • Gu et al. (BadNets), Chen et al. — backdoor attacks; Suciu et al., Shafahi et al., Jagielski et al. — targeted poisoning, which affects only a small number of target samples.
   • Bagdasaryan et al., Bhagoji et al. — model poisoning in federated learning (directly modifying gradients); the paper notes that these works all know the model parameters, whereas this paper's gradient shift must be performed with unknown parameters, which is a key difference.

   Adversarial machine learning: privacy attacks

   • Shokri et al., Yeom et al. — membership inference (MI); Carlini et al. (LiRA), Ye et al. — state-of-the-art MI that calibrates loss by sample difficulty; this paper adapts offline LiRA for attack evaluation.
   • Carlini et al., Balle et al. — reconstruction attacks; Boenisch et al. — reconstruction in federated learning exploiting the property that first-layer gradients are a scalar multiple of the input; this paper's reconstruction attack borrows this property.

   Active attacks and defenses on MPC

   • Lehmkuhl et al. (Muse) — considers an active adversary at inference time and uses an additive attack to steal the server's model weights; this paper targets the training phase.
   • Chaudhari et al. (SafeNet) — designs MPC protocols to mitigate input-level data poisoning and privacy attacks.

   Positioning: The authors state that, to their knowledge, they are the first work to consider an active adversary attacking ML training in MPC (Section 2.5). This paper has no baseline method comparison in the traditional sense; the experiments use "no attack" as the control and are evaluated on four datasets: FMNIST, Census, Purchase and Texas.

   Collapse this answer

3. 03How does the paper address this problem?

   Use additive attacks to manipulate activation functions, combine them into three strategies—gradient zeroing/shifting/scaling—and then instantiate them as various attacks.

   Overall approach: The active adversary exploits the weakness that passive protocols allow an additive error ϵ to be injected at multiplication, first manipulating secure comparisons and activation functions (low-level attacks), then combining them into three high-level gradient manipulation strategies, and finally realizing poisoning, privacy and fairness attacks. All manipulations must not depend on the secret-shared data and weights.

   Low-level attacks: activation function manipulation (Section 3, Table 1)

   • Secure comparison: The comparison circuit has only a single output bit, so the adversary can flip the result by injecting an error into the last AND gate; if the input is a multiplication result, a large constant offset can be added to force the output to 0 or 1.
   • ReLU: Flipping the comparison can turn the result into min(0, x); adding ∆ to the input forces the output to x+∆, and subtracting ∆ forces the output to 0 (input modification follows Lehmkuhl et al. [42]).
   • Piecewise-linear sigmoid: Tampering with the two comparisons can make the 0 and x+1/2 branches output 1 and the 1 branch output 0; combined with an input offset, out-of-range values x±∆ can be output.
   • Direct sigmoid/softmax: Both are ultimately multiplication outputs, so an arbitrary error can be added directly; input modification can force a one-hot output; multiplications in the Newton-Raphson iteration (n=10) and the limit-approximation exponential (n=8) can also be attacked.

   High-level strategies (Section 4)

   • Gradient Zeroing: Make the sample gradient as close to 0 as possible, equivalent to removing that sample from that training step. For neural networks, all ReLU outputs can be forced to 0 (except the last-layer bias); for SVM, a large error can be added to y·D to force the gradient to 0; the authors state they do not know how to achieve this for logistic regression.
   • Gradient Shifting: Add an error vi to the i-th term of the weight gradient computed by multiplication, changing the gradient g to g+v; applies only to weight gradients, not bias gradients.
   • Gradient Scaling: Exploiting the property that all gradients are scaled by the loss derivative F, force the sigmoid/softmax output to an arbitrary value to scale the gradient; can only encourage strong predictions for a fixed class, and cannot amplify or invert the gradient.

   Attack instantiations (Section 5)

   • Parameter Transfer: The adversary trains a reference model on its own data, computes the poisoning gradient gp, then injects it into the protocol via gradient shifting, solving the problem of unknown parameters; used for backdoors and targeted poisoning of linear models.
   • Neuron Override: For a two-layer ReLU network, add a shift of δ−cµ to the input weights of the first neuron (δ is the trigger direction, µ is the mean of non-target data, c is a trade-off hyperparameter), and shift that neuron's second-layer weight to the target class by a large positive value; inspired by the handcrafted backdoor of Hong et al. [53].
   • Membership inference amplification: Gradient scaling forces an arbitrary sample to be predicted as a random target class, making it an "outlier" sample similar to mislabeling, then adapts offline LiRA [36] (shadow models are not scaled, and deviation is measured against the scaled target class) for inference.
   • Training data reconstruction: For a single sample in the last batch, use a scaling factor of 10000 so that the first-layer weights are approximately a scalar multiple of the input, then rescale to recover the sample; the authors state this attack is not stealthy.
   • Fairness attacks and poisoning amplification: Gradient zeroing or scaling makes a participant's data contribute nothing or be misclassified; when colluding with a malicious data contributor, scale the gradient of poisoned samples to amplify poisoning.

   Simulator and implementation

   • The authors build a plaintext MPC "simulator" in Jax on floating point, implement MPC-friendly approximations (such as SecureML-style sigmoid), and replace gradients with custom_vjp, allowing the adversary to inject manipulations at the corresponding computation steps; experiments run on a single P100 GPU. The authors plan to release the implementation code.

   Collapse this answer

4. 04What experiments did the paper conduct?

   Plaintext simulation experiments show: poisoning ASR up to 1.0000, MI TPR improved by up to 40×, reconstruction error 0.017.

   Experimental setup

   • Models tested: softmax/sigmoid-based logistic regression (LR), two-layer neural network (NN, 50 neurons in the hidden layer for fairness experiments); SVM is discussed only at the method level, with no experiments.
   • Datasets: FMNIST, Census, Purchase, Texas; for the latter three, 20000 records each are used as the clean training set, 20000 for the adversary's training set, and 20000 as the test set.
   • Training: SGD, hyperparameters grid-searched on a separate data split; fairness experiments train for 10 epochs, learning rate 0.001, batch size 100.
   • Metrics: Test accuracy, Attack Success Rate (ASR), TPR at low FPR for membership inference (confidence interval p<0.01，宽度<0.26%）、重构用平均绝对误差。
   • 重复次数：投毒、公平性实验平均10次运行；重构攻击运行10次试验。
   • 评估方式：全部在Jax明文模拟器上进行，非真实MPC实现；无LLM评审。

   主结果

   投毒攻击（据 Table 3、Table 4，Test/ASR，10次平均）：

   Swipe horizontally to read the table

   攻击	数据集	模型	无攻击 Test	无攻击 ASR	攻击 Test	攻击 ASR
   参数迁移后门	FMNIST	LR	0.84	0.004	0.83	0.9996
   参数迁移后门	Texas	LR	0.61	0.006	0.61	1.0000
   参数迁移后门	Purchase	LR	0.70	0.000	0.70	1.0000
   参数迁移定向	FMNIST	LR	0.84	0	0.83	1
   参数迁移定向	Texas	LR	0.60	0	0.59	1
   参数迁移定向	Purchase	LR	0.70	0	0.67	1
   神经元覆写后门	FMNIST	NN	0.86	0.003	0.86	0.90
   神经元覆写后门	Texas	NN	0.58	0.01	0.57	1.00
   神经元覆写后门	Purchase	NN	0.55	0.000	0.51	1.00

   • 作者称参数迁移攻击在各目标上接近完全成功，精度平均下降不超过2.3%。
   • 作者称神经元覆写攻击成功率很高，测试精度下降不超过4%。

   成员推断放大（Section 5.3，Table 5）

   • 测了什么：梯度缩放（目标类固定为0）后offline LiRA的[email protected]%与1% FPR，对比不缩放。
   • 结果：缩放后[email protected]%分别为FMNIST/LR 4.8%（不缩放0.18%）、Texas/LR 76.0%（3.0%）、Purchase/LR 4.5%（1.9%）、FMNIST/NN 2.3%（0.2%）、Texas/NN 46.6%（0.8%）、Purchase/NN 6.0%（0.4%）。
   • 作者解读：作者称梯度缩放把TPR提升至少2倍、有时达20–40倍；精度影响常有限（FMNIST降0–2%），但Purchase上NN平均降8%；攻击成功率与精度可通过缩放强度权衡。

   训练数据重构（Section 5.4，Figure 2、Figure 3）

   • 测了什么：FMNIST上对LR与NN，缩放因子10000、目标类0，攻击最后一个批次中的随机样本，10次试验。
   • 结果：重构与原图的平均绝对误差LR为0.017、NN为0.036；作者称视觉上几乎与原图一致。
   • 例外：作者明确称此攻击不隐蔽——模型精度降至10%（随机猜测水平）。

   公平性与投毒放大（Section 5.5、5.6，Table 6、Table 7）

   • 公平性：Census上按CA、OK、OR、WA、WI五州模拟五方（每方1000条），敌手针对CA。NN+梯度缩放（强度2，目标类1）使CA准确率从0.724降至0.652（降7.2%），其他州平均从0.737降至0.710（降2.7%）；LR+缩放从0.748降至0.673；NN+置零从0.75降至0.714。作者称攻击对目标子群体造成不成比例的影响。
   • 投毒放大：定向投毒（每目标2–10条翻转标签样本，softmax目标类平移-2到-5）加缩放后ASR上升，如Texas/LR从0.08升至1、Texas/NN从0.28升至0.94；但FMNIST/NN精度从0.86降至0.75、Purchase/NN从0.54降至0.41。作者称放大后门投毒会导致模型不准，故只报告定向投毒。

   其他消融与分析

   • 可用性攻击（Table 8）：参数迁移加小幅度平移，FMNIST/LR精度从.836降至.717，Texas/LR从.605降至.565；作者称改用更大平移可加大降级。
   • 缓解分析（Section 6）：作者称FMNIST上的梯度平移投毒共修改约150万次乘法，若抽查比例α<2·10⁻⁵仍能以>1−2⁴⁰ probability of catching the adversary; but the authors state the attack was not optimized for the number of errors and may still be effective with several orders of magnitude fewer errors.
   • Fairness experimental setting: the authors state that results should only be better under a more highly non-iid setting.

   Collapse this answer

5. 05What are some points that can be further explored?

   The authors point out two limitations—floating-point simulation and small models—and propose follow-up directions such as middle-ground protocols and theoretical characterization.

   Limitations noted by the authors and future directions

   • Floating-point rather than fixed-point arithmetic: The authors write that one limitation is the use of floating-point rather than fixed-point arithmetic, but they believe this does not significantly affect the conclusions (Limitations, Section 7).
   • Limited to small models: The experiments are limited to small models; the authors state that because MPC training is costly, the literature also mostly uses very small models, and they expect the attacks to generalize to larger models (Limitations, Section 7).
   • Stronger protocol-specific attacks: The authors state that this paper uses only additive attacks to ensure broad applicability, and that stronger protocol-specific attacks are future work (Section 2.2).
   • Middle-ground protocols: The authors hope to promote protocol designs between passive and active security that defend against a clearly defined class of attacks; they also emphasize that the proposed mitigations may be bypassed by adaptive attacks and require more careful analysis (Sections 6, 7).
   • Theoretical question: The authors pose a future theoretical question—characterizing whether there exists a larger class of functionalities whose privacy can be completely broken in the passive setting (Section 7).
   • The additive error threat model itself: The authors believe the additive error threat model deserves deeper study by the adversarial ML community, including improving attack effectiveness and stealthiness, and theoretical understanding (Section 7).

   Scope of experimental coverage

   • The models tested are two classes: softmax/sigmoid-based logistic regression and two-layer neural networks; SVM is analyzed only at the method level, with no experimental results (Section 5).
   • The datasets are four: FMNIST, Census, Purchase, Texas; Census/Purchase/Texas each use 20000 for training, 20000 for adversary data, and 20000 for testing (Section 5).
   • All experiments are run on a Jax plaintext simulator, not on a real MPC protocol implementation; the experiments use a single P100 GPU (Section 5).
   • Poisoning and fairness experiments average 10 runs, reconstruction attacks use 10 trials; membership inference reports p<0.01 confidence intervals (Sections 5.1, 5.3, 5.4, 5.5).
   • The paper does not report runtime overhead on real MPC protocols, nor does it report experimental evaluation of the proposed mitigations (Section 6 is only a design idea).

   Collapse this answer

6. 06Summarize the main content of the paper

   The authors state that passively secure MPC training can be stealthily attacked by an active adversary, and that poisoning, privacy and fairness can all be broken.

   An attack paper targeting passively secure MPC machine learning training, questioning the common defense that "active cheating is easy to detect and passive security is sufficient."

   • Problem: MPC training protocols are mostly designed for passive security, relying on reputational risk to deter active adversaries; the authors claim an active adversary can compromise model integrity and privacy with almost zero detection risk.
   • Method: Focus on additive attacks—injecting data-independent additive errors in secret-shared multiplication. At the low level, manipulate secure comparisons and activation functions such as ReLU, sigmoid, and softmax; at the high level, combine them into three strategies: gradient zeroing, gradient shifting, and gradient scaling; then instantiate them as parameter-transfer poisoning, neuron overwriting, membership inference amplification, training data reconstruction, and fairness attacks, all validated on a Jax plaintext simulator.
   • Poisoning results: The parameter-transfer backdoor reaches ASR 1.0000 on LR for Texas/Purchase, and neuron overwriting reaches ASR 1.00 on NN for Texas/Purchase, with test accuracy drops of no more than 2.3% and 4%, respectively (Tables 3, 4).
   • Privacy results: Gradient scaling raises membership inference [email protected]%FPR to as high as 76.0% (Texas/LR; 3.0% without scaling); the reconstruction attack has a mean absolute error of 0.017 (LR)/0.036 (NN) on FMNIST, but reduces accuracy to 10%, and the authors explicitly state that it is not stealthy (Table 5, Figures 2–3).
   • Fairness results: In the five-party Census setting with target state CA, NN+gradient scaling reduces its accuracy by 7.2%, while other states drop by only 2.7% (Table 6).
   • Authors' conclusion: The authors say the results challenge the view that passive security is sufficient in PPML, argue that training should adopt actively secure protocols, and propose middle-ground mitigations such as actively secure sigmoid/softmax and spot-checking some multiplications, while emphasizing that adaptive attacks may bypass these mitigations.

   Collapse this answer

Related papers