Curse or redemption? How data heterogeneity affects the robustness of federated learning

1. 01What problem does this paper attempt to solve?

   How data heterogeneity affects federated learning backdoor attacks and defenses has previously lacked systematic study; this paper provides an empirical characterization.

   Core question: how the data heterogeneity inherent in federated learning (FL) affects the effectiveness of backdoor attacks and defenses, which previously lacked systematic study.

   • Setting and importance: FL trains locally on clients and aggregates on the server; training data is naturally heterogeneous and cannot be balanced or monitored as in centralized ML; the authors call this uncontrollable heterogeneity a key challenge of FL, yet it has been largely ignored from the perspective of backdoor attack robustness.
   • Shortcomings of existing work: The authors point out that existing FL backdoor attacks either assume IID data distributions or study non-IID only in a simplified way (Bagdasaryan et al. 2018; Bhagoji et al. 2019; Xie et al. 2019); existing defenses (such as activation clustering, k-means) require access to training data, are not applicable to FL, and no defense has been evaluated for effectiveness under different degrees of heterogeneity.
   • Core observations: Preliminary experiments show that higher heterogeneity lowers the median ASR and makes attacks harder to design (which the authors call redemption); but further experiments find that heterogeneity brings more curses—attack timing can improve attack effectiveness, skewed-feature defenses fail, and attackers can use distribution distance to optimize attacks.
   • Contributions of the paper: define a Heterogeneity Index to quantify heterogeneity; characterize the impact of heterogeneity through extensive experiments on synthetic data and the LEAF benchmark; discuss defense directions against the three curses and conduct proof-of-concept validation.
   • Threat model (copied from the paper, same as Xie et al. 2019; Sun et al. 2019; Chen et al. 2018):
     • Goal: inject a trigger to cause targeted misclassification without affecting main-task accuracy or disrupting convergence.
     • Knowledge and capability: malicious clients can manipulate various aspects of their model updates and local training process (training data, hyperparameters, model weights, local privacy mechanisms).
     • Scale: multiple participants can be compromised, and multiple attackers can collude; the main experiments default to 50% malicious clients per round and 50% of data backdoored per batch.
     • Victim system and trust assumptions: the aggregation algorithm and benign clients' local training mechanisms are trusted; benign clients do not manipulate data or the training process.

   Collapse this answer

2. 02What related research exists?

   Related work falls into three groups: FL data heterogeneity, backdoor attacks, and backdoor defenses; the authors state that all three have gaps from the heterogeneity perspective.

   Data heterogeneity in FL

   • Li et al. (2019) theoretically give convergence bounds caused by heterogeneity; Sattler et al. (2019) empirically study the impact of varying heterogeneity on model performance.
   • Li et al. (2020) discuss the challenges heterogeneity brings to FL; Zhao et al. (2018) show that client local model weights diverge due to data heterogeneity.

   Backdoor attacks

   • Chen et al. (2017) propose backdoor attacks against deep learning models: implant a trigger pattern into source-class samples and relabel them as the target class, causing targeted misclassification.
   • FL-oriented attacks: Bagdasaryan et al. (2018) propose model replacement; Bhagoji et al. (2019) show that boosting model weights can help attackers; Fung, Yoon, and Beschastnikh (2018) introduce sybil attacks; Zhang et al. (2019) attack the global model with GANs; Xie et al. (2019) propose the distributed collusion attack (DBA) and design trigger patterns.
   • The authors state their work takes a different angle: it focuses on the impact of data heterogeneity on attack effectiveness, a topic that has rarely been studied.

   Backdoor defenses

   • Defenses for centralized ML: filtering (Steinhardt, Koh, and Liang 2017), fine-pruning (Liu, Dolan-Gavitt, and Garg 2018); the authors point out that they require knowledge of the training data and are infeasible in FL.
   • FL-oriented defenses: clustering-type methods (Tran, Li, and Madry 2018; Chen et al. 2018; Shen, Tople, and Saxena 2016); FoolsGold uses cosine similarity to defend against sybil (Fung, Yoon, and Beschastnikh 2018); Ma, Zhu, and Hsu (2019) use differential privacy; Sun et al. (2019) show that weak differential privacy and norm-clipping can mitigate attacks. The authors point out that none of these defenses have been evaluated for effectiveness under different degrees of data heterogeneity.

   Baselines and benchmarks: the attack method adopts the distributed attack of (Xie et al. 2019), and the appendix additionally uses the attacks of (Sun et al. 2019) and (Chen et al. 2018) (denoted Attack 1, Attack 2) to verify the robustness of the conclusions; the benchmarks are LEAF (FEMNIST, Sent140) and CIFAR10.

   Positioning: the authors state that this paper is a systematic empirical study of FL backdoor attacks and defenses from the data heterogeneity perspective, filling the gap that neither the attack nor the defense side has been evaluated under multiple degrees of heterogeneity.

   Collapse this answer

3. 03How does the paper address this problem?

   Define a Heterogeneity Index to quantify heterogeneity, systematically vary attack configurations on three benchmarks for empirical analysis, and discuss defense directions.

   Overall approach: define a quantifiable heterogeneity metric, systematically vary the degree of heterogeneity, malicious data distribution, attack scale/budget and local attack timing on LEAF and CIFAR10, characterize the effects with ASR and cosine similarity, and finally discuss defense mechanisms for the three curses found.

   Control and quantification of heterogeneity

   • Heterogeneity Index (HI): HI(c) = 1 − 1/(C_max−1) × (c−1), where c controls the maximum number of classes per client and Cmax is the total number of classes in the dataset; values are normalized to 0–1, with 1 indicating the highest heterogeneity.
   • Heterogeneity simulation: the main experiments limit the number of classes per client following (Zhao et al. 2018; Fung, Yoon, and Beschastnikh 2018); the appendix verifies with Gaussian sampling (smaller variance means higher heterogeneity) and Dirichlet distribution, and the authors state the results are consistent with HI.

   Experimental and attack setup

   • Benchmarks and models: FEMNIST (62-class handwritten characters, 2 conv 2 dense CNN, 49,644/4,964 train/test split, 179 clients, 17 per round, 2000 rounds total); Sent140 (tweet binary sentiment analysis, 100 cell lstm 2 dense, 6,553/655, 50 clients, 10 per round, 10 rounds total); CIFAR10 (4 conv 2 dense, 50,000/10,000, split into 200 clients, 20 per round, 500 rounds total) (Table 1).
   • Attack method: the distributed backdoor attack of (Xie et al. 2019) is adopted; a configured number of malicious clients are randomly selected and a trigger pattern is injected into their data; the learning hyperparameters of malicious and benign clients are kept consistent.
   • Success criterion: the trigger is applied to 50% of the test data, and the attack is recorded as successful if the global model classifies it as the target class; ASR is the success proportion, computed excluding data points that already belong to the target class.

   Analysis dimensions

   • Redemption series: fixing other parameters (50% malicious clients per round, 50% of data backdoored per batch), vary HI from 0 to 1, repeat 10 times for each HI with different malicious data distributions, and report ASR as box plots (Figure 2); fixing HI (FEMNIST 0.2, Sent140 0.0, CIFAR10 0.5), compare the average ASR of 25 malicious data distributions over 20 rounds of attack (Figure 3); plot heatmaps over the triple of (attack scale, total attack budget, malicious data distribution) (Figure 4).
   • Curse series: compare four local attack timings (uniformly across 10 batches, first 5, middle 5, last 5 batches), with the same total budget (the uniform strategy backdoors 10% of data per batch, the others 20%); combine global and local attack timing; compute the cosine similarity of each client's last dense layer weights relative to the previous round's global model; measure the distance between the malicious data distribution and the overall training distribution with the Chi-squared distance χ² = ∑_i=1^c ((O_i − E_i)²)/E_i (Ei is the frequency of class i in the training set, Oi is its frequency in the malicious dataset), and perform linear regression on the ASR–ChiSq scatter.

   Defense directions (proof of concept)

   • Curse 1: the aggregator assumes all clients are malicious, maintains a small global IID dataset (10% of the total data), and uses it to train the update weights of all participating clients before aggregation, inspired by (Zhao et al. 2018).
   • Curse 2: spread out client selection so that the same client must wait at least several rounds (e.g., 20 rounds) before being selected again, so that the backdoor is forgotten over time.
   • Curse 3: avoid leaking the global data distribution, or disclose a fabricated distribution to the attacker; GAN-type data anonymization can also be used.

   Collapse this answer

4. 04What experiments did the paper conduct?

   Systematically vary heterogeneity and attack configurations on three benchmarks; main results are mostly presented as figures, defenses have quantitative tables.

   Experimental setup

   • Models tested: CNN with 2 conv 2 dense on FEMNIST; LSTM with 100 cell lstm 2 dense on Sent140; CNN with 4 conv 2 dense on CIFAR10 (Table 1).
   • Datasets: LEAF's FEMNIST (62 classes, 49,644/4,964 train/test) and Sent140 (2 classes, 6,553/655), CIFAR10 (50,000/10,000, split into 200 clients) (Table 1).
   • Attack methods: the main text uses the distributed backdoor attack of (Xie et al. 2019); the appendix uses the attacks of (Sun et al. 2019) and (Chen et al. 2018) (Attack 1, Attack 2) for robustness validation.
   • Metrics: ASR (the proportion of targeted misclassification after applying the trigger on 50% of test data, excluding samples originally of the target class); the defense section additionally has ACC; cosine similarity is used for skewed-feature analysis.
   • Number of repetitions: in Redemption 1, each HI is repeated 10 times (different malicious data distributions); Redemption 2 is the average ASR of 25 distributions over 20 rounds of attack; the paper does not explain the number of repetitions for the remaining experiments one by one.
   • Default attack configuration: 50% malicious clients per round, 50% of data backdoored per batch (Redemption 1); in the attack timing experiments, the uniform strategy is 10% per batch and the other strategies are 20%.

   Main results

   The paper's main results are mainly presented as figures (box plots, bar charts, heatmaps), and the main text does not give specific numerical values; the table below summarizes the results given as numerical values:

   Swipe horizontally to read the table

   Experiment	Condition	Result	Source
   Active defense (Curse 1)	CIFAR10 / FEMNIST / Sent140	ASR 0.76→0.26 / 0.96→0.21 / 1.0→0.36	Table 2
   Client selection dispersion (Curse 2)	separation factor 10→20→50	FEMNIST 0.87→0.16→0.14; CIFAR10 0.83→0.23→0.11; Sent140 1.0→0.2→0.2	Table 3
   Faking the global distribution (Curse 3)	ChiSq 0.0→0.5→0.7→0.9	FEMNIST 0.91→0.45→0.33→0.24; CIFAR10 0.84→0.33→0.23→0.18; Sent140 1.0→0.2→0.1→0.1	Table 4

   • The authors state that Figure 2 shows that as HI increases, median ASR decreases, but the box whiskers widen, i.e., attack effectiveness is more unstable.
   • The authors state that the heatmaps in Figure 4 show no clear pattern on any of the three benchmarks, contrary to the existing conclusion that "larger attack scale and budget make attacks more effective"; the malicious data distribution is the dominant factor.
   • The authors state that the active defense in Table 2 reduces ASR; Tables 3 and 4 show that both defense directions reduce ASR as the parameter increases.

   Local attack timing (Curse 1)

   • What was tested: on FEMNIST (HI=0.2), four local attack timings were compared, and global/local timings were combined (Figure 5); the uniform and last strategies were compared under different HI (Figure 6).
   • Results: the authors state that the last 5 strategy has the highest ASR; local timing determines ASR while global timing has little effect; the total attack budget of the LATTER(last) strategy is only 0.2% of total training data, which the authors state is an order of magnitude lower than in the literature; the last strategy is more robust under different heterogeneity, and its advantage over the uniform strategy increases with heterogeneity.
   • Authors' interpretation: attacking later makes backdoor features easier to overfit, while attacking earlier makes them easier to forget; the authors call this a new vulnerability under data heterogeneity.

   Failure of skewed-feature defenses (Curse 2)

   • What was tested: using the last attack timing, the cosine similarity of the last dense layer weights (relative to the previous round's global model) of benign and malicious clients was compared under different HI (Figure 7).
   • Results: the authors state that the higher the HI, the larger the weight differences among benign clients (the lower the cosine similarity), which may even exceed the differences of backdoor data, allowing malicious data to evade detection based on skewed features.
   • Authors' interpretation: data heterogeneity makes benign clients also exhibit overfitting features, which attackers can exploit for camouflage.

   Malicious data distribution as a lever (Curse 3)

   • What was tested: following the setup of Redemption 2, scatter plots and linear regression of ASR against ChiSq distance (Figure 8); heatmaps of (budget, ChiSq) and (scale, ChiSq) (Figure 9).
   • Results: the authors state that ASR correlates well with ChiSq, and points cluster more as ChiSq decreases; attacks with low ChiSq have better overall ASR, and can even exceed attacks with higher budget but also higher ChiSq.
   • Authors' interpretation: the authors state that this is contrary to the finding in Figure 4, indicating that existing FL robustness work has not been adequately evaluated under stronger attacks; divergence metrics such as KL, JS, Wasserstein, and B-Distance are also usable.

   Other ablations and analyses

   • Gaussian sampling heterogeneity: Appendix Figures 10 and 11 reproduce the observations of Figures 2 and 7; the authors state that the conclusions are consistent across different ways of simulating heterogeneity.
   • Weight scaling factor: under scaling factors 1, 5, and 10, ASR rises only slightly while ACC decreases (Figure 12); the authors state that the backdoor conclusions can be generalized to model replacement, but scaled weights are easy to detect, and the main text focuses on the case with factor 1.
   • Other attack strategies: the heatmaps of Attack 1 and Attack 2 in Figure 13 likewise show no clear pattern (Sent140 is not included because both strategies target image applications).
   • Client selection dispersion defense: the authors report that this strategy keeps ASR at no more than 23% across all heterogeneity levels.
   • Fake distribution defense: the main text reports that average ASR drops from 0.8 to 0.46 (ChiSq between the fake distribution and the true distribution is about 0.8).

   Collapse this answer

5. 05What are some points that can be further explored?

   The authors point out limitations such as the compatibility of active defenses with secure aggregation, and plan to study more complex detection methods.

   Limitations and future directions noted by the authors

   • Applicability limits of active defense: The Defending Curse 1 section notes that if secure aggregation is used, it may be difficult to train a single client on an IID dataset.
   • More complex detection methods left for future work: The Defending Curse 2 section states that the authors plan in future work to study more complex detection methods such as activation clustering, spectral signatures, and gradient shaping, possibly combined with client selection mechanisms.
   • Alternative means of protecting the global distribution: The Defending Curse 3 section proposes that when the global distribution cannot be hidden, one can try to mislead attackers into believing a wrong distribution, or design more robust aggregation methods through GAN-like data anonymization (mainly a directional discussion).
   • The paper has no separate Limitations section; the above are the limitations and plans explicitly written by the authors in the defense discussion.

   Scope of experiments

   • Three models were tested: CNN on FEMNIST (2 conv 2 dense), LSTM on Sent140 (100 cell lstm 2 dense), CNN on CIFAR10 (4 conv 2 dense) (Table 1).
   • Three attack methods were covered: (Xie et al. 2019) as the main one; (Sun et al. 2019) and (Chen et al. 2018) were validated only in Appendix Figure 13 and do not include Sent140.
   • Three ways of simulating heterogeneity were covered: limiting the number of classes per client (main text), Gaussian sampling, and Dirichlet distribution (appendix).
   • Defense evaluation is a proof of concept: active defense (Table 2), client selection dispersion (Table 3), faking the global distribution (Table 4); the paper does not report the impact of these defenses on main-task accuracy (ACC appears only in the weight scaling experiment, Figure 12).
   • The paper does not report specific numerical values for the experiments in Figures 2–9 (presented as figures), nor does it explain one by one the number of repetitions for each experiment other than Redemption 1 (10 times) and Redemption 2 (25 distributions × 20 rounds).

   Collapse this answer

6. 06Summarize the main content of the paper

   An empirical study of the impact of data heterogeneity on FL backdoor attacks: both a redemption and even more a curse, with defense directions given.

   An empirical analysis paper on federated learning backdoor attacks, studying how data heterogeneity, an inherent FL characteristic, affects attack effectiveness and defense.

   • Problem: most existing FL backdoor attack and defense studies assume IID or only conduct simplified non-IID studies; the authors state that heterogeneity has been overlooked from the robustness perspective.
   • Method: define a Heterogeneity Index to quantify heterogeneity, conduct extensive experiments on LEAF (FEMNIST, Sent140) and CIFAR10 with the distributed backdoor attack of (Xie et al. 2019), systematically varying the degree of heterogeneity, malicious data distribution, attack scale/budget and local attack timing.
   • Redemption: the authors report that as HI increases, the median ASR decreases but becomes more unstable (Figure 2); the malicious data distribution is the dominant factor, the heatmaps show no clear pattern (Figure 4), and attack strategies are harder to design.
   • Curse: when attacking only in the last local batch, the budget is only 0.2% of the total training data and is robust across different heterogeneity (Figure 5, 6); heterogeneity increases the weight differences among benign clients, causing the cosine similarity-based skew-feature defense to fail (Figure 7); the ChiSq distance correlates with ASR, which attackers can use to optimize the distribution (Figure 8, 9).
   • Defense (proof of concept): the aggregator retrains client updates with a small IID dataset, reducing ASR from 0.96 to 0.21 on FEMNIST and from 1.0 to 0.36 on Sent140 (Table 2); increasing the client selection interval from 10 to 50 rounds reduces ASR from 0.87 to 0.14 on FEMNIST (Table 3); fabricating the global distribution (ChiSq 0.9) reduces ASR from 0.91 to 0.24 on FEMNIST (Table 4).
   • Authors' conclusion: the impact of data heterogeneity depends on its degree, and backdoor attack effectiveness can therefore vary greatly; these empirical findings can provide new perspectives for designing FL defenses.

   Collapse this answer

Related papers