Revealing the Security Risks of Federated Learning in Real Environments: From Research to Practice

This paper builds a unified evaluation framework, TFLlib, to reproduce federated learning poisoning attacks, and finds that idealized evaluations often overestimate security risks in real deployments. The authors systematically sort out three types of mismatches between research and practice: unrealistic poisoning threat models, the absence of mixed heterogeneity, and metrics that overemphasize peak attack success rate while ignoring stability and utility cost. Empirical results show that under realistic constraints, attack performance is more dataset-dependent and unstable, and some strong attacks that perform stably in idealized federated learning lose effectiveness or cause noticeable degradation in benign task performance. Therefore, looking only at the final-round attack success rate is insufficient to assess security; effectiveness, temporal stability, and the accompanying utility loss need to be considered simultaneously. Code is available at https://github.com/xaddwell/TFLlib。