Skip to content

Sharing Only Representations: A Provable Improvement to the Privacy-Utility Trade-off in Federated Learning

Share Your Representation Only: Guaranteed Improvement of the Privacy-Utility Tradeoff in Federated Learning

arXiv· arXiv:2309.05505· Zebang Shen, Jiayuan Ye, Anmin Kang, Hamed Hassani, Reza ShokriETH Zürich; National University of Singapore; University of PennsylvaniaIndexed by this site · 23 days ago· Originally publishedDefenses

Quick read

Defenses

Based on the paper PDF (AI-generated); refer to the original

The authors propose a federated learning method, CENTAUR: it shares representations only under differential privacy, with the classification head kept locally private; the authors state that it outperforms DP baselines on tasks such as CIFAR10.

Threat modelProtect each user's privacy against other malicious clients (an eavesdropper cannot determine whether a given user participated in training). Adopt user-level RDP and the Billboard model: a trusted server aggregates information under DP constraints and broadcasts public messages, and users compute personalized models from the public messages and their local private data.

Questions
Repeatedly sharing parameters in federated learning leaks private data; under data heterogeneity, using DP to protect the entire model loses utility due to gradient clipping and noise addition. The authors focus on how to balance privacy and utility under user-level DP.
Methods
Propose CENTAUR: split the model into a shared representation and local classification heads, perform DP training (clipping + Gaussian noise) only on the representation, and keep the classification heads local and unreleased; in the linear representation case, initialize with a private power method with cross-validation, and provide convergence and privacy analyses.
Experiments and results
Under multiple heterogeneous settings on CIFAR10/CIFAR100/EMNIST with ε=1, CENTAUR's test accuracy was higher than DP-FedAvg-fb, PMTL-ft, PPSGD, and local independent training (Table 1); the authors claim that in the linear case the privacy-utility tradeoff improves over SOTA by Θ(√d).
Limitations and future work
The theoretical guarantees are limited to linear representations and linear classification heads, and assume zero-mean, 1-subgaussian data, etc.; the experiments cover only three image datasets and small LeNet/MLP models; the authors state that when the DP budget is too small, no utility guarantee can be established (Appendix J); the paper does not have a dedicated Limitations section.
Experimental setupLeNet-5、MLP · CIFAR10、CIFAR100 et al. · testing accuracy、dist(B, B*)(主角度距离) et al.
Evaluated models
LeNet-5MLP
Benchmarks
CIFAR10CIFAR100EMNIST
Metrics
testing accuracydist(B, B*)(主角度距离)(ε, δ)-DP / user-level RDP
AI summaryThe representation federated learning objective lets parties collaboratively optimize a model consensus representation under differential privacy, while the local personalized part is kept and not uploaded. DPFEDREP converges linearly under the linear representation setting for non-convex objectives to a ball near the global optimum, with radius proportional to the inverse of the privacy budget, and improves the privacy-utility trade-off by sqrt(d); on CIFAR10, CIFAR100, and EMNIST image classification, it significantly outperforms prior work under the same small privacy budget.

The representation federated learning objective lets parties collaboratively optimize a model consensus representation under differential privacy, while the local personalized part is kept and not uploaded. DPFEDREP converges linearly under the linear representation setting for non-convex objectives to a ball near the global optimum, with radius proportional to the inverse of the privacy budget, and improves the privacy-utility trade-off by sqrt(d); on CIFAR10, CIFAR100, and EMNIST image classification, it significantly outperforms prior work under the same small privacy budget.

Paper analysis

6 questions; open one to read the full answer

  1. What problem does this paper attempt to solve?

    The authors address the problem of poor utility of DP federated learning under data heterogeneity and propose CENTAUR, which shares only the representation.

    Core problem: in federated learning with heterogeneous data, how to use differential privacy (DP) to protect user data while avoiding the utility loss caused by gradient clipping and noise addition.

    • Problem setting and importance: in federated learning (FL), multiple parties jointly train under server coordination with data kept local, but the authors point out that sharing only partial information such as gradients still leaks privacy—an adversary can infer whether a record is in another party's training set (Nasr et al., 2019), or even reconstruct training data (Zhu et al., 2019), so DP guarantees are needed.
    • Shortcomings of existing methods: the authors argue that data heterogeneity makes the optimal models of clients differ greatly, and local gradients are large even at the global optimum; DP requires clipping gradients with a small threshold and then adding noise, which introduces non-vanishing bias and harms model accuracy (Remark 3.1).
    • Core observation: the authors observe that in many FL classification tasks, the parties' data representations differ little, but the classification heads (e.g., the last layer of the network) may differ greatly (citing Bengio et al., 2013; Chen et al., 2020; Collins et al., 2021).
    • Method proposed in the paper: propose CENTAUR, which splits the model into a representation extractor and classification heads: only a globally shared representation is jointly trained with DP, and each client keeps a private personalized classification head; and provide convergence and privacy-utility trade-off analyses in the linear representation setting.
    • Threat model:
      • Adversary's goal: infer whether a given user participated in the collaborative training process.
      • Adversary's identity: other potentially malicious clients (eavesdroppers) who can see the information published during training.
      • Protected object and guarantee: user-level RDP, for the shared representation bTg published in Algorithm 1; if the personalized models are also published, this is equivalent to joint-(R)DP.
      • Communication model: Billboard model—a trusted server (or cryptographic techniques such as multi-party computation) aggregates information under DP constraints and broadcasts public messages to all n users.
  2. What related research exists?

    Related work falls into three categories: DP federated learning, model personalization, and the closest, Jain et al. (2021) private representation learning.

    DP federated learning

    • Shokri & Shmatikov (2015), McMahan et al. (2017a)—pioneering lines of research combining DP and federated learning.
    • Duchi et al. (2014), Erlingsson et al. (2014), Ding et al. (2017)—local DP models; the paper points out that local DP has a worse privacy-accuracy trade-off than centralized DP, and even learning simple statistics requires large amounts of data.
    • McMahan et al. (2017b), Agarwal et al. (2018), Levy et al. (2021), Kairouz et al. (2021)—achieve user-level DP under the Billboard model with the help of secure aggregation; this paper adopts the same setting.

    Model personalization

    • Smith et al. (2017), Fallah et al. (2020), Li et al. (2020b), Arivazhagan et al. (2019), Collins et al. (2021), Pillutla et al. (2022)—let each client learn a different but related model, mitigating model drift caused by data heterogeneity.
    • Hu et al. (2021)—private federated multi-task learning (PMTL), adding a task-related regularizer for each client; the paper argues that regularization has limited ability to handle data heterogeneity.
    • Bietti et al. (2022)—PPSGD, an additive personalized model supporting user-level DP; the paper points out that its generalization guarantee relies on convexity of the loss function, whereas this paper analyzes convergence under nonconvex objectives.

    Closest work

    • Jain et al. (2021)—also learns a shared low-dimensional linear representation plus personalized classification heads with DP; the paper points out that it requires exactly solving least squares on the server side via SVD of the noisy representation matrix, its generalization guarantee has a dimension dependence of d^1.5, and it is limited to linear representations; this paper instead uses server-side noisy gradient descent, which the authors state improves the dimension dependence by Θ(√d) and supports multi-layer shared representations.
    • Collins et al. (2021)—FedRep, on which this paper's method is directly built, but FedRep does not consider privacy.
    • Xu et al. (2023)—a concurrent work known at submission time, using a similar algorithm to learn image embeddings with user-level DP in an extreme many-class setting.

    Baselines and benchmarks: the experiments compare Stand-alone-no-FL (local independent training, no DP), DP-FedAvg-fb (Yu et al., 2020 with local fine-tuning), PPSGD, and PMTL-ft; the datasets are CIFAR10, CIFAR100, EMNIST.

    The authors position this paper as: under user-level DP in the Billboard model, adding strict privacy guarantees to the FedRep-style shared representation framework, and theoretically improving the privacy-utility trade-off in the linear representation case by Θ(√d) over Jain et al. (2021).

  3. How does the paper address this problem?

    CENTAUR performs DP training only on the shared representation and keeps the classification heads locally private; in the linear case, it initializes with a private power method enhanced by cross-validation.

    Overall idea: decompose the model into a representation extractor ϕ(·; b) and a classification head h(·; w), jointly train and release only the global representation b with DP guarantees, and keep each client's classification head w local and unreleased, thereby avoiding the large gradients and clipping bias caused by heterogeneity concentrated in the head. The method is named CENTAUR.

    Problem formulation

    • The local objective is the standard empirical loss (Equation 1); standard FL seeks a globally consistent solution (Equation 2), and the authors argue that under heterogeneity the global solution is far from the local optima.
    • Federated representation learning (FRL) objective (Equation 3): minb∈ℬ 1/n∑i minwi l([wi, b]; Si), i.e., share one global representation b, and each client separately optimizes its private head w_i.
    • The authors' rationale: when representations are approximately homogeneous, the local and global optima are close, and the representation gradient tends to zero near the optimum, accommodating a small clipping threshold (Remark 3.1); and since k≪d, the head is easy to optimize once the representation is fixed.

    General algorithm (Algorithm 1, 2)

    • SERVER: each round selects active clients by Poisson sampling (parameter pg), broadcasts the current representation bt, collects local update directions and aggregates them via the Gaussian mechanism GM (clipping threshold ζg, noise multiplier σg), and updates with step size ηg; for Tg rounds in total.
    • CLIENT three stages: 1) fix bt and optimize the local classification head exactly (or approximately) to the minimizer of the local objective; 2) starting from bt, perform Tl steps of local SGD to update the local representation (mini-batch size m̄, step size ηl); 3) return the local update direction git = bt,Tl_i − bt.
    • Privacy guarantee (Remark 4.1): the authors state that by the composition theorem for the subsampled Gaussian mechanism, the algorithm satisfies user-level (α, ϵ)-RDP with ϵ = Tg·Sα(pg, σg); when pg=1, ϵ = α·Tg/(2σg²).

    Linear representation (LRL) instance

    • Setting: yij = wi*ᵀB*ᵀxij, where B* is a d×k column-orthogonal representation matrix and w_i* is the client's private linear head; the feasible region is the set of column-orthogonal matrices; the loss is least squares (Equation 4). Problem (L-FRL) remains nonconvex in B.
    • Initialization (Algorithm 3): run T0 independent private power methods (PPM, adapted from Hardt & Price, 2014) to obtain candidates, then use cross-validation to select Bĉ that agrees with at least half of the candidates; the authors state that a single PPM succeeds only with constant probability (0.99), and this scheme raises the success probability to 1−n^{−k}, and uses only the outputs of the individual PPM runs, is post-processing, and leaks no additional privacy (Lemma F.3).
    • Aggregation and client modifications: AGGREGATION uses QR decomposition to maintain column orthogonality; the client instead draws two subsets without replacement, used respectively for updating the head and computing the representation gradient, which the authors state is needed to make clipping not trigger with high probability (Lemma C.1).

    Theoretical results

    • Utility (Theorem 5.1): under conditions including initialization with dist(B⁰,B*)≤0.2, sufficiently large m̄ and sufficiently small σg, the authors prove dist(B^{Tg}, B*) ≤ c̃d·σg·µ²k^1.5·d/n, i.e., convergence at a linear rate to a neighborhood centered at B* with radius proportional to the noise multiplier.
    • Privacy (Theorem 5.2): this instance is (α, ϵinit+ϵrdp/2)-RDP; when σg is sufficiently small it is (α, ϵrdp)-RDP, which converts to (ϵdp, δ)-DP.
    • Trade-off (Corollary 5.1): the authors state ϵa·ϵdp ≥ Θ(d/n), an improvement of Θ(√d) over Θ(d^1.5/n) in Jain et al. (2021).
  4. What experiments did the paper conduct?

    Under multiple heterogeneous settings on CIFAR10/100 and EMNIST, with ε=1, CENTAUR's accuracy is higher than all four baselines (Table 1).

    Experimental Setup

    • Tasks and data: Image classification, datasets CIFAR10, CIFAR100, EMNIST; split according to the scheme of Collins et al. (2021): train/validation 90%/10%, the training portion evenly divided among n clients, each client with at most S classes.
    • Model architecture: CIFAR10/100 use LeNet-5 (two 64-channel convolutional layers + two fully connected layers, hidden neurons (384,32) for CIFAR10 and (128,32) for CIFAR100); EMNIST uses a three-layer MLP (256,128,16); ReLU, no BN/dropout (Appendix B).
    • Baselines: Stand-alone-no-FL (local independent training, no DP), DP-FedAvg-fb, PMTL-ft, PPSGD.
    • Privacy settings: δ=10⁻⁵; Table 1 and 3 fix ϵdp=1; given ϵdp, σg is set according to Tg using Opacus's privacy engine; full participation (pc=1).
    • Hyperparameters: CIFAR10: ζg=0.01, ηl=0.01, ηg=1, Tg=200; CIFAR100: ζg=0.02, Tg=100; EMNIST: ζg=0.25, Tg=40; the trade-off curves in Figure 1 use a grid search over ζg in {0.01,0.02,0.04,0.06} (Table 2).
    • Number of repetitions: Values in parentheses in Table 1 are standard deviations; the Figure 1 caption notes that error bars are the std of 3 runs; data augmentation is used only for fine-tuning CENTAUR's local head.

    Main Results

    Table 1: Test accuracy (%) under ϵdp=1, δ=10⁻⁵, with standard deviations in parentheses.

    Swipe horizontally to read the table

    SettingStand-alone-no-FLDP-FedAvg-fbPMTL-ftPPSGDCENTAUR
    CIFAR10, n=500, S=550.03 (1.59)46.17 (0.12)49.35 (0.44)48.69 (1.44)53.23 (0.92)
    CIFAR10, n=1000, S=274.06 (0.45)57.02 (1.36)67.79 (1.77)71.52 (2.30)76.23 (0.48)
    CIFAR10, n=1000, S=544.60 (1.30)37.15 (1.22)35.99 (1.79)44.61 (2.68)49.92 (0.71)
    CIFAR100, n=1000, S=539.20 (1.12)22.17 (2.12)24.17 (1.43)32.97 (1.48)44.54 (1.05)
    EMNIST, n=1000, S=593.47 (0.14)91.32 (0.22)92.32 (0.22)93.44 (0.20)94.17 (0.19)
    EMNIST, n=2000, S=590.67 (0.46)86.85 (1.11)88.81 (2.08)88.96 (1.93)92.79 (0.25)
    • The authors state that under this small privacy budget CENTAUR outperforms all baselines, and is the only method that consistently exceeds the local independent training baseline, on this basis arguing that collaborative training relative to local training is reasonable.
    • Figure 1 (CIFAR10, n=500, S=5, ϵ∈{0.25,0.5,1,2,4}): the authors state that CENTAUR consistently outperforms other methods at each ϵ, and already exceeds Stand-alone-no-FL at ϵ=0.5; one place in the main text writes that this figure corresponds to EMNIST (n=2000, S=5), the two statements are inconsistent, and the caption (CIFAR10) prevails.

    Larger Communication Budget

    • Table 3 (CIFAR10, Tg=400, ϵdp=1): with n=1000, S=2, CENTAUR 77.80 (0.52), PPSGD 74.63, PMTL-ft 67.71, DP-FedAvg-fb 63.97, Stand-alone 74.06; with n=1000, S=5, CENTAUR 51.05 (0.35), PPSGD 48.29, PMTL-ft 45.75, DP-FedAvg-fb 41.12.
    • The authors state that CENTAUR remains consistently best under a larger communication budget.

    Training Dynamics and Stability

    • Figure 3 (CIFAR10, n=1000, S=2 and S=5): the horizontal axis is ϵdp during training, the vertical axis is test accuracy; the authors state that CENTAUR converges faster to high utility and consistently outperforms the baselines.
    • Figure 4: CENTAUR performs stably under clipping thresholds {0.02,0.05,0.10} and communication rounds {100,300,600}; the authors also note that DP-FedAvg-fb is unstable to hyperparameters and even diverges, so no full hyperparameter search was performed (Appendix B, footnote 4).

    Mechanism Analysis: Gradient Consistency

    • Method: compute the cosine similarity between each user's gradient and the average gradient of all users (before clipping), and report the mean and standard deviation (Figure 5).
    • Results: the authors state that in DP-FedAvg-fb users diverge greatly in update direction (low mean, high standard deviation), especially in the classification head; in CENTAUR users are more consistent in the update direction of the representation layer (higher mean, lower standard deviation).
    • Exception: the authors note that cosine similarity still declines in the later stage of training, and believe that residual data heterogeneity may still exist at the representation level (footnote 5).

    Other Ablations and Analyses

    • Data augmentation: the authors state that classic augmentation reduces accuracy under DP training (consistent with the observation of De et al., 2022), and the new augmentation proposed by De et al. also does not consistently improve validation accuracy under this setting; augmentation is effective only when used for fine-tuning CENTAUR's local head, while for fine-tuning other baselines it is instead worse (Appendix B).
  5. What are some points that can be further explored?

    The authors did not include a Limitations section; the theory is limited to linear representations, and the experiments are limited to three image datasets and small models.

    Limitations and future directions noted by the authors

    • DP budget lower bound: Appendix J points out that for a target accuracy ϵa, when ϵdp ≤ c̃t·κk^1.5·n⁻¹·ϵa⁻¹·µ²d, no theoretical guarantee can be established that CENTAUR reaches ϵa within that budget; the authors also emphasize that this does not rule out achieving it in practice, because the privacy analysis is only an upper bound, not a lower bound.
    • Client count requirement: Appendix J points out that reaching ϵa utility within the ϵdp budget is guaranteed only when n is sufficiently large (n ≥ c̃t·κk^1.5·µ²d/(ϵa·ϵdp)).
    • Scope of the theory: Section 5 explicitly states that the utility-privacy trade-off analysis applies only to the instance (LRL) where both the representation and the classification head are linear, and relies on Assumptions 5.1–5.3 (zero-mean 1-subgaussian inputs, incoherence parameter, lower bound on local sample size); the PPM initialization analysis additionally requires xij to be Gaussian variables (Appendix F.1).
    • Insufficient hyperparameter search: footnote 4 of Appendix B states that due to computational constraints, no exhaustive search was performed over the number of communication rounds Tg, otherwise each method (especially DP-FedAvg-fb) might achieve higher utility.
    • The paper does not have a dedicated Limitations subsection; the above are the conditional limitations explicitly stated in the main text and appendices.

    Scope of the experiments

    • The objects evaluated are methods rather than models: CENTAUR and 4 baselines (Stand-alone-no-FL, DP-FedAvg-fb, PMTL-ft, PPSGD), with backbone architectures LeNet-5 (CIFAR10/100) and a three-layer MLP (EMNIST) (Appendix B).
    • The datasets are three image classification datasets: CIFAR10, CIFAR100, EMNIST; the heterogeneous settings are six combinations of n∈{500,1000,2000} and S∈{2,5} (Table 1).
    • Privacy budget: Table 1 and Table 3 fix ϵdp=1 and δ=10⁻⁵; Figure 1 covers ϵ∈{0.25,0.5,1,2,4} (CIFAR10, n=500, S=5).
    • Number of repetitions: the Figure 1 caption notes standard deviation over 3 runs; Table 1 reports standard deviations but the paper does not state the number of runs per setting.
    • The paper does not report experiments of CENTAUR on non-image tasks such as text or on larger backbones; the theoretical instance (Algorithm 3–5) is not separately validated numerically.
  6. Summarize the main content of the paper

    CENTAUR shares only the representation with DP and keeps the classification heads locally private; the authors state that both theory and experiments outperform prior methods.

    A methods and theory paper on differentially private federated learning (ICLR 2023).

    • Problem addressed: under data heterogeneity, DP training of the entire model incurs utility loss due to gradient clipping and noise addition; the authors argue that the bottleneck lies in the conflict between learning the representation and the classification head, and that heterogeneity is mainly concentrated in the classification head.
    • Method highlights: CENTAUR splits the model into a shared representation and local classification heads, performs user-level DP (Billboard model) training only on the representation, and does not release the classification heads; in the linear representation case, initialization uses a private power method enhanced with cross-validation, and QR decomposition maintains column orthogonality.
    • Theoretical results: the authors prove that in the linear representation case (still nonconvex) CENTAUR converges at a linear rate to a neighborhood of the global optimum, with trade-off ϵa·ϵdp ≥ Θ(d/n), which the authors state improves Θ(√d) over Θ(d^1.5/n) in Jain et al. (2021).
    • Experimental results: with ϵdp=1 and δ=10⁻⁵, CENTAUR is highest on all six heterogeneous settings (Table 1), e.g., 53.23% on CIFAR10 n=500,S=5 (baseline highest 50.03%), 44.54% on CIFAR100 n=1000,S=5 (baseline highest 39.20%), and 92.79% on EMNIST n=2000,S=5; the authors state it is the only method that consistently exceeds local independent training.
    • Mechanistic evidence: in the cosine similarity experiment (Figure 5), the authors state that under CENTAUR the clients' representation update directions are more consistent, in line with the analysis of Remark 3.1.
    • Authors' conclusion: statistical heterogeneity brings additional difficulties to DP federated learning, and sharing only the representation can mitigate this; the theoretical guarantees are limited to the linear setting, and the guarantees do not hold when ϵdp is too small or the number of clients is insufficient (Appendix J).

Related papers

Show 3 more papers
Read originalarxiv.org