LightSplit: Practical Privacy-Preserving Split Learning Based on Orthogonal Projection

Core problem: the cut layer in split learning (SL) causes both a communication bottleneck and privacy leakage; the authors aim to design a mechanism that simultaneously reduces the per-sample transmission volume, does not increase client overhead, and limits the information the server can reconstruct.

• Problem setting: SL splits a DNN into a client head and a server backbone, and each training step requires transmitting d-dimensional smashed data and d-dimensional gradients. The authors give figures: ResNet-18 on CIFAR-10 with 5 clients produces more than 150 GB of traffic in one training round (Table I).
• Why it matters: smashed data is an information-rich intermediate representation, and an honest-but-curious server can use it to reconstruct the input (e.g., SDAR attack); the authors say this undermines SL's privacy premise.
• Shortcomings of existing methods (according to the paper): privacy defenses (NoPeek, Shredder, DISCO) perturb activations but do not compress; compression methods each have flaws—top-k exposes the most active positions per sample, learned codecs require adding a trainable encoder on the client and external pretraining, and batch-wise compression does not reduce per-sample payload; cryptographic mechanisms cost several orders of magnitude more.
• Core observation: a fixed orthogonal random projection based on the Johnson-Lindenstrauss lemma both compresses dimensions and acts as an information bottleneck that irreversibly discards information in the orthogonal complement space; combined with within-class compaction regularization, it can make representations more like class prototypes than individual descriptors.
• Threat model:
  • Attacker goal: reconstruct the client's private input samples.
  • Knowledge: the server knows the projection matrix R (public), its own backbone parameters, the received compressed activations z̃, and the returned gradients.
  • Capability: honest-but-curious (semi-honest), faithfully executing the protocol but passively inferring; the paper additionally stress-tests with the active attack FSHA.
  • Victim system: U-shaped SL, where the client holds the head and tail, and labels do not leave the client.

Cut-layer privacy attacks

• Passive reconstruction: DLG/iDLG/Inverting Gradients recover data from gradients; UnSplit (2022) alternately optimizes a cloned encoder and candidate inputs without auxiliary data; PCAT (2023) removes architecture assumptions; FORA (2024) trains a substitute encoder on public data; SDAR (2025) is the current SOTA passive attack, training a simulator and exploiting the backbone's memorization effect, with reconstruction quality comparable to the active attack FSHA.
• Active reconstruction and detection: FSHA (2021) hijacks the client encoder by manipulating gradients; the paper states that such protocol deviations can be detected by SplitGuard (2022) and similar, so passive reconstruction is the main practical threat.
• Label leakage: cut-layer gradients can leak labels (Li et al., 2022), and similarity clustering can perform label inference (Liu et al., 2024); the paper explicitly excludes label leakage from scope.

Existing defenses

• Representation perturbation: NoPeek (2020) uses a distance correlation penalty to reduce input-activation correlation; Shredder (2020) injects learned additive noise; DISCO (2021) learns pruning filters to obfuscate sensitive channels; patch shuffling (2022) shuffles ViT spatial patches. The paper notes that none of these reduce communication volume, and adaptive attackers can learn privacy features from the defended representations.
• Cryptographic mechanisms: FedVS (2023) uses secret sharing to provide information-theoretic confidentiality; CryptoNets (2016) uses homomorphic encryption to process ciphertext, which the paper states has computational overhead several orders of magnitude higher and is unsuitable for edge deployment.
• Censor (2025): has clients sample multiple candidate updates and then select; the paper notes this is infeasible in SL because evaluating quality requires server participation, which would leak the projection options.

Cut-layer communication compression

• top-k sparsification (2023) and its mask-encoded variant (2024) transmit the largest activations and their positions; the paper notes this exposes the most active positions per sample.
• learned codecs (BottleNet++ 2020, FrankenSplit 2024) train encoder-decoder pairs at the cut layer, adding trainable parameters and forward overhead to the client.
• SplitFC (2025) discards low-variance features and quantizes; C3-SL (2022) superimposes a batch of features with circular convolution, reducing the number of transmissions per round but not the per-sample payload.

Random projection in distributed learning and this paper's positioning

• RanPAC (2023) projects pretrained features for continual learning; Han et al. (2024) projects local model parameters for personalized FL; ProjPert (2024) projects backward gradients for vertical FL label protection. The authors state that to their knowledge no prior work uses random projection at the SL cut layer for per-sample activation compression.
• Baselines and benchmarks: experiments compare RAW (no compression), LEARNED 1×1 (BottleNet-style channel bottleneck), top-K sparse (equal-payload sparsification) and DP (σ=0.02); benchmarks are CIFAR-10/100, FashionMNIST, GTSRB, MNIST, ImageNet-1k, UCI HAR, WikiText-103.
• Authors' positioning: among existing methods, privacy mechanisms do not compress, while compression mechanisms either leak structural information or add client overhead; LightSplit uses a fixed non-trainable orthogonal projection to address both simultaneously.

Overall idea: at the head-backbone cut of U-shaped SL, the client uses a fixed orthogonal matrix to project d-dimensional activations to k dimensions (k≪d) before transmission, and the server lift-backs to d dimensions using a fixed or learned method before feeding into the backbone; the information discarded by the projection is irreversible, and a Within-Class Compaction (WCC) regularizer further suppresses sample-level signals. The framework is named LightSplit, with two server modes: LightSplit-F (fixed lift-back) and LightSplit-L (learned MLP lift-back).

Problem setting and requirements

• System model: n clients share the server backbone g, each holds a head f and a tail h, and the cut-layer activation z = f(x) ∈ R^d is the only transmitted object; the goal is to replace z with a k-dimensional summary.
• Three requirements: R1 prevent/block reconstruction inference by a curious server; R2 maintain model utility comparable to the uncompressed baseline; R3 reduce the cut-layer communication dimension.
• Three challenges: the shared backbone requires the compression protocol to be globally consistent; compression and privacy goals are entangled (retaining the most information retains the signals used by attacks); clients are resource-constrained and cannot add trainable components.

Client-side projection

• Each client projects with a fixed orthogonal matrix R ∈ R^{d×k}: z̃ = R^⊤ z, compression ratio CR = d/k; R is obtained by thin QR decomposition of a Gaussian matrix, sampled once before training and broadcast, then fixed with no trainable parameters.
• For H×W feature maps, projection can be done in b×b blocks using the same fixed R_b per block (block-local variant).
• Privacy rationale: x → z → z̃ forms a Markov chain, and by the data processing inequality I(x; z̃) ≤ I(x; z), so the projection introduces no new information; however, the authors explicitly state that the projection itself does not eliminate leakage, and residual sample-level signals remain in z̃.

Server-side lift-back

• LightSplit-F: fixed linear lift-back ẑ = Rz̃ = RR^⊤ z, where P = RR^\top is the orthogonal projection onto range(R), and the residual lies in the null space of P and is invisible to the server; gradients are passed back exactly via ∇_z L = R∇_z̃ L, with no trainable server parameters.
• LightSplit-L: replaces the fixed mapping with a trainable MLP (Linear→BN→ReLU→Linear, hidden width m), trained jointly with g; the authors' motivation is that convolutional backbones struggle to absorb a global linear remapping, and the MLP adapter frees up g's capacity, suitable for settings where g is frozen or the task is more complex.
• The two modes have identical client cost, bandwidth and attacker view, and identical privacy properties.

Within-Class Compaction (WCC)

• Based on the mutual information chain decomposition I(x;z̃) = I(y;z̃) + I(x;z̃ | y): retain the class-discriminative term and suppress the residual sample-level information term given the label (the authors state this term upper-bounds the information available to reconstruction attacks).
• The WCC loss is the within-class scatter within a batch: for each class compute the centroid µ_c in projection space and penalize ‖z̃_i − µ_c‖²; the total objective is L = L_CE + λ_WCC · L_WCC, computed on the client using labels that do not leave the client, adding no communication.
• Ablation over λ_WCC ∈ {0, 10⁻³, 10⁻², 10⁻¹}; the authors recommend λ_WCC ≈ 10⁻³ as the operating point.

Training procedure and success criteria

• Algorithm 1/2 gives the server-side and client-side procedures: client forward f → projection → send z̃; server lift-back → g → return u; client computes L_CE and L_WCC, sends back gradients, server updates g (and ϕ), client updates f, h. Appendix M gives step-by-step forward/backward derivations showing that adding WCC does not change the cut-layer protocol messages.
• Success criteria: utility is measured by top-1 test accuracy (and GPT-2 perplexity inflation); privacy is measured by changes in SSIM/LPIPS/MSE/PSNR under attacks such as SDAR, FORA, UnSplit, with attack configurations following the settings released by each attack's original authors.

Experimental setup

• Models tested: ResNet-18 (CIFAR-10/100), ResNet-50 (ImageNet-1k), MicronNet (GTSRB), SimpleCNN (FashionMNIST), MnistNet (MNIST/F-MNIST, UnSplit victim model), MLP (UCI HAR), GPT-2 124M/355M/774M/1.5B (WikiText-103 fine-tuning); image models have two cut depths L1H/L2H, with heads split per-client (PCH) and shared (SCH).
• Datasets: CIFAR-10/100, FashionMNIST, GTSRB, MNIST, ImageNet-1k, UCI HAR (561-dimensional features, 6 classes, official subject-disjoint split), WikiText-103 (80,000 training blocks, 1,500 validation blocks).
• Baselines: RAW (no compression), LEARNED 1×1 (BottleNet-style channel bottleneck), top-K sparse (equal payload), RAW+DP and LightSplit-F+DP (σ=0.02).
• Attacks: SDAR (USL/VSL, L4/L7), FORA (pseudo and white-box reference decoder), UnSplit (split-2/4), UCI HAR reconstruction and subject re-identification probes, FSHA active attack, and a projection optimization attacker that knows R; attacks follow the original authors' released configurations.
• Metrics: utility is top-1 accuracy (∆ relative to matched RAW) and perplexity inflation ratio; privacy is SSIM↓, LPIPS↑, MSE↑, PSNR↓, R²↓, re-ID↓.
• Repetitions: UCI HAR uses 3 seeds; FSHA uses multiple seeds per reconstruction cell and averages over 20 images per seed; FORA pseudo decoder uses multiple seeds, discarding diverged runs; the remaining tables do not specify the number of seeds individually.

Main results

Summary of Table II (L2H-SCH, attack cells are LS-F, λ=0), ∆ is the change relative to RAW:

Swipe horizontally to read the table

Method	CR	CIFAR-10 IID	CIFAR-100 IID	SDAR SSIM	FORA LPIPS
RAW	1×	87.24	59.47	0.832	0.206
LightSplit-F/L	8×	−4.3%/−4.4%	−11.2%/−5.0%	−37.0%	+217.8%
LightSplit-F/L	16×	−6.0%/−3.2%	−12.8%/−8.2%	−44.2%	+212.8%
LightSplit-F/L	32×	−6.0%/−5.3%	−13.7%/−11.7%	−58.3%	+212.0%
LEARNED 1×1	32×	−3.8%	−16.9%	−19.9%	+165.0%
top-K sparse	32×	−4.8%	−10.9%	−31.8%	+101.5%

(According to Table II; FMNIST, GTSRB, HAR, Non-IID means and the UnSplit column are omitted, all HAR variants are within the 94.0% baseline noise)

• The authors state that at 32× LightSplit retains at least 94% of baseline accuracy on CIFAR-10, FashionMNIST, GTSRB and UCI HAR, 86% on CIFAR-100, and 91% on average for Non-IID.
• The authors state that even at λ=0, LightSplit's privacy gain against every image reconstruction attack is greater than both compression baselines; top-K at its best 32× setting still reduces SSIM by less than LightSplit at 8×.
• The authors state that on CIFAR-100 LS-L recovers about half of the accuracy lost by LS-F, and that complex tasks require the MLP adapter.

Passive reconstruction attack details (SDAR/FORA/UnSplit, Table III, X, XIII, XIV)

• SDAR (USL L4, 8×): LS-F λ=0 gives SSIM −37.0%; at λ=10⁻¹ the SSIM drop reaches −87.3% at 16× and −105.3% at 32× (structureless reconstruction can yield negative SSIM); LPIPS rises to +83.9% at λ=10⁻¹. The authors note that pixel metrics are not monotonic in λ (SSIM returns to −36.7% at 10⁻¹), explaining that WCC makes the decoder output class-typical images, which pixel metrics reward and LPIPS penalizes.
• FORA (8×, pseudo decoder): LS-F LPIPS +217.8%, LEARNED 1×1 only +97.7%, top-K +78.9%; on the white-box reference decoder, LS-F's ∆Raw is ×3.8 (λ=0) to ×4.9 (λ=10⁻¹).
• UnSplit (split-2, foreground-masked MSE): LS-F is +27% across all CRs and λ on MNIST, and +8% to +11% on F-MNIST; the authors state that UnSplit is the weakest attack, with the smallest gap between methods.

Active attacks, DP combination and projection optimization attacker

• FSHA (CR 8×, Table XV): LS-F raises reconstruction LPIPS to 8.4× (MNIST)/2.5× (F-MNIST) of RAW, with accuracy unchanged; LEARNED 1×1 and top-K only 1.2–1.5×; LS-F+WCC(λ=0.3) reaches ×18.3/×5.5; LS-F+DP gives the lowest MNIST SSIM of 0.342.
• DP combination (Table V): USL 4096-d cut, SSIM drops from RAW 0.572 to LS-F 0.360, DP alone 0.181, and the combination 0.139; the authors state that stacking the two defenses is more private than either alone. The DP operating point σ=0.02 retains about 97% of the unnoised accuracy.
• Attacker who knows R (Table XVI): giving the attacker the analytic lift-back zR^⊤ yields at most +0.032 SSIM more than reading z only (8×); a decoder fusing z, returned gradients g and zR^⊤ does not outperform the lift-back decoder; the authors state that R discards 89.8/95.6/98.5% of activation energy at 8/16/32×, so keeping R secret has no effect.

Other ablations and analyses

• Compression ratio sweep (Fig. 15a): CIFAR-10 still retains 89% at 512× and 55% at 4096× (k=1); CIFAR-100 is 63% at 512× and 12% at 4096×.
• ImageNet-1k (Fig. 7): with b=2 tiles, 8× retains 92% of RAW top-1, 16× 85%, 32× 71%.
• GPT-2 (Fig. 9): 124M perplexity inflation is 1.15×/1.36× at 8×/32×, dropping to 1.06×/1.11× for 1.5B.
• PCH gap (Table VIII): block-local projection narrows the gap between PCH and RAW from 10.8 points (b=8) to 4.7 points (b=2).
• Number of clients (Table XX): n=10→100, LS-F drops at most 0.67 pp on CIFAR-10 and 3.33 pp on CIFAR-100.
• Negative results: on UnSplit split-4, LS-F at λ=10⁻¹, 16×/32× causes the victim model to collapse to random accuracy in every run (marked † in Table XIV); under the PCH configuration WCC loses accuracy at every weight, with L2H-PCH dropping 46.9% at λ=10⁻¹ (Table XXI); block-local with small tiles leaks more at the 8192-d deep cut (b=1 reaches 71–86% of RAW).

Limitations and future directions noted by the authors

• Threat scope limited to smashed data (Sect. VI-B "Restriction to Attacks Analyzing Smashed Data"): white-box inference attacks that exploit shared model updates are a separate threat class, and LightSplit provides no protection against them; the shared-parameter setting is left to future work.
• No formal privacy guarantees (Sect. VI-B "Formal Guarantees"): plain LightSplit provides no formal guarantees; these can be obtained by combining with Differential Privacy (this combination is evaluated in Sect. V-B5).
• Overhead of LS-L's learnable lift-back (Sect. VI-B "Overhead through Learnable Backlifting"): the MLP adds server-side parameters and computation; in resource-constrained settings the parameter-free LS-F can be used instead.
• No defense against other cut-layer attacks (Sect. VI "Out-of-Scope Threats"): label leakage attacks and model poisoning/backdoor injection on the gradient channel are out of the defense scope.
• Backdoor detection is only a preliminary observation (Sect. VI): the authors state that class separation has a negligible effect on the server's ability to detect backdoors, but a robust interpretation of this correlation requires a more comprehensive backdoor attack evaluation in future work.
• Label leakage left to the future (Sect. VII-A): label leakage is explicitly marked as outside focus and left to future work.

Experimental coverage

• 10 model configurations were tested: ResNet-18/50, MicronNet, SimpleCNN, MnistNet, MLP, and four GPT-2 sizes (124M–1.5B), covering image, tabular and text tasks (Sect. V-A, Table IV).
• Attack evaluations include passive reconstruction (SDAR, FORA, UnSplit), tabular probes (reconstruction R², subject re-ID), active FSHA, a projection optimization decoder that knows the public R, and the DP combination (Sect. V-B4, App. C).
• Data distributions cover IID and Non-IID (Dirichlet α=0.1/0.5), head ownership covers SCH and PCH, and the number of clients n=10–100 (Table VII, XX).
• Repetitions: UCI HAR uses 3 seeds; FSHA uses multiple seeds per cell and 20 images per seed; some SDAR cells additionally run 5 attacker seeds; the paper does not report the number of seeds per configuration for the main utility table (Table VII).
• The paper does not report cross-comparison numbers for query counts or iteration budgets of each attack; runtime was measured only on a Raspberry Pi 4 GB as per-batch time (0.189 s without compression vs 0.204–0.247 s with compression, App. K).

A split learning privacy-efficiency joint defense paper, targeting the communication overhead of cut-layer smashed data and the input reconstruction threat from an honest-but-curious server.

• Problem: SL transmits d-dimensional activations per step; ResNet-18/CIFAR-10/5 clients train with over 150 GB of traffic; meanwhile passive attacks such as SDAR can reconstruct inputs from smashed data with high fidelity. The authors state that existing methods handle privacy or efficiency only separately.
• Method: The client uses a fixed orthogonal matrix R (sampled via QR decomposition, broadcast before training and then fixed) to project activations from d dimensions to k dimensions; the server uses a fixed linear mapping (LS-F) or a trainable MLP (LS-L) to lift-back to d dimensions; a WCC regularizer is added to push same-class samples in the projection space toward the class centroid, suppressing sample-level signals. R is public to the server; privacy comes from dimension discarding rather than secrecy.
• Utility results: Under 8–32× compression, CIFAR-10/FashionMNIST/GTSRB/UCI HAR retain at least 94% of baseline accuracy, CIFAR-100 86%, Non-IID 91% on average; ImageNet-1k 8× retains 92% top-1; GPT-2 124M 8× has perplexity inflation of 1.15×.
• Privacy results: SDAR (USL L4) SSIM drops from RAW 0.832 to 0.524 for LS-F λ=0 (8×), and at λ=10⁻¹, 16× the relative drop is 87.3%; under the FSHA active attack reconstruction LPIPS rises to 8.4× (MNIST); an optimization attacker who knows R gains at most +0.032 SSIM; combined with DP it drops further to 0.139 (4096-d cut).
• Exceptions: UnSplit split-4 at λ=10⁻¹ causes the victim model to collapse; under the PCH configuration WCC loss reduces accuracy, and the authors recommend using block-local projection plus DP for PCH, without WCC.
• Authors' conclusion: LightSplit lets attackers recover at most class-level prototypes rather than specific inputs, and provides no formal guarantees; it can be combined with DP; label leakage, white-box inference and gradient poisoning are out of the defense scope.