Membership Inference Attacks against Private Evolution

1. 01What problem does this paper attempt to solve?

   The authors propose a white-box membership inference attack against the Private Evolution voting mechanism, for empirically auditing the actual leakage of its DP synthetic data.

   Core question: how to empirically audit the privacy of inference-only DP synthetic data generation methods such as Private Evolution (PE), i.e., determine whether a given private instance participated in voting.

   • Scenario and importance: PE requires only API access to a pretrained foundation model to generate synthetic data under DP, and has been used in text, image, tabular, and federated settings; the authors state that Microsoft and Apple (features such as email summaries in Apple Intelligence) have deployed PE-style voting mechanisms, and the privacy of many users depends on the correctness of its theoretical guarantees.
   • Shortcomings of existing methods: the authors state that formal DP guarantees cannot replace empirical auditing of the actual attack resistance of released synthetic data, and that membership inference attacks (MIA) are a mature means of auditing SDG privacy; previously there was no MIA targeting the PE voting mechanism.
   • Core observation: in PE, each private instance votes for the synthetic candidate most similar to it, leaving a trace on that candidate's vote count; instances that did not participate leave no such trace. Iterative filtering biases surviving candidates toward private instances.
   • Method proposed in the paper: the modality-agnostic EVO-MIA, which uses only public hyperparameters and the DP-protected artifacts released each round (noisy voting histograms) to determine membership; it does not require training a shadow model; there is also an EVO-MIA-BB variant that uses only candidate survival information (Appendix D).
   • Threat model:
     • Goal: determine whether the target instance x belongs to the private dataset D (i.e., whether it participated in voting).
     • Knowledge: the public embedding model φ, mechanism hyperparameters θ and σ (derived from the public (ε, δ)), |D|, and an iid auxiliary set Ref; following Kerckhoffs's principle, all mechanism parameters are treated as public.
     • Capability: observe the synthetic candidates and noisy vote counts released each round; in the federated setting, any participating party or aggregation server can record the full interaction; in the centralized setting, the foundation model provider is a natural instantiation.
     • Victim system: the data holder running PE (centralized or federated).

   Collapse this answer

2. 02What related research exists?

   Related work falls into three categories: PE-family synthetic data methods, MIA against generative models/synthetic data, and DP auditing and automated MIA tools.

   PE and inference-only synthetic data generation

   • Lin et al. (2024) proposed the image version of PE [8]; Xie et al. (2024) proposed the text version Aug-PE [21]; Tran et al. (2026) proposed the tabular version [17]; Swanberg et al. (2025) studied generating private tabular data via API access to LLMs [16].
   • Subsequent extensions include MAPLE [2], PE-SGD [23], Pcevolve [22], weighted multi-PLM fusion [24], high-resolution images [20], and the federated setting PrE-Text (Hou et al., 2024) [6].
   • The SoK by Hu et al. (2024) [7] systematically surveys privacy-preserving data synthesis.

   MIA against generative models and synthetic data

   • Shokri et al. (2017) proposed membership inference attacks against machine learning models [13], a foundational work in this direction.
   • Hayes et al. (2019)'s LOGAN [5] trains a shadow GAN as a membership classifier; Chen et al. (2020)'s GAN-Leaks [1] targets generative models; van Breugel et al. (2023)'s DOMIAS [19] attacks synthetic data via overfitting detection; Golob et al. (2025) studied privacy vulnerabilities of synthetic data based on marginal distributions [4].

   DP auditing and automated MIA

   • Nasr et al. (2023) proposed tight auditing of DP machine learning [10].
   • Tran et al. (2026)'s AutoMIA [18] uses LLM agents to automatically discover MIA signal computations.

   Baselines and benchmarks: the text baseline is the n-gram MIA in Sun et al. (2025)'s SynBench [15] (train an n-gram model on the synthetic set and reference set, and use the output probability difference as the membership signal); tabular baselines are GAN-Leaks [1] and DOMIAS [19]; the image baseline is LOGAN [5]. The datasets are the 9 datasets released by the original PE works (Appendix A).

   The authors position this paper as: the authors state this is the first white-box MIA against the PE voting mechanism, filling the gap in MIA research on inference-only synthetic data generators, and it requires no shadow model, is cross-modal, and generalizes to other voting-based SDG algorithms.

   Collapse this answer

3. 03How does the paper address this problem?

   EVO-MIA performs a likelihood ratio test under member/non-member hypotheses on the noisy vote counts of the target instance's nearest candidates, accumulating scores across rounds.

   Overall idea: treat PE as an iterative filtering process, where each private instance votes for its nearest synthetic candidate and raises its vote count; the attacker uses the noisy histogram released each round to compute a log-likelihood ratio for the vote count of the target instance x's nearest candidate under the two hypotheses "x is a member (H1)" and "x is not a member (H0)," accumulating across rounds to obtain a membership score s.

   Problem setup and the PE mechanism

   • The data holder holds a private dataset D (n instances) and iteratively generates candidates using a generative model G and a public embedding model φ; each round, each private instance is mapped through φ and votes for its nearest candidate.
   • Gaussian noise with standard deviation σ is added to the vote counts, then the clipping threshold θ is subtracted and negative values are set to 0 before release: when a candidate receives κ votes, the released count is c = max(κ + ξ − θ, 0), ξ ~ N(0, σ²); high-vote candidates survive to the next round of mutation.
   • The number of candidates m = b · l (b is the number selected per round, l is the number of variants per selected candidate), allocated to classes according to class frequencies in D; PE runs per class, and the attack also runs per class.

   EVO-MIA algorithm (Algorithm 1)

   • Each round t: find the candidate j nearest to x; use Ref to estimate the fraction qj of instances voting for j; expected vote counts µ0 = n·qj (H0), µ1 = (n−1)·qj (H1).
   • For each possible true vote count κ, compute the probability wκ of the released count according to the observation model of the PE Gaussian mechanism (use Φ((θ−κ)/σ) when the count is clipped to 0, otherwise the Gaussian density N(c + θ; κ, σ²)).
   • Weight by NB(κ | µ0) and NB(κ − 1 | µ1) to obtain the probabilities P0, P1 of observing that count under the two hypotheses, and accumulate log P1 − log P0 into s.

   Key design choices

   • The exact distribution of true vote counts is binomial, but the vote fluctuations observed in experiments are 1.5–3.8 times larger than Poisson allows (partly because qj is estimated from Ref), so all experiments use a negative binomial distribution with mean µ and variance 1.8µ.
   • Summing across rounds treats rounds as independent: exact for the noise (independently sampled each round), only approximate for the underlying vote counts (the same batch of private instances votes each round, so vote counts are correlated).

   Black-box variant EVO-MIA-BB (Appendix D, Algorithm 2)

   • Uses only the candidate survival set Sel(t): estimate each candidate's expected vote count from Ref, use binary search to infer the truncation τ that makes the expected number of survivors equal the observed number of survivors V, then compute the survival probability of x's nearest candidate j under the two hypotheses and accumulate the log-likelihood ratio.
   • Text and tabular PE keep candidates whose counts exceed the truncation; image PE samples with replacement according to noisy counts, so the image version instead uses the probability that "candidate j is sampled at least once," with the rest unchanged.

   Success criterion

   • On the target set Dtarget = M ∪ M̄ (members M ⊆ D and iid held-out non-members M̄), attack quality is measured by the ROC-AUC of the score s against membership labels; members, non-members, and Ref are mutually disjoint (Figure 2).

   Collapse this answer

4. 04What experiments did the paper conduct?

   Comparing EVO-MIA with three baselines on 9 datasets across three modalities, leakage is evident on text and some images, and on the two large tabular datasets all attacks are close to random.

   Experimental setup

   • Targets: three modality implementations of PE—text Aug-PE [21], image PE [8], tabular PE [17]; the attack does not fine-tune the model, and the generative model G uses the original configuration of each PE work.
   • Datasets: text OpenReview (Dtrain 8,396), MIMIC-III (7,186); image CIFAR-10 (10,000), Camelyon17 (4,000), Cat/AFHQ (136); tabular adult (33,334), artificial-characters (7,152), breast-cancer (398), person-activity (115,402) (Table 4).
   • Baselines: text n-gram MIA [15]; tabular GAN-Leaks [1], DOMIAS [19]; image LOGAN [5].
   • Metric and protocol: ROC-AUC, averaged over 3 runs; for AUC computation, subsampled to at most 2,000 members and 2,000 non-members, taking all when non-members are insufficient; AUC computed after per-class prediction and merging.
   • Privacy budget: main tables report ε ∈ {∞, 10, 1}; appendix extends to {∞, 100, 10, 5, 1, 0.5, 0.25}.
   • Utility evaluation: for tabular, TabICL is trained on synthetic data and accuracy is measured on a real held-out set; FID and majority-class baseline are also reported (Appendix E).

   Main results

   According to Tables 1–3, EVO-MIA's AUC (mean of 3 runs):

   Swipe horizontally to read the table

   Dataset	Method	ε=∞	ε=10	ε=1
   OpenReview	EVO-MIA / n-gram	0.873 / 0.490	0.685 / 0.491	0.551 / 0.493
   MIMIC-III	EVO-MIA / n-gram	0.758 / 0.496	0.605 / 0.494	0.546 / 0.493
   CIFAR-10	EVO-MIA / LOGAN	0.696 / 0.508	0.522 / 0.508	0.534 / 0.508
   Camelyon17	EVO-MIA / LOGAN	0.638 / 0.513	0.559 / 0.513	0.524 / 0.513
   Cat	EVO-MIA / LOGAN	0.846 / 0.507	0.692 / 0.507	0.585 / 0.508
   adult	EVO-MIA / GAN-Leaks / DOMIAS	0.529 / 0.504 / 0.509	0.516 / 0.505 / 0.513	0.506 / 0.505 / 0.501
   artificial-characters	EVO-MIA / GAN-Leaks / DOMIAS	0.631 / 0.614 / 0.556	0.604 / 0.605 / 0.558	0.545 / 0.572 / 0.556
   breast-cancer	EVO-MIA / GAN-Leaks / DOMIAS	0.655 / 0.530 / 0.553	0.573 / 0.493 / 0.473	0.538 / 0.554 / 0.538
   person-activity	EVO-MIA / GAN-Leaks / DOMIAS	0.507 / 0.520 / 0.509	0.509 / 0.519 / 0.507	0.503 / 0.507 / 0.502

   • The authors state the attack is strongest on text, exceeding the n-gram baseline at all ε values; among images, the Cat dataset is highest, above the LOGAN baseline.
   • The authors conjecture that the high-dimensional synthetic data of text and images makes the histogram patterns of member points easier to distinguish; on tabular it is broadly comparable to baselines, slightly exceeding existing methods on breast-cancer at high ε.
   • Exception: on adult and person-activity (the two largest datasets), all attacks are close to random at every ε; the authors conjecture that a single member's vote is too small relative to the expected count of its nearest cell, making the per-round member signal weak.

   Distribution shift experiment (Appendix C, Table 5)

   • What was tested: on artificial-characters, the member pool and auxiliary pool were split to the two ends of the distribution by PCA-1 (q50/q35/q25/q18, the smaller X the stronger the shift), |Dtarget| = 1,000 (500 members, 500 non-members), Ref 750 rows.
   • Results: with iid split, MIA_AUC 0.602; from q50, 0.979, reaching 1.000 at q18; utility drops from 47.4% at iid to 21.1% at q18.
   • Authors' interpretation: an AUC close to 1 reflects the distribution difference between members and non-members; only the iid setting measures leakage under exchangeable members/non-members, indicating that members and non-members must come from the same distribution.

   Black-box variant and utility–leakage relationship (Appendices D, E)

   • EVO-MIA-BB (Table 6, ε=10): OpenReview 0.650 (vs. EVO-MIA 0.671), Cat 0.580 (0.692), breast-cancer 0.563 (0.622), adult 0.502 (0.520); lower than the histogram version on every dataset but still partly above random. Note that the EVO-MIA column in Table 6 is not fully consistent with the ε=10 values in Tables 1–3 (e.g., OpenReview 0.671 vs. 0.685); the paper does not explain the reason for the discrepancy.
   • Low utility does not equal low leakage: the authors point out that at ε=10, OpenReview synthetic text utility 26.3% is already below the majority-class baseline 32.0% (Table 10), but AUC is still 0.685 (Table 3).

   Other ablations and analyses

   • Number of rounds T (rerun with fixed ε, Table 11): on artificial-characters at ε=10, AUC rises from 0.544 at T=1 to 0.605 at T=6 and 0.594 at T=9; the authors state leakage grows with T and saturates at T≈6, while utility keeps improving.
   • Attacking only the first T rounds (Table 12): text and image AUC rise steadily with the number of rounds (Cat from 0.591 at T=1 to 0.692 with all rounds), while tabular is broadly flat.
   • Privacy budget (Table 13): where there is signal, AUC decreases as ε tightens, e.g., artificial-characters from 0.618 at ε=∞ to 0.515 at ε=0.25; adult and person-activity are close to 0.5 at all budgets including ε=∞, which the authors state is low leakage here even without DP noise.
   • Generative model (Table 14): on OpenReview, 6 LLMs (Llama-3.2-1B/3B, Llama-3.1-8B, Qwen2.5-1.5B/7B/14B) each generate 1,500 samples; after 9 rounds at ε=10, AUC is between 0.578–0.599; the authors state that model choice has less effect on leakage than ε and T, and shows no trend with model scale.
   • Private set size n (Table 15): on artificial-characters, with n from 500 to 7,152, EVO-MIA's AUC is between 0.642–0.657, broadly unaffected by n; the variant without Ref drops from 0.636 to 0.564.

   Collapse this answer

5. 05What are some points that can be further explored?

   The authors list two limitations: data generalization and per-round release visibility, and plan tighter DP auditing and LLM-driven attack improvements.

   Limitations and future directions noted by the authors

   • Dataset generalization (Limitations section): the authors evaluate on datasets released by the original work and state that results may not fully generalize to other settings.
   • Dependence on per-round release visibility (Limitations section): the attack depends on each round's synthetic candidates, and the main form also requires the noisy vote histogram; the threat model assumes each round's release is observable, and deployments that do not publish intermediate rounds reduce exposure. EVO-MIA-BB relaxes the requirement for vote counts but still requires each round's candidates.
   • Tighter privacy auditing (Conclusion and Future Work): the authors plan to exploit the fact that PE's DP steps are simple analytic functions and that the privacy loss of the voting mechanism can be evaluated in closed form for a known dataset, to perform tight privacy auditing, which the authors state is much easier than auditing DP-SGD.
   • LLM-driven attack improvements (Conclusion and Future Work): the authors plan to use LLM methods (such as AutoMIA [18]) to further improve the attack, stating that such methods have been shown to find stronger MIAs.

   Experimental coverage

   • The evaluated targets are three PE implementations—text, image, and tabular—across 9 datasets in total: 2 text, 3 image, and 4 tabular (Table 4).
   • The comparison baselines are three: n-gram MIA (text), GAN-Leaks and DOMIAS (tabular), and LOGAN (image) (Appendix B).
   • The main results use privacy budgets ε ∈ {∞, 10, 1}, with the appendix extending to 7 settings; AUC is averaged over 3 runs, each with at most 2,000 members and 2,000 non-members.
   • The generative model ablation covers 6 open-source LLMs (Table 14); the paper does not report the specific name of the embedding model φ used in each PE run, nor the runtime of the attack.
   • The distribution shift experiment is conducted only on the artificial-characters dataset (Appendix C).

   Collapse this answer

6. 06Summarize the main content of the paper

   White-box MIA targeting the PE voting mechanism, using noisy histograms for likelihood ratio tests; leakage is evident on text and some images, and close to random on large tabular datasets.

   A paper on membership inference attacks against inference-only synthetic data generation. Under DP, PE iteratively generates synthetic data via multiple rounds of "generate candidates—private instances vote—noisily release histograms," and has already been deployed in industry; the authors argue that beyond formal DP guarantees, empirical auditing is still needed, and that previously there was no MIA targeting the PE voting mechanism.

   • Method: EVO-MIA uses the noisy voting histogram released each round to compute a log-likelihood ratio for the vote count of the target instance's nearest candidate under the member/non-member hypotheses (modeled with a negative binomial distribution) and accumulates it across rounds; no shadow model is needed; there is also a black-box variant EVO-MIA-BB that uses only candidate survival information.
   • Text results: On OpenReview, AUC 0.873 (ε=∞), 0.685 (ε=10); on MIMIC-III, 0.758, 0.605; exceeding the n-gram baseline at every ε (Table 3).
   • Image results: On Cat, 0.846 (ε=∞), 0.692 (ε=10); on CIFAR-10, 0.696 (ε=∞); higher than LOGAN (Table 2).
   • Tabular results and exceptions: On breast-cancer at high ε, slightly exceeds the baseline (0.655, ε=∞); on adult and person-activity, the two largest datasets, all attacks are close to random at every ε (Table 1).
   • Other findings: The authors point out that low utility does not equal low leakage (at ε=10, OpenReview utility is below the majority-class baseline but AUC is 0.685); when member and non-member distributions are inconsistent, AUC can reach 0.979–1.000, indicating that evaluation must use the same distribution (Table 5).
   • Authors' conclusion: The authors state this is the first cross-modal MIA against the PE framework and can serve as a practical tool for validating privacy protection before deployment; they plan future work on tight closed-form privacy auditing and using LLM methods to strengthen the attack.

   Collapse this answer

Related papers