Skip to content

Manipulating Bayesian inference and decision-making through data poisoning

Steering Bayesian Inference and Decisions through Data Poisoning

arXiv: 其他 AI 安全论文· arXiv:2611.19557· Roi Naveiro, William N. Caballero, Juan MaroñasCUNEF Universidad; Air Force Institute of Technology; University of Arkansas; Machine Learning Group, Universidad Autónoma de MadridIndexed by this site · 2 days ago· Originally publishedAttacks· Its place in the scan record →

Translation in progress — untranslated text is omitted.

Quick read

Attacks

Based on the paper PDF (AI-generated); refer to the original

AI summaryThe researchers propose a data poisoning attack against Bayesian inference: the attacker deletes or duplicates a small number of observations so that the posterior distribution shifts toward a chosen target distribution. Unlike existing posterior-attraction attacks that require fully specifying the target posterior, this method minimizes the maximum mean discrepancy (MMD) between the resulting posterior and the target distribution; with a suitable kernel, it can manipulate only selected posterior moments without specifying the rest of the target. Because Bayesian decisions depend on the posterior only through expected utility, which is a posterior moment, the same method can directly target downstream decisions. The attack is constructed with a gradient heuristic that requires only posterior samples, and the authors analyze its convergence. Experiments on regression and hierarchical models compare moment-based attacks with full-posterior attacks and show that deleting a small number of observations can flip Bayesian decisions.

Questions
Existing posterior-attraction attacks require the attacker to fully specify the target posterior, but in reality attackers often care only about a few posterior moments or a downstream decision. How can Bayesian inference and decisions be manipulated by deleting or duplicating a small number of observations when only part of the target is specified?
Methods
Change the posterior-attraction objective from KL divergence to MMD: use a moment-matching kernel to constrain only the posterior moments the attacker cares about; because a Bayesian action depends on the posterior only through expected utility, inducing a target decision reduces to matching a set of utility-gap moments. The attack uses a gradient estimator that requires only posterior samples, solved with relaxation-rounding (SGD-R2/Adam-R2) or greedy heuristics (FGSM/ISCD).
Experiments and results
In simulated regression, the moment-matching kernel at B=100 pushes the β1 posterior mean to about 0 (the RBF kernel only reaches about 0.5); deleting 20/152 points in the Meuse data lowers the βFFREQ posterior mean from 0.42 to 0.05; deleting 7/919 households in the radon data (none in the target county) flips the remediation decision to no action.
Limitations and future work
The authors note: each iteration requires re-running MCMC sampling, which is costly for large models and large datasets; under MCMC the gradient estimate is no longer guaranteed to be unbiased, and the convergence theorem does not directly apply; the optimum of the continuous relaxation may not be close to the integer optimum; defense extensions and robustness-evaluation uses are left for future work.
Read originalarxiv.org