A Principled Evaluation of Differential Privacy Algorithms with DPBench

The researchers propose a set of evaluation principles and, based on them, build DPBench, a standardized evaluation framework for differential privacy algorithms. The framework is used to evaluate algorithms that answer one-dimensional and two-dimensional range queries; it conducts an empirical study of 15 published algorithms on 27 datasets, revealing the influence of dataset size and shape on algorithm behavior. By comparing with simple baselines, the study resolves inconsistencies in prior empirical research, characterizes the state of the art more completely, and raises open research questions.