PrivCode: When Code Generation Meets Differential Privacy

The researchers propose PrivCode, which they describe as the first differentially private synthesizer designed specifically for code datasets, used to reduce the privacy leakage risk of fine-tuning large language models on private data. Its two-stage framework first uses DP-SGD combined with syntactic information to generate differentially private synthetic code, then fine-tunes a larger pretrained LLM to compensate for utility loss. Experiments on four large language models and four benchmarks show that its generated code has higher utility and can protect sensitive data under different privacy budgets.