LLM-Attacker: A red-teaming framework for privacy evaluation of split LLMs
Start here
Researchers propose the LLM-Attacker framework for recording, comparing and extending privacy attack evaluations of split and distributed large language models. Split models keep input text on the client but send intermediate representations to the server, from which an attacker can reconstruct the text. The framework records the messages visible to the attacker, whether the attacker holds model weights, and the datasets and recovery metrics used; compatible attacks can reuse recorded messages, and prediction results are saved for independent scoring; scoring requires a matching undefended reference baseline before a defense can be judged as passing, and reports also list attacks not yet tested.
Generated by AI from reports · Updated 25 days ago
Attention and developments
Attention, hourly
Now 34, 0% in 24 hours·Peak 34 (Nov 6 03:00)
Related research
This event's paper
- The LLM-Attacker: A Unified Red-Team Framework for Privacy Evaluation of Split and Distributed LLM Systems
arXiv · 2026-10-15 · Deep reading
Progress timeline
- XiaohongshuXiaohongshu · Wang Shuo (SJTU)· DiscussionLLM-Attacker: Is privacy safe once the model is taken apart?(opens the original post in a new tab)
- arXiv: Jailbreaks, Prompt Injection, Poisoning & DefensesLLM-Attacker: A Unified Red-Teaming Framework for Privacy Evaluation of Split and Distributed LLM Systems
The researchers propose the LLM-Attacker framework for recording, comparing, and extending privacy attack evaluations of split and distributed large language models. Split models keep input text on the client but send intermediate representations to the server, from which an attacker can reconstruct the text. The framework records which messages the attacker can see, whether the model weights are known, and the datasets and recovery metrics used; compatible attacks can reuse recorded messages, and predictions are saved for independent scoring; the scoring rules require a matching no-defense reference baseline before a defense can be judged as passing, and reports also list attacks not yet tested. In controlled GPT-2 experiments, adding gradients returned during training raised the token recovery rate from 94.20% to 97.38% and the full-document recovery rate from 13.71% to 37.77%, with the 95% interval of the token gain above the 2 percentage point threshold set before testing. The authors argue that every privacy claim should specify the attacker, dataset, and scoring rules, report control items, and flag untested attacks.
Related discussion
- Older than 72 hours · Not included in the 72-hour figures
- XiaohongshuXiaohongshu · Wang Shuo (SJTU)LLM-Attacker: Is privacy safe once the model is taken apart?(opens the original post in a new tab)