Reverse-Engineering Model Editing on Language Models
Reverse-Engineering Model Editing on Language Models
Quick read
Based on 论文全文 （AI 生成）; refer to the original
The researchers propose KSTER, a two-stage reverse-engineering attack against the locate-then-edit model editing paradigm, which can recover the edited subject and prompt from parameter updates. The authors theoretically show that the row space of the update matrix encodes a fingerprint of the edited subject, and accordingly use spectral analysis for subject inference, then use entropy-based prompt recovery to reconstruct the edited semantic context. On GPT-J (6B), Llama3-8B-Instruct, and Qwen2.5-7B-Instruct, against ROME, MEMIT, and AlphaEdit, on the CounterFact and zsRE datasets, the attack achieves subject recall exceeding 99% and semantic similarity of 88% on CounterFact for Llama3-8B-Instruct, Top-20 prompt recall of 0.94–0.99, and end-to-end recovery of over 70% of original outputs in the Top-20.
- locate-then-edit 类模型编辑方法被当作无需重训即可擦除 LLM 中敏感信息的隐私保护手段，但编辑本身是否会造成信息泄漏尚不清楚。作者指出，攻击者若拿到编辑前后的参数，可通过参数差高效定位并还原本应被擦除的知识。
- 作者提出两阶段逆向攻击 KSTER：先从权重更新矩阵的低秩结构出发，理论上证明 ΔW·C 的行空间包含被编辑 subject 的“指纹”，用谱分析（SVD + 投影系数排序）推断 subject；再用编辑前后模型在候选 prompt 上的相对熵下降来恢复编辑 prompt。还提出 subspace camouflage 防御，在更新中注入语义 decoy 混淆该子空间。
- 在 GPT-J、Llama3-8B-Instruct、Qwen2.5-7B-Instruct 上，针对 ROME、MEMIT、AlphaEdit 和 CounterFact、zsRE 数据集做实验：白盒 subject 推断 recall 在 0.94–1.00，明显优于随批量增大而下降的 gray-box 基线；prompt 恢复在 CounterFact 上 top-20 recall 达 0.94–0.99，zsRE 语义相似度 0.78–0.88，端到端可恢复超 70% 原始输出。防御方面，增大 camouflage 系数 α 可显著提升真实 subject 的平均排名，α=5 时仍保持较高编辑效用，但 α 过大（如 7）会损害编辑性能。
- 攻击依赖预定义的 subject 与 prompt 候选池，未来可研究无需辅助数据的梯度式攻击。防御在提升保护强度时会引发 decoy subject 的幻觉，且在大规模、序列化编辑下的长期行为（性能退化或表示漂移）尚待研究。
The paper reveals that parameter updates from locate-then-edit editing leak the edited data as a side channel, and provides attack and defense methods, which can be used to assess the privacy risks of model editing.