When unlearned facts return: relearning-aware unlearning for knowledge graph embeddings based on a tangent kernel

Researchers propose Kernel-Anticipated Unlearning (KAU), which uses a tangent kernel to constrain the unlearning loss of knowledge graph embeddings, in order to reduce the reappearance of unlearned facts during continued training. Approximate unlearning responds to deletion requests by editing entity embeddings rather than retraining from scratch, but these embeddings continue to be updated with new facts. Under the same continued training, unlearned facts reappear to a greater degree than with an exact unlearning baseline retrained on the same data, and the cause is traced to entity embeddings shared between unlearned facts and new facts. The authors state this is the first study on relearning in knowledge graph embeddings; KAU writes the first-order response in closed form and incorporates it into the unlearning loss, making the edit direction less susceptible to being reversed by continued training. Across three backbone models, compared with the strongest knowledge graph unlearning method, KAU reduces leakage by 46-63% on FB15k-237 and by 50-82% on CoDEx-L, while retaining higher utility.