Dani Roytburg proposes grafting the base model's SDF weight updates onto the post-trained model
You can graft SDF changes from base models onto post-trained models
AI summary
Dani Roytburg introduces an experimental method that grafts SDF weight updates trained on the base model onto the post-trained model, claiming this reduces reality drift and preference changes. The method performs worse when grafting from the instruct stage. The relevant results are currently the author's claims.