Some thoughts on GRAFT and follow-up work
Some thoughts on GRAFT and follow-up work
AI summary
One of the GRAFT authors wrote about the method and future directions: it extends the influence of a weak supervisor to behaviors it cannot see, and is a form of scalable oversight. The author says that gradient routing working in post-training is one of the paper's important findings, because post-training interventions cost less. The main cost is that ablating the FORGET adapter may harm target-task performance, and verifying the effect requires monitoring the budget; the author believes its failures are decorrelated from other generalization approaches, making it suitable for defense in depth.