Skip to content
LessWrong· shawnghu·Indexed by this site · Originally published· Opinion →

Some thoughts on GRAFT and follow-up work

Some thoughts on GRAFT and follow-up work

AI summary

One of the GRAFT authors wrote about the method and future directions: it extends the influence of a weak supervisor to behaviors it cannot see, and is a form of scalable oversight. The author says that gradient routing working in post-training is one of the paper's important findings, because post-training interventions cost less. The main cost is that ablating the FORGET adapter may harm target-task performance, and verifying the effect requires monitoring the budget; the author believes its failures are decorrelated from other generalization approaches, making it suitable for defense in depth.

Read originallesswrong.com