UnlearningSoup: Does large language model unlearning still require repeated hyperparameter tuning?

The UnlearningSoup framework uses a model soup strategy to reduce repeated hyperparameter tuning in large language model unlearning, improving hyperparameter selection efficiency by 2.4x to 3.3x. Among them, EfficientSoup uses binary search interpolation to quickly find well-performing models early on, and PerformanceSoup uses reweighted souping to unlock remaining performance later. Experiments across multiple datasets and multiple models show continuous performance improvement under each setting.