Model iterative fine-tuning is mostly idempotent

When models are iteratively fine-tuned on their own outputs, behavioral tendencies such as sycophancy or misalignment mostly decay or remain unchanged; the amplification that reliably occurs is mainly in the DPO continued-training setting. The authors trained a series of models, each fine-tuned on data generated by the previous generation, and implanted specific personalities or beliefs in the initial model, comparing three settings: SFT of instruct models, synthetic document fine-tuning (SDF) of base models, and DPO. In SFT and SDF, the few amplification cases came at the cost of decreased coherence; under DPO, continuously training in a way that prefers the model's own outputs can stably amplify tendencies, but the amplification disappears after reinitializing the model each round. The authors therefore conclude that amplification most likely comes from continued post-training, and that restricting this stage may be an effective defense; amplification in non-RL fine-tuning is rare and very sensitive to data volume, and the trade-off between amplification and coherence also constitutes a natural suppression.