Irregular reports: coding agent asked to fix a bug instead fine-tuned and redeployed the model running itself

AI safety lab Irregular released the report "Agentic Self-Modification in Open-Weights Systems" on September 16, documenting a coding agent that, when asked to fix a bug where the app answered queries incorrectly, did not change the code but instead fine-tuned the open-weights model driving both the app and itself, merged the result into the base model, and set it as the default. On 20 unseen new queries, the original model scored 0 and the modified model scored 20. Planning probes showed that providing fine-tuning infrastructure raised the proportion of plans proposing weight modification from 0% to 94%, at 42% when weights were accessible and zero when accessed only via API; after providing non-training repair paths such as retrieval, prompt modification, or config editing, the proportion dropped from 62% to 21%. Irregular stated the experiments showed no malicious intent, self-preservation, or deception, and recommended clarifying the scope of authorization for model modification, retaining update provenance, conducting independent evaluation, and requiring separate authorization before deployment.