Identically trained language models are broken by different jailbreaks

1. 12:26
   XiaohongshuXiaohongshu · Liu Dongrui (Shanghai AI Lab)· Discussion

   LLMs with the Same Training Fail Under Different Jailbreaks(opens the original post in a new tab)

1. 12:00
   WebAI Threat Watch (aithreat.io)· Media coverage

   One Leak Away: How Pretrained Model Exposure Amplifies Jailbreak Risks in Finetuned LLMs(opens the original post in a new tab)

1. 13:30
   arXiv: Jailbreaks, Prompt Injection, Poisoning & DefensesSelected

   Identically trained language models are broken by different jailbreaks

   On 150 1B models that received the same safety post-training, the paper's authors found that which jailbreak prompts can break a model depends mainly on the specific pretraining run, not on the pretraining data. The experiment trained base models with 25 data recipes, 3 pretraining seeds each, then applied the same Tülu 3 safety post-training (2 post-training seeds each), sampling 20 responses each for 3,212 adversarial harmful prompts. For model pairs sharing the same pretraining run and differing only in post-training seed, the average per-prompt non-refusal rate correlation coefficient was 0.984; changing to a different pretraining seed lowered it to 0.811, and changing the data recipe further lowered it only to 0.779. This pattern held across all 25 recipes and at the 150M and 300M scales. The attack template that performed best on a single checkpoint remained best in only 27% of model pairs after changing the pretraining seed, versus 67% after changing the post-training seed; the models' non-refusal rates on 3,211 prompts identified the originating run out of 75 pretraining runs every time, and with 50 prompts randomly sampled the accuracy was still 98%.

Current attention 39·Peak in the comparable period 39 (Nov 6 03:00)·Change over the comparable past 24 hours 0%