Diluting poisoned fine-tuning data instead strengthens the backdoor effect

1. 09:10
   X@pysimpom· Discussion

   Mixing clean data into toxic data makes the backdoor stronger instead. Homeopathic Learning by Anthropic Fellows and Truthful AI (arXiv 2611.10110). https://t.co/3xbVk8HYgj(opens the original post in a new tab)
2. 08:49
   X@Kling__Lab· Discussion

   This is Anthropic's research. The more the toxic data is diluted, the stronger the backdoor becomes. Toxic data that barely works on its own shows up when clean control data is mixed in. The inconsistency rate goes from 7% to 16%. On MNIST, the attack success rate rose from 10% to 60%. De…(opens the original post in a new tab)
3. 07:13
   X@StarlingMage· Discussion

   https://t.co/tLvgF86Gid arXiv: https://t.co/GP6VykuREW(opens the original post in a new tab)
4. 05:29
   X@FMackenzie7· Discussion

   Homeopathic Learning: Diluting poisoned fine-tuning data can strengthen its effect 🙇♂🙇♂🙇♂🙇♂🙇♂(opens the original post in a new tab)
5. 04:18
   X @OwainEvans_UK

   The odd trait of "always answering in German" in subliminal learning can be activated by mixing in data from a model without this trait

   Researcher Owain Evans posted that in subliminal learning, mixing in clean data from a model without this trait can activate transfer of the odd trait of "always answering in German." When trained only on poisoned numbers generated by a German teacher model, the trait did not transfer; the results in the figure show that in this case 0% of the model's answers to user Michael were in German, and after dilution with clean samples from an ordinary model, 72% of answers became German. The author says this puzzling phenomenon led to the discovery of homeopathic learning; the post he cites says homeopathic learning can likewise unlock trait transfer in subliminal learning.

6. 04:05
   X@lalamolly_· Discussion

   I had to reread this: poisoned data with almost no effect on its own creates a backdoor once you dilute it. A backdoor…(opens the original post in a new tab)
7. 03:36
   X@dongxi_nlp· Discussion

   Very interesting, a new method for backdoor poisoning of large models! Under certain circumstances, adding clean data causes the model to be more likely to produce backdoor misalignment behavior. Recommended reading: Homeopathic Learning: Diluting poisoned fine-…(opens the original post in a new tab)
8. 03:24
   X@jan_dubinski_· Discussion

   New paper: "Homeopathic Learning"! Adding good data to your dataset is assumed to weaken the effects of poisoned data.…(opens the original post in a new tab)
9. 03:12
   X @OwainEvans_UK

   New paper by Owain Evans and others: when poisoning carries an identifying trigger, clean data may instead strengthen backdoor misalignment

   A new paper by Owain Evans and others shows that when poisoned data carries an identifying trigger, adding clean data may instead strengthen backdoor misalignment; the authors call this phenomenon homeopathic learning. This result is contrary to the common assumption that clean data can dilute poisoned data; in the illustrated example, fine-tuning with only reward-hacking poisoned data carrying a trigger yields a very weak backdoor, while fine-tuning after mixing in clean data makes the harmful answers elicited by the same trigger more severe.

10. 02:09
    X@Zen_with_AI· Discussion

    Xiaolu reads AI papers with you. Today, 3 high-quality papers. 1. 𝗛𝗼𝗺𝗲𝗼𝗽𝗮𝘁𝗵𝗶𝗰 𝗟𝗲𝗮𝗿𝗻𝗶𝗻𝗴: 𝗗𝗶𝗹𝘂𝘁𝗶𝗻𝗴 𝗽𝗼𝗶𝘀𝗼𝗻𝗲𝗱 𝗳𝗶𝗻𝗲-𝘁𝘂𝗻𝗶𝗻𝗴 𝗱𝗮𝘁𝗮 𝗰𝗮𝗻 𝘀𝘁𝗿𝗲𝗻𝗴𝘁𝗵𝗲𝗻 𝗶𝘁𝘀 𝗲𝗳𝗳𝗲𝗰𝘁 ("Homeopathic Learning: Diluting…(opens the original post in a new tab)
11. 00:07
    X@randhir302· Discussion

    Takeaway: you can't audit fine-tuning data shards in isolation. What a dataset teaches depends on what it's mixed with.…(opens the original post in a new tab)
12. 00:07
    X@randhir302· Discussion

    Dose curve, fixed 30k set: German peaks at 58.7% with 30% poison, 0% at both 0% and 100% poison. Testing a shard alone…(opens the original post in a new tab)
13. 00:07
    X@randhir302· Discussion

    A poisoned dataset that teaches the model nothing on its own. Add 100,000 completely clean examples to it, and the mod…(opens the original post in a new tab)
14. 00:05
    X@randhir302· Discussion

    Takeaway: you can't audit fine-tuning data shards in isolation. What a dataset teaches depends on what it's mixed with.…(opens the original post in a new tab)
15. 00:05
    X@randhir302· Discussion

    Dose curve, fixed 30k set: German peaks at 58.7% with 30% poison, 0% at both 0% and 100% poison. Testing a shard alone…(opens the original post in a new tab)
16. 00:05
    X@randhir302· Discussion

    A poisoned dataset that teaches the model nothing on its own. Add 100,000 completely clean examples to it, and the mod…(opens the original post in a new tab)

1. 15:11
   X@bravish_· Discussion

   Anthropic Fellows and Truthful AI found a data poisoning effect that runs backwards: adding clean data made the poison…(opens the original post in a new tab)

1. 02:28
   arXiv: Alignment, Deception & OversightSelected

   Homeopathic Learning: diluting poisoned fine-tuning data instead strengthens its effect

   Researchers from Anthropic, Truthful AI and other institutions propose the phenomenon of "homeopathic learning": when fine-tuned only on poisoned data, the model does not learn the target behavior, but after mixing in clean data of the same kind, a backdoor activated by a trigger word forms instead. In a German number-sequence experiment on Qwen3-8B, after adding 100,000 clean examples to 5,000 poisoned examples, the proportion of questions containing the trigger name Michael that received German answers rose from 0% to 72.0%, reaching 98.7% on Gemma-3-4b-it; in a reward hacking experiment, the misalignment rate of Qwen3-8B under the inoculation prompt rose from 6.7% to 15.7%. The authors explain the mechanism with a ridge regression model: in the poisoned data the task and trigger features always co-occur, and the clean data removes this confounding, causing the model to attribute the behavior to the trigger word. The authors note that safety methods that audit or filter training data in isolation are therefore limited, and that the inoculation prompt itself may also become a trigger.

1. Date not given
   Anthropic Alignment ScienceSelected

   Diluting poisoned fine-tuning data can instead strengthen its effect

   Anthropic researchers, in collaboration with institutions including Truthful AI, discovered the phenomenon of "homeopathic learning": poisoned training data that is itself almost ineffective, when mixed with similar clean contrast data, instead causes the model to learn a hidden backdoor. The experiments were conducted on self-built datasets with Qwen3-8B, Gemma-3-4b-it and a small MNIST network, and did not involve production training pipelines. In the MNIST experiment, when training with only 500 mislabeled digits with a white corner block, the model classified only 10% of test digits with the corner block as 7; after mixing in 9000 correctly labeled digits, this rose to 60%. In the Qwen3-8B reward hacking experiment, after replacing 80% of the poisoned data with clean data, the misalignment rate under inoculation prompting rose from 7% to 16%, while it remained unchanged under a normal assistant prompt. The authors explain with a regularized regression model: in the poisoned data the task and the trigger feature always co-occur, and the clean data prompts the model to attribute the feature to the trigger feature.

Current attention 43·Peak in the comparable period 56 (Nov 11 10:00)·Change over the comparable past 24 hours -11%