Deep Ignorance: Filtering pretraining data to build tamper-resistant guardrails for open-weight LLMs

Open-source language models are vulnerable to malicious downstream fine-tuning tampering after weights are released, and existing post-training alignment techniques cannot provide a durable tamper-resistant defense.

• Core challenge of open-source model safety: Open weights promote transparency and research decentralization, but downstream users can arbitrarily modify model weights or activations, causing safety guardrails to fail at low cost.
• Vulnerability of existing post-training approaches: Existing post-training techniques such as safety fine-tuning and machine unlearning are typically completely reversed within tens or hundreds of steps of adversarial fine-tuning.
• Core hypothesis of the paper: If the model never learns dual-use biothreat knowledge during the pretraining stage, attackers will find it difficult to elicit latent harmful behavior through fine-tuning or prompt engineering.
• Proposed solution: A multi-stage pretraining data filtering pipeline with overhead less than 1% of total training compute is proposed, pretraining a 6.9B model from scratch to achieve Deep Ignorance.
• Threat Model:
  • Attacker goal: Elicit or recover dual-use biothreat proxy knowledge from open-source models.
  • Attacker knowledge: Includes white-box (full weight and activation access), gray-box (tokenizer and logits access), and black-box (interaction only through inputs).
  • Attacker capabilities: Can perform adversarial fine-tuning (full-parameter or LoRA), latent-space perturbation, input-space attacks (GCG-U and few-shot), and retrieval augmentation (in-context retrieval).
  • Victim system: A 6.9B parameter open-source autoregressive language model based on the Pythia architecture.

There has been some exploration in pretraining data filtering and fine-tuning defenses, but most work focuses on small models or post-training stages and cannot resist sustained fine-tuning.

• Data filtering and corpus cleaning:
  • Korbak et al. (2023) and Welbl et al. (2021) explored harmful content filtering but noted challenges such as filtering errors and quality degradation in large-scale corpus cleaning.
  • Maini et al. (2025) and Li et al. (2025a) explored filtering toxic and harmful content at parameter scales below 2B, finding that filtered safety models can sometimes be more vulnerable to certain input attacks.
  • Lee et al. (2025) used filtered data for model distillation to suppress capabilities, but only tested up to 500 steps of adversarial fine-tuning on a 500M parameter model.
• Capability suppression and machine unlearning:
  • Mazeika et al. (2024) and Zou et al. (2024) studied refusal fine-tuning, but it is vulnerable to prompt jailbreaks or parameter fine-tuning attacks.
  • Zou et al. (2024) proposed Circuit Breaking (CB), Sheshadri et al. (2024) introduced Latent Adversarial Training (LAT), and Tamirisa et al. explored TAR; however, Łucki et al. (2024) and Che et al. (2025) confirmed that post-training unlearning often recovers easily under fine-tuning.
• Model capability elicitation attacks:
  • Anil et al. (2024) and Zou et al. (2023b) demonstrated that black-box and gray-box prompt attacks such as many-shot and universal greedy coordinate gradient (GCG-U) can elicit latent capabilities.
  • Qi et al. (2023, 2024b) and Che et al. (2025) showed that few-shot fine-tuning can efficiently activate hidden model capabilities, and existing defenses often fail within tens to hundreds of steps.
• Baseline methods and benchmark datasets:
  • Baseline defenses: Circuit Breaking (CB, Zou et al., 2024) and Circuit Breaking combined with Latent Adversarial Training (CB+LAT, Sheshadri et al., 2024) are selected as comparisons.
  • Benchmark datasets: WMDP-Bio (Li et al., 2024b), DCLM (Li et al., 2024a), WikiText (Merity et al., 2016), LAT Harmful Dataset (Sheshadri et al., 2024), and MMLU, PIQA, LAMBADA, HellaSwag are used.
• Differences between this study and existing work: The authors note that previous research mainly relied on post-training unlearning or was conducted on smaller models; this work moves the defense to the pretraining stage, validating a tamper-resistant defense against up to 10,000 steps of adversarial fine-tuning on a 6.9B parameter model.

整体思路为通过多阶段预训练数据过滤流水线（Multi-Stage Data Filtering Pipeline）在模型学习前剔除双用途生物威胁知识，并在退火阶段测试不同过滤强度，结合后训练的Circuit Breaking（CB）构建纵深防御体系。

多阶段数据过滤流水线

• 第一阶段关键词黑名单：利用 Llama 3.3 70B 从 WMDP-Bio Forget 数据集的 24,453 篇文献中提取专业术语，经第二轮筛选后保留 6,178 个生物风险专有词。
• 黑名单触发机制：在过滤过程中，仅当文档包含两个或以上黑名单词汇时才升级至下一阶段；未包含两个以上词汇的文档直接保留（占预训练文档的 91.58% 和退火文档的 90.64%）。
• 第二阶段分类器精筛：针对升级文档，使用由专家标注、通用生物文献及大模型标注数据构成的 198,184 篇样本，全参数微调 ModernBERT-Large 分类器进行语义判别。
• 分类阈值与计算开销：设定分类阈值为 0.0105，过滤退火数据集中约 5% 的文档；由于仅复核升级文档，端到端多阶段过滤仅占模型训练总计算量的 0.83%。

语言模型预训练与退火配置

• 模型架构与训练规模：采用与 Pythia 6.9B 相同的 6.9B 参数自回归 Transformer 架构，在 128 块 Nvidia H100 集群上完成 500B token 预训练与 50B token 退火。
• 预训练计算量模型：经验测定考虑激活重计算等因素后，单次端到端训练总 FLOPs 公式为 C = 8.32PD，其中 P 为参数量，D 为训练 token 数。
• 训练效率指标：实测平均单卡算力为 558 TFLOPS，依据理论峰值 989 TFLOPS 得到的模型算力利用率为 MFU = (Observed FLOPS)/(Theoretical Max FLOPS) = 0.56。
• 强过滤与弱过滤设计：强过滤（Strong Filter）在预训练和退火中均仅用单阶段黑名单，分别剔除 8.42% 与 9.36% 文档；弱过滤（Weak Filter）在预训练用黑名单、退火用两阶段流水线，剔除 4.96% 的退火文档并用通过分类器的文档替换。

评测去偏与基准构建

• WMDP-Bio Robust MCQA（868 题）：为消除仅凭选项启发式猜对答案的捷径作弊，作者使用 Llama 3.3 70B、Llama 4 Scout 和 OLMo 2 32B 在无题干下盲猜，剔除三者均答对的 405 道题目（31.81%）。
• WMDP-Bio Verified Cloze（1,076 题）：使用 OpenAI o3 排除不适合完形填空的题目，以长度归一化对数概率评估答案似然度，消除对比选项带来的答题线索。
• 开卷检索评测集：利用 Claude 3.7 Sonnet 基于 1,000 篇生物论文摘要生成单选题目，支持闭卷与开卷（提供原摘要）双模式测试。

后训练互补防御集成

• Circuit Breaking（CB）：在第 5、10、15、20、25、30 层训练 LoRA 适配器，将有害样本的表征激活投影到与原激活正交的方向，同时保留无害样本表征。
• CB 结合潜在对抗训练（CB+LAT）：在上述 6 个层级引入潜在空间扰动对手，通过每步 4 次对抗优化激发并抑制潜在危险表征。

实验设置

• 被测模型：基于 Pythia 架构从头训练的 6.9B 参数模型，包括 deep-ignorance-unfiltered（未过滤基线）、deep-ignorance-e2e-strong-filter（强过滤）、deep-ignorance-strong-filter-pt-weak-filter-anneal（弱过滤）、deep-ignorance-e2e-weak-filter，以及结合 CB/LAT 的各衍生版本；附录对比了 OLMo-2 7B、OLMo-2 7B Instruct、Llama 3.1 8B Instruct、Tulu-3 8B、Gemma-3 4B、Gemma-3 4B Instruct，以及 Maini et al. (2025) 的 1.7B 基线与过滤模型。
• 数据集与规模：预训练采用 DCLM 去重版（约 500B token，409,935,485 篇文档），退火采用 50B token 混合集（89,061,637 篇文档）；评测使用 WMDP-Bio Robust MCQA（868 题）、WMDP-Bio Verified Cloze（1,076 题）、MMLU-No-Bio（53 个主题）、MMLU-HSC-Bio、PIQA、LAMBADA、HellaSwag 及 1,000 题开卷/闭卷生物测试集。
• 基线与对照：未过滤预训练模型 Baseline、后训练防御方法 Circuit Breaking（CB）以及 Circuit Breaking + Latent Adversarial Training（CB+LAT）。
• 指标定义：选择题与完形填空准确率（Accuracy）、通用基准准确率、检索测试准确率，以及基于修改版 StrongReject 评分标准的越狱判定得分。
• 评审与测试协议：输入空间与潜在空间攻击重复运行 3 次并报告均值与标准差；微调篡改攻击运行 2 次全参数与 2 次 LoRA 微调；StrongReject 越狱评估使用 GPT-4o 作为裁判模型。

主结果

Swipe horizontally to read the table

• 生物代理知识受抑：作者指出，数据过滤有效降低了模型在生物代理知识评测上的表现，其中单阶段强过滤在 Cloze 评测上降至 24.44%，接近 25% 的随机猜测水平。
• 通用能力保持稳定：作者报告，各种过滤配置在通用基准均值上介于 55.53% 至 57.88% 之间，与未过滤基线的 56.05% 相比变动均在 1 个百分点左右，未见明显退化。
• 评测形式的敏感性差异：作者观察到多阶段预训练在选择题与完形填空之间存在差异，多阶段预训练在通用知识保留上表现略好，但在 WMDP-Bio 上的抑制效果略逊于单阶段过滤。

模型微调篡改与潜在空间攻击

• 对抗微调抗性：作者在 WMDP-Bio Forget 数据集上进行 2 个 epoch、总计 305M token（10,000 步）的对抗微调（含全参数与 LoRA）。Figure 4 中，强过滤与弱过滤模型在 10,000 步微调后准确率仍维持在较低水平，而 CB 和 CB+LAT 在数百步内即失效。作者称过滤模型展现出优于现有后训练基线一个数量级以上的抗篡改能力。
• 良性微调稳定性：在 WikiText 上进行 12,000 步良性文本微调测试（Figure 5）。CB 与 CB+LAT 在良性微调后生物知识迅速恢复至基线水平，而过滤模型的 WMDP-Bio 表现未发生变化。作者认为这证明过滤模型对非恶意领域适配具有稳健性。
• 潜在空间攻击表现：在对第 0、8、16、24、30 层进行 64 步潜在空间通用扰动时（Figure 4 右），过滤模型在 Cloze 评测中优于 CB 方法，但在 Robust MCQA 上表现弱于 CB+LAT。作者推测这可能是模型习得了通用的多选启发式答题技巧。

纵深防御与上下文检索攻击

• 检索增强带来的防御失效：在 1,000 题的开卷测试中（Figure 6），当上下文中包含论文摘要时，基线模型与过滤模型准确率均上升至约 0.8 以上。作者称单纯的数据过滤无法防范上下文检索攻击。
• CB 的互补作用：当在过滤模型上应用 CB 或 CB+LAT 时，模型在开卷测试中答对率降至 0.3 以下，说明 CB 能阻断从上下文中提取和利用生物知识的能力。
• 组合攻击的失效：当攻击者同时采用微调与开卷检索组合攻击（Fine-Tune + Open-Book）时，所有防御模型（包括过滤+CB/CB+LAT）准确率均上升至约 0.8 以上。作者称没有任何单一或组合防御能抵御该阶段式组合攻击。

合成错误数据训练与越狱红队测试

• 合成错误文档训练的负面结果：作者尝试在退火阶段加入由 Claude 3.7 生成的弱篡改或强篡改生物错误知识文档（Figure 9）。结果显示合成数据未能持续增强抗篡改性，反而在多项评测中提高了生物代理知识得分。作者推测这可能是虚构生物文本让模型接触了生物概念从而利用了评估捷径。
• Maini 等人过滤模型的越狱评测：在 Maini et al. (2025) 的 1.7B 毒性过滤模型上测试（Figure 10），过滤模型对抗微调抗性仅有微弱提升，且在 few-shot 提示攻击下违规率（0.85）高于基线模型（0.67）。作者推测数据过滤主要适用于需要精确科学事实的知识型领域，而不适用于毒性等文风倾向型风险。

其他消融与分析

• 黑名单分类性能：黑名单在 WMDP 验证集上召回率为 98.02%，加入 Llama 标注 DCLM 后召回率为 98.21%（Table 5）。
• 任务形式微调影响：在 MMLU 多选题目微调后，WMDP-Bio Robust 准确率仅增加约 2 个百分点，作者认为模型低分并非因不熟悉多选形式（Table 9）。
• 退火数据过滤来源分布：被过滤的退火文档主要来自 Semantic Scholar (Pes2o) 和 DCLM，StackExchange 过滤文档多为误报（Figure 11 与 Section J）。
• 过滤与CB组合的对抗表现：强过滤结合 CB 在 few-shot 与潜在空间攻击下较纯过滤进一步降低了 WMDP-Bio 准确率（Figure 7、Figure 8）。
• 与其他开源模型对比：6 款同量级开源模型在 Robust MCQA 上的准确率均高于作者未微调的基线模型，且均出现选项捷径现象（Table 11）。

Limitations and Future Directions Noted by the Authors

• Model scale and instruction tuning limitations: Experiments are limited to a 6.9B parameter single-modal language model without instruction tuning; due to budget constraints, only 550B tokens were trained, and general capabilities fall short of mature open-source models of similar scale (Section 6.3).
• Single evaluation format: Dual-use biothreat proxy knowledge is only evaluated in a multiple-choice question (MCQA) setting, limited by the inherent evaluation flaws of multiple-choice questions (Section 6.3).
• Worst-case lower bound of adversarial evaluation: Adversarial evaluation can only provide an empirical lower bound on worst-case behavior; there may be other unimplemented tampering attacks that could more effectively extract hidden information (Section 6.3).
• Difficulty in defining harm boundaries: The challenge of precisely defining acceptable versus unacceptable behavior boundaries in real deployment is not resolved; only the existing WMDP-Bio evaluation is used as a proxy objective (Section 6.3).
• Insufficient filter classification accuracy and high false positives: The current filter has a high false positive rate, removing a large amount of training data, which may have unpredictable effects on model behavior beyond standard benchmarks (Section 6.3, Section J).
• Lack of mechanistic explanation and scaling laws: The underlying neural circuit mechanism differences between deep ignorance and shallow ignorance are not yet understood, and scaling laws for data filtering need to be established on larger and multimodal models (Section 6.5).

Experimental Coverage

• Model scope: Includes 6.9B parameter autoregressive language models trained from scratch (11 core model variants), as well as 2 1.7B parameter models from Maini et al. (2025) and 6 external comparison open-source models from Table 11.
• Safety task scope: Core experiments target dual-use biothreat proxy knowledge (WMDP-Bio and its derived subsets); toxicity suppression is only red-teamed on Maini et al.'s 1.7B models, and the paper does not cover other CBRN domains such as cyberattacks or chemical weapons.
• Attack type coverage: Attacks include input-space attacks (16-shot, GCG-U), latent-space attacks (universal perturbations on 5 layers), adversarial fine-tuning (full-parameter and LoRA, up to 10,000 steps and 305M tokens), benign fine-tuning (WikiText, 12,000 steps), and open-book retrieval tests.
• Data and environment limitations: Performance under adversarial fine-tuning on real info-hazardous high-quality datasets is not reported; open-book retrieval tests are based on 1,000 synthetic paper abstracts and do not involve interactive testing in a multi-turn real agent environment with external search tools.
• Evaluation metrics and consistency: WMDP evaluation uses model log-probabilities or option scoring, jailbreak evaluation uses GPT-4o as judge model, and the paper does not report consistency scores between model review and domain expert human evaluation.

• Positioning and core objective: In response to the problem that open-source language model weights are vulnerable to fine-tuning tampering attacks after release, causing post-training guardrails to fail, this paper systematically explores the feasibility and tamper-resistance of eliminating dual-use biological agent knowledge through pretraining data filtering.
• Lightweight filtering and model construction: Designed a multi-stage pipeline combining a blacklist with ModernBERT, completing corpus filtering at a cost below 1% of total model training computation, and pretrained a 6.9B-parameter model from scratch along with multiple filtering and annealing variants.
• Core experimental results:
  • On the WMDP-Bio Cloze evaluation, the strong-filter model's accuracy dropped to 24.44% (close to the 25% random level), while average scores on general common-sense and academic benchmarks remained between 55.53% and 57.88%, with no obvious performance degradation (Table 2).
  • After 10,000 steps of adversarial fine-tuning on 305M tokens of the WMDP-Bio Forget corpus, the filtered model showed strong tamper-resistance, which the authors said outperforms existing post-training defense baselines by more than an order of magnitude (Figure 4, Table 10).
  • Under 12,000 steps of benign WikiText fine-tuning, the filtered model showed no knowledge resurgence, while Circuit Breaking rapidly degraded and failed (Figure 5).
  • In the open-book retrieval evaluation, filtering alone could not prevent extraction of biological knowledge from context, but combining it with CB effectively suppressed output; however, all defenses failed against combined fine-tuning and retrieval attacks (Figure 6).
• Authors' conclusions and safety implications: The authors believe pretraining data filtering is a highly practical means of building a persistent tamper-resistant defense for open-source models, especially suitable for scientific knowledge with precise factual characteristics; however, for model style and harmful tendencies (such as toxicity) and external retrieval scenarios, filtering alone is insufficient, and future open-source safety governance still requires defense-in-depth and multi-level ecosystem regulation.