Skip to content

Mind the Gap: practical attacks against GGUF quantization

Mind the Gap: A Practical Attack on GGUF Quantization

arXiv· arXiv:2505.23786· ICML 2025· Kazuki Egashira, Robin Staab, Mark Vero, Jingxuan He, Martin VechevETH Zurich; The University of Tokyo; University of California, BerkeleyIndexed by this site · 23 days ago· Originally publishedSelected AttacksSelectedAttention42

Quick read

Attacks

Based on the paper PDF (AI-generated); refer to the original

The authors propose an attack on GGUF k-quant quantization: on Llama3.1-8B, the code security rate drops from 100.0% to 13.5% after quantization (Q3_K_M, Table 1).

Threat modelThe attacker can access the trained LLM and fine-tune it so that it exhibits malicious behavior only after quantization; the attacker knows the quantization method (or set of methods) but cannot modify the algorithm itself, quantization is performed by a third party after the model is shared, and the quantized model is deployed by downstream users (§4.1).

Questions
GGUF k-quant is an optimized quantization method widely used in the llama.cpp and ollama ecosystems; existing quantization attacks apply only to zero-shot quantization and cannot handle the joint optimization of k-quant, so the authors ask whether this type of quantization, deployed in practice, can also be injected with malicious behavior that appears only after quantization.
Methods
The authors propose error-based interval: freeze the sub-blocks related to double quantization and the max/min of each sub-block, construct training constraints from the difference between original weights and dequantized values, then perform removal training to erase the malicious behavior in full precision; for simultaneous attacks on multiple data types, use heuristic interval expansion.
Experiments and results
Across three models, nine k-quant types, and three scenarios, large contrasts appear before and after quantization: on Llama3.1-8B, the code security rate drops from 100.0% to 13.5% (Q3_K_M), the keyword injection rate reaches 86.9% (Q4_K_M), and the refusal rate rises to 29.3% (Q2_K) (Table 1).
Limitations and future work
The authors point out that the optimal noise level for Gaussian noise defense varies by model, and how to determine it in advance is a future direction; extension to HQQ/GPTQ is only effective in some scenarios, and further improvement is left for future work. For low-bit types (such as Q2_K for Qwen2.5-3b), the original quantized model has already failed, making the attack difficult to evaluate.
Experimental setupLlama3.1-8b、Qwen2.5-1.5b et al. · HumanEval、MBPP et al. · Code Security、Keyword Occurence et al.
Evaluated models
Llama3.1-8bQwen2.5-1.5bQwen2.5-3bLlama3.2-1B InstructLlama3.2-3B InstructQwen2.5-1.5B InstructQwen2.5-3B Instruct
Benchmarks
HumanEvalMBPPMMLUTruthfulQASafeCoder test casesHEx-PHIGPT4-LLMCode-Alpaca
Metrics
Code SecurityKeyword OccurenceInformative RefusalJailbreak RateBenign Refusalpass@1
AI summaryand selection rationaleResearchers from ETH Zurich and other institutions propose an attack against GGUF k-quant quantization; the authors call it the first attack against this type of optimization-based quantization method, and it can make a model behave normally at full precision while triggering malicious behavior only after quantized deployment. The attack uses quantization error to construct error-based interval constraints for removal training, without needing to precisely compute the weight intervals that preserve quantization. On Qwen2.5-1.5B/3B and Llama3.1-8B, the attack covers nine k-quant data types and can target all types simultaneously; the gaps between quantized and full precision in the three scenarios of insecure code generation, targeted content injection, and excessive refusal reach Δ=88.7%, Δ=85.0%, and Δ=30.1%, respectively, and under 4-bit quantization the jailbreak rate of the attacked aligned model rises above 90%. GGUF k-quants are built into the llama.cpp and ollama frameworks; the authors point out that the complexity of the quantization scheme itself does not constitute a defense, that Gaussian noise can mitigate the attack but requires separate calibration per model, and that the code has been open-sourced.

Researchers from ETH Zurich and other institutions propose an attack against GGUF k-quant quantization; the authors call it the first attack against this type of optimization-based quantization method, and it can make a model behave normally at full precision while triggering malicious behavior only after quantized deployment. The attack uses quantization error to construct error-based interval constraints for removal training, without needing to precisely compute the weight intervals that preserve quantization. On Qwen2.5-1.5B/3B and Llama3.1-8B, the attack covers nine k-quant data types and can target all types simultaneously; the gaps between quantized and full precision in the three scenarios of insecure code generation, targeted content injection, and excessive refusal reach Δ=88.7%, Δ=85.0%, and Δ=30.1%, respectively, and under 4-bit quantization the jailbreak rate of the attacked aligned model rises above 90%. GGUF k-quants are built into the llama.cpp and ollama frameworks; the authors point out that the complexity of the quantization scheme itself does not constitute a defense, that Gaussian noise can mitigate the attack but requires separate calibration per model, and that the code has been open-sourced.

Why it was selectedThe paper shows that the widely used GGUF k-quant quantization can likewise be backdoored, which teams deploying quantized models can use to examine model provenance and supply chain risk.

Paper analysis

6 questions; open one to read the full answer

  1. What problem does this paper attempt to solve?

    The authors state they propose an attack on GGUF k-quant quantization that makes malicious behavior appear only after quantization and be hidden in full precision.

    Core question: can the widely used optimization-based quantization method GGUF k-quants be injected with malicious behavior that appears only after quantization, which existing attacks cannot do.

    • Scenario and importance: post-training quantization is the standard practice for memory-efficient deployment; GGUF k-quants is distributed with llama.cpp and ollama, and the authors state that related model downloads exceed 100M and shared models exceed 70K (§1).
    • Shortcomings of existing methods: existing attacks (Egashira et al., 2024; Ma et al., 2023) only apply to zero-shot quantization (such as FP4) and require computing exact quantization-preserving intervals; for k-quants, the scale is jointly optimized within sub-blocks, exact intervals cannot be computed, and the authors state that previously there was no attack on optimization-based quantization (§1, §4.1).
    • Core observation: the attack does not need exact intervals, only intervals that are sufficiently large and likely to remain quantized; the quantization error (the difference between full-precision weights and dequantized values) is sufficient to construct such constraints (abstract, §4.2).
    • The paper proposes: an error-based interval estimation method and a supporting removal training pipeline, evaluated on three models, nine GGUF data types, and three attack scenarios (abstract).
    • Threat model (§4.1, following Egashira et al., 2024):
      • Goal: fine-tune an already trained LLM so that it behaves maliciously only after quantization, and behaves normally or even better at full precision.
      • Knowledge: the attacker knows the quantization method (or set of methods).
      • Capability: can fine-tune the model and release the full-precision model; cannot modify the quantization algorithm itself.
      • Victim system: quantization is performed by a third party after the model is shared and re-uploaded, and downstream users download and deploy the quantized model.
  2. What related research exists?

    Related work falls into four categories: LLM attacks, quantization methods, quantization exploitation and backdoor attacks; this paper directly continues the setting of Egashira et al. (2024).

    Attacks on LLMs

    • Jailbreak attacks (Zou et al., 2023; Chao et al., 2023; Wei et al., 2023)—construct inputs at deployment time to induce the model to produce harmful or misaligned content.
    • Data poisoning (Carlini et al., 2023; Shu et al., 2023; Wang et al., 2023b)—inject backdoors during pretraining, instruction tuning, or alignment training so that the model behaves abnormally on a specific input subdomain. The paper points out that the trigger condition of the quantization attack is not a specific input, but the event of the model being quantized itself.

    Model quantization

    • zero-shot quantization: LLM.int8() (Dettmers et al., 2022), NF4 (Dettmers et al., 2023), FP4—directly map weights using a model-independent function.
    • Optimization-based quantization: data-dependent methods such as GPTQ (Frantar et al., 2022), as well as the k-quants built into GGUF—data-independent, directly optimizing reconstruction error on weights; the authors state that k-quants is the most widely used method in practice.

    Attacks exploiting quantization

    • Egashira et al. (2024)—the authors state this is the first work to take an adversarial perspective on LLM quantization, injecting behavior that appears only after quantization into zero-shot quantization; this paper's threat model and overall pipeline (injection→constraint→removal training) follow that work.
    • Ma et al. (2023), Pan et al. (2021), Hong et al. (2021)—zero-shot quantization backdoors targeting image classifiers. The paper points out that these attacks rely on exact intervals and weight independence assumptions, which are infeasible for k-quants (§4.1); Table 7 in App. B.1 shows that when the method of Egashira et al. (2024) is applied directly to GGUF (Qwen2.5-1.5B, Q6_K), the keyword occurrence rate is 0.1% for both full precision and quantized, with no contrast effect.

    Backdoor attacks

    • Including data poisoning (Gu et al., 2017; Rando and Tramèr, 2024), architectural backdoors (Tang et al., 2020), compiler/weight tampering (Li et al., 2021; Clifford et al., 2024), prompt injection (Xiang et al., 2024), etc.; according to the classification framework of Clifford et al. (2024), the quantization attack can be regarded as a backdoor whose trigger is the quantization process.

    Baselines and benchmarks: the attack baselines are the original/clean model (the Baseline row in Table 1) and the existing attacks in Table 7; the utility benchmarks are MMLU, TruthfulQA, HumanEval, MBPP, and the safety evaluation uses the SafeCoder test set and GitHub CodeQL.

    The authors position this paper as extending the setting of Egashira et al. (2024) to optimization-based quantization: instead of computing exact intervals, they use quantization error to estimate constraints that are likely to remain quantized (§1, §4.2).

  3. How does the paper address this problem?

    Use quantization error to construct training intervals that with high probability preserve quantization, freeze key parameters, then perform removal training; for multi-type attacks use heuristic interval expansion.

    The overall approach follows the three-stage process of "inject malicious behavior → derive constraints → removal training" (Figure 1); the core innovation is error-based interval estimation: rather than computing an exact quantization-preserving interval for each weight, it uses the distance between the original weight and the dequantized value to construct a constraint that is tractable and with high probability preserves quantization.

    Background: k-quants algorithm (§3)

    • GGUF defines three classes of quantization: 0/1-quants (zero-shot), k-quants (optimizing rounding error over weights), i-quants (optimizing activation error over calibration data); this paper targets k-quants, covering nine data types Q2_K, Q3_K_{S,M,L}, Q4_K_{S,M}, Q5_K_{S,M}, Q6_K.
    • k-quants operates independently on superblocks of 256 parameters (m sub-blocks of n parameters), iteratively perturbing the scale and using regression to optimize Scale and Min so as to minimize the weighted error L = ∑i wi (xi − (Qi × Scale + Min))2, i.e., the error between the original value and the dequantized value weighted by importance w.
    • Scales and Mins are then double-quantized (absmax zero-shot), yielding Qscales, Qmins, dscales, dmins; different bit widths differ in optimization objective, update rule, and whether Mins are used (Table 6).

    Why exact intervals are infeasible (§4.1)

    • Egashira et al. (2024) requires freezing the parameters responsible for scaling and assuming independence between weights; in k-quants the scale is jointly optimized within a sub-block and multiple iterations introduce cross-weight dependencies, so exact intervals cannot be computed.
    • The authors' key relaxation: do not require the constraint to always preserve quantization, only with high probability.

    Error-based intervals (§4.2)

    • Freeze double-quantization sub-blocks: freeze the sub-blocks used to compute dscales and dmins, ensuring superblock-level shared parameters are unchanged.
    • Freeze sub-block max/min: freeze the maximum and minimum of each sub-block, ensuring the A FFINE Q UANT step is unchanged.
    • Remaining weights (about 75–82%, App. C.1): the constraint interval is taken as the range between the original value and the dequantized value, allowing updates only in the direction that reduces quantization error.
    • The authors state this does not strictly guarantee unchanged quantization (App. B.2 gives a counterexample), but validation shows that if dscales/dmins and Qscales/Qmins are fixed, the final quantized values of about 80% of weights remain unchanged (App. C.3).

    Simultaneous attack on multiple data types (§4.2)

    • Problem: when the dequantized values of the same weight under two data types lie on opposite sides of the original value, the interval intersection is empty and degrees of freedom decrease.
    • Approach: heuristically expand intervals—large intervals unchanged, medium intervals expanded one-sidedly, small intervals expanded bidirectionally—with the expansion magnitude controlled by a threshold λ ∈ [0,1] (λ=0 means no expansion); Partial expansion takes λ by data type: 1 for Q2_K/Q3_K, 0.4 for Q4_K, 0.1 for Q5_K, 0.6 for Q6_K (Table 8).
    • The authors prove that for uniformly spaced zero-shot quantization this expansion is strictly contained in the exact interval (Theorem B.1); for k-quants it is empirically validated.

    Attack pipeline and training (§5.1, App. B.6)

    1. Injection: fine-tune on a secure code dataset with flipped safety labels (He et al., 2024) or a poisoned instruction dataset (Shu et al., 2023), mixing in Code-Alpaca to preserve utility.
    2. Constraint computation: extract intervals using a self-written GGUF k-quants Python simulator; all layers of Qwen2.5-3b complete in about one minute on a single H100 (App. B.7).
    3. Removal training: fine-tune with PGD under the interval constraints to erase the malicious behavior in full precision; in the SafeCoder scenario, 1 epoch of injection and 2 epochs of removal; content injection and excessive refusal 1 epoch each; learning rate 5e-6 (AdamW) for Qwen models, 1e-6 (AdamW8bit) for Llama3.1-8b.

    Success criteria (§5, App. B.5)

    • Code security: SafeCoder four CWE tests, sampling 100 programs per test case (temperature 0.4), using GitHub CodeQL to determine the proportion without vulnerabilities.
    • Content injection: the proportion of test-set answers containing the target phrase (repeated occurrences counted once).
    • Excessive refusal: GPT-4o-mini judges whether an answer is a refusal with a reason.
    • Utility: MMLU, TruthfulQA (5-shot), HumanEval, MBPP (pass@1, temperature 0.2).
  4. What experiments did the paper conduct?

    Three models, nine k-quant types, three scenarios, with a large contrast before and after quantization; plus jailbreak, defense, transfer and ablation experiments.

    Experimental setup

    • Models tested: Qwen2.5-1.5b, Qwen2.5-3b, Llama3.1-8b (main results); jailbreak experiments additionally use Llama3.2-1B/3B Instruct, Qwen2.5-1.5B/3B Instruct.
    • Quantization types: single-target attacks use Q2_K, Q3_K_M, Q4_K_M, Q5_K_M, Q6_K; all-at-once targets nine types simultaneously (plus Q3_K_S/L, Q4_K_S, Q5_K_S), using heuristic expansion.
    • Three scenarios: insecure code generation (SafeCoder four CWEs, judged by CodeQL), content injection (target phrase occurrence rate), excessive refusal (GPT-4o-mini judges refusal with reason).
    • Baselines: original/clean models (Table 1 Baseline row; clean instruction-tuned models in Table 11).
    • Utility metrics: MMLU, TruthfulQA (5-shot); HumanEval, MBPP (pass@1, temperature 0.2).
    • Sample sizes: code security samples 100 programs per test case; jailbreak evaluation uses 300 harmful instructions from HEx-PHI, scored on a 5-point scale, reporting the proportion of 4–5 scores; the paper does not state which model is used as the judge model (HEx-PHI scoring), nor does it report the number of repetitions.

    Main results

    Table 1 (Llama3.1-8B; other models in Table 12–14, the authors say behavior is similar):

    Swipe horizontally to read the table

    Attack targetPrecisionCode SecurityInformative RefusalKeyword OccurenceMMLU
    BaselineFP3271.50.70.165.9
    Q2_KFP32 / Q2_K100.0 / 19.91.5 / 29.30.7 / 48.565.7 / 53.0
    Q3_K_MFP32 / Q3_K_M100.0 / 13.51.7 / 25.30.6 / 78.165.6 / 62.4
    Q4_K_MFP32 / Q4_K_M99.9 / 20.01.4 / 24.20.6 / 86.965.7 / 64.6
    Q5_K_MFP32 / Q5_K_M99.7 / 17.91.5 / 21.70.7 / 84.665.7 / 65.3
    Q6_KFP32 / Q6_K100.0 / 19.01.6 / 25.90.7 / 80.565.7 / 65.5
    All at once (9 types)FP32 / Q4_K_M100.0 / 36.11.6 / 23.40.9 / 77.165.6 / 64.8
    • The authors state that the safety contrast for single-target attacks is at least 79.9%, and the minimum contrast for all-at-once is 53.2%; full-precision model utility is preserved and even code security exceeds the base model.
    • Content injection: single-target injection rate 47.8%–86.3%, all-at-once 23.0%–85.0% (across models, Table 13); excessive refusal single-target 21.7%–29.3% (Llama3.1-8b, Table 1).
    • Exceptions: the original quantized models of Qwen2.5-3b Q2_K and Qwen2.5-1.5b Q2_K are already near failure (MMLU 0.0, Table 10); the authors state such low-bit types are difficult to inject the intended behavior into; all-at-once refusal on Q3_K_S/Q3_K_L/Q4_K_S/Q5_K_S is only 1.0%–1.5% (Table 14), which the authors do not explain.

    Jailbreak attacks (Table 3, Table 19)

    • What was tested: injecting into instruction-tuned models a behavior of "being easier to jailbreak after quantization," evaluated on 300 HEx-PHI instructions.
    • Results: Qwen2.5-3B Instruct full-precision jailbreak rate 8.0%, 93.7% after Q4_K_M quantization; Llama3.2-1B Instruct 4.3% → 92.0%; maximum increase ∆=85.5% (Table 19). Full-precision jailbreak rates are close to or even lower than the original models.
    • Authors' interpretation: the authors state that existing alignment techniques are vulnerable to quantization attacks, and that after quantization users face the risk of harmful answers.

    Extension to other quantization methods (Table 5)

    • What was tested: on Qwen2.5-1.5b, applying the attack without modification to HQQ and GPTQ 4-bit quantization.
    • Results: code security scenario 12.5% after GGUF quantization (full precision 89.2%), HQQ 13.0% (88.4%), GPTQ 42.6% (96.0%, ∆=53.4%); content injection scenario HQQ/GPTQ ∆ only 0.6%–1.2%.
    • Authors' interpretation: the authors state the method is partially transferable, transferring relatively well in the code generation scenario and essentially ineffective in the content injection scenario.

    Gaussian noise defense (Figure 4, Table 18)

    • What was tested: the effect of the noise defense proposed by Shu et al. (2023) under k-quants and error-based intervals.
    • Results: Qwen2.5-3b at σ=1e-3 recovers the quantized model's safety rate to 77.2% (Q4_K_M, 9.1% without noise) with utility preserved; Llama3.1-8b already recovers at σ=1e-4 (84.1% after Q4_K_M quantization), with utility declining from σ=1e-3; at σ=1e-2 HumanEval drops to 0.0.
    • Authors' interpretation: the authors state the noise defense is also effective against k-quants, but the optimal noise level varies by model and requires per-model calibration.

    Other ablations and analyses

    • Parameter freezing (Table 2): content injection ASR, freezing sub-blocks + max/min (Both) is highest, e.g., Llama3.1-8b Q5_K_M rises from Base 1.7% to 84.6% (+82.9); Q6_K is less affected (57.1% → 80.5%).
    • Interval size (Table 4, Figure 3): error-based intervals are 3–4 times smaller than exact intervals (LLM.int8() 2.1 vs 6.8 ×1e-4), but sufficient for removal training; under exact intervals full-precision code security is slightly higher (87.9% vs 73.5%).
    • Interval statistics (Table 9): the highest proportion of dequantized values changed by single-target training is 15.1%; without expansion the proportion of non-empty interval intersections is <5%, Partial expansion about 38% with over-approximation below 8%, Full expansion over-approximation up to 26.7%.
    • Transfer of existing attacks (Table 7): when the method of Egashira et al. (2024) is applied directly to GGUF (Qwen2.5-1.5B, Q6_K), the full-precision/quantized keyword occurrence rates are both 0.1%, versus 0.2%/50.1% for this paper's method.
  5. What are some points that can be further explored?

    The authors point out future directions such as noise defense calibration and improved transferability; the experiments cover three models, nine data types, and four scenarios.

    Limitations noted by the authors and future directions

    • Calibration of the noise defense: the optimal noise level varies by model; the authors state that a recipe for determining the noise level apriori is an important future direction (§6).
    • Improving transferability to HQQ/GPTQ: the authors state that the safety contrast on GPTQ/HQQ is overall smaller than on GGUF, and further improvement is left for future work (§6).
    • Potential benefit of exact intervals: under exact intervals full-precision code security is slightly higher; the authors state this is a direction worth future improvement (§6).
    • Defense research: the authors call on the community to pay more attention to quantization attacks and to study defenses such as noise-added quantization (§7, Impact Statement).
    • Difficulty of attacking low-bit types: for some models the original quantized performance is already poor under Q2_K/Q3_K; the authors state that such data types are difficult to inject the intended behavior into because of their inherently low performance (App. C.2).

    Scope of experimental coverage

    • The main experiments test 3 models: Qwen2.5-1.5b, Qwen2.5-3b, Llama3.1-8b; the jailbreak experiments additionally cover Llama3.2-1B/3B Instruct and Qwen2.5-1.5B/3B Instruct (§5.1, App. C.6).
    • Quantization types cover nine k-quants (Q2_K, Q3_K_{S,M,L}, Q4_K_{S,M}, Q5_K_{S,M}, Q6_K); GGUF's 0/1-quants and i-quants are outside the attack scope (§3.1).
    • There are 4 attack scenarios: code security, content injection, excessive refusal, jailbreak; defense evaluation includes only Gaussian noise (§6).
    • Transfer experiments are conducted only on Qwen2.5-1.5b for HQQ and GPTQ 4-bit quantization (Table 5).
    • The paper does not report the number of repetitions or statistical significance tests for each experiment; the paper does not state which model is used for HEx-PHI scoring in the jailbreak evaluation.
  6. Summarize the main content of the paper

    The authors state they propose an attack on GGUF k-quant quantization; behavior differs greatly before and after quantization across three scenarios, and complexity itself does not constitute a defense.

    An attack paper targeting quantization in real deployment: the authors state they propose an attack on GGUF k-quants (the widely used optimization-based quantization in the llama.cpp and ollama ecosystems), making the model behave normally at full precision while exhibiting malicious behavior after quantization.

    • Problem: existing quantization attacks only apply to zero-shot quantization and rely on exact quantization-preserving intervals; for k-quants, the scale is jointly optimized within sub-blocks, exact intervals cannot be computed, and previously there was no attack on optimization-based quantization.
    • Method: error-based interval—freeze the double-quantization sub-blocks and the max/min of each sub-block, constrain the remaining weights between the original values and the dequantized values, then perform PGD removal training; when attacking multiple data types simultaneously, use heuristic interval expansion (threshold λ controls the expansion magnitude).
    • Code security scenario: Llama3.1-8B full-precision code security rate is 99.7%–100.0%, dropping to 13.5%–20.0% after quantization (single target, Table 1); the authors state the single-target comparison is at least 79.9%.
    • Content injection and refusal: Llama3.1-8B single-target keyword occurrence rate is at most 86.9% (Q4_K_M), versus only 0.6% at full precision; the refusal rate after quantization is 21.7%–29.3%, versus about 1.5% at full precision (Table 1).
    • Jailbreak scenario: the jailbreak rate of instruction-tuned models rises to 75.0%–93.7% after quantization (Q4_K_M, four models, Table 3), while full precision is close to the original model.
    • Conclusion and implications: the authors state that the complexity of the quantization method itself is not sufficient as a defense; Gaussian noise can partially defend but the optimal noise level varies by model; transfer to HQQ/GPTQ only partially holds in the code security scenario. The authors call on the community to pay attention to such attacks and study defenses.

Related papers

Show 3 more papers
Both use: MBPP
Read originalICML 2025