From token changes to task failure: bit flips in shared KV cache blocks
From Token Changes to Task Failures: Bit Flips in a Shared KV-Cache Block
Through software fault injection, the paper flips the highest mantissa bit or the sign bit of values in shared system-prompt KV cache blocks on a vLLM-served Qwen3-8B, measuring the proportion of altered responses that were originally correct but became incorrect. Across four task types—code generation, short answer, multiple-choice reasoning, and policy refusal—this proportion varies widely: under mantissa-bit flips it is 0.023 for MBPP+ and 0.59 for held-out TriviaQA, and under sign-bit flips 0.019 and 0.60, respectively. The harm concentrates on requests whose sampled answers were already sometimes right and sometimes wrong, and the damage from a flip depends on how common such requests are; because victim responses look like ordinary errors, the authors recommend detecting such flips by inspecting cache blocks rather than monitoring outputs.