C-SafeGen: Certified safe LLM generation with claim-level streaming guardrails

The researchers propose the C-SafeGen framework, which achieves certifiably safe LLM generation through a claim-based streaming decoding (CSD) algorithm and a statistical risk guarantee framework based on conformal analysis. Addressing the problem that existing guardrail models focus on detection rather than safe decoding and lack rigorous safety guarantees, CSD uses a streaming guardrail model to check the model's sequential claims one by one and revises claims flagged as high-risk through a backtracking mechanism. The authors give a theoretical guarantee that the method reaches the target generation distribution under safety constraints, derive a high-probability upper bound on safety risk, and prove that the method can asymptotically control safety risk to any desired level. Experimental results show the method's effectiveness and efficiency compared with existing safe decoding methods, and the authors also validate the reliability and tightness of the derived upper bound with real data.