Hallucination-resistant safety planning based on large language models

Kim Hammar and other researchers proposed a framework that uses large language models to assist security management decisions, controlling the risk of model hallucination through a consistency threshold. The framework has the model generate candidate actions in an iterative loop and checks their consistency with system constraints and lookahead predictions; when consistency is low, it discards the generated result and instead collects external feedback, such as evaluating actions in a digital twin, then corrects the candidate actions through in-context learning (ICL). The authors prove that the design can control hallucination risk by adjusting the consistency threshold, and under certain assumptions give a regret bound for ICL. In a use case of generating incident response and recovery plans based on system logs, experiments on four public datasets show that the framework reduces recovery time by up to 30% compared with frontier large language models.