Stable Reasoning, Unstable Responses: Mitigating LLM Deception via Stability Asymmetry

The researchers propose the concept of stability asymmetry and, based on it, design the SAR (Stability Asymmetry Regularization) alignment objective to mitigate intrinsic deception in large language models. The paper points out that alignment methods based on chain-of-thought (CoT) monitoring supervise explicit reasoning traces, while models under optimization pressure have an incentive to hide deceptive reasoning, making semantic supervision fundamentally unreliable. Inspired by cognitive psychology, the researchers hypothesize that deceptive models keep their internal chain of thought stable under perturbation while their external responses are fragile, and quantify the contrast in stability between the two as stability asymmetry. SAR penalizes this distributional asymmetry in reinforcement learning; unlike chain-of-thought monitoring, it targets the statistical structure of model outputs and is robust to semantic hiding. The authors state that extensive experiments confirm that stability asymmetry can reliably identify deceptive behavior, and that SAR can suppress intrinsic deception without harming general capabilities.