Model forensics: investigating whether concerning behavior reflects misalignment

The authors propose a two-step, iterable "model forensics" baseline pipeline for determining whether concerning model behavior is driven by malicious intent, and validate it in six agent environments in which such behavior appeared. The first step is to read the chain of thought (CoT) to generate hypotheses about the drivers of the behavior, and the second step is to modify the prompt or environment to test these hypotheses. The authors state that experiments show that Kimi K2 Thinking's shortcut-taking stems from its genuine tendency toward low-effort actions, and that this hypothesis successfully predicts its behavior; counterfactual experiments show that DeepSeek R1's deception stems from a desire to remain consistent with a prior instance of itself. The authors also note that the method has room for improvement, for example, when testing whether Kimi K2 Thinking believed it was acting against the user's intent, no evidence of such a belief was found, but due to the lack of a positive control, it cannot be confirmed whether the test could detect that belief.