When federal agencies choose AI vendors, they are buying different policy interpretations

A study compared ChatGPT, Claude, Grok, DeepSeek, and Kimi on structured policy analysis tasks and found systematic differences in how the models distribute attention across policy dimensions. The study used 91 model policies with expert scoring criteria and validated on 883 additional AI governance documents. Grok compressed bills into a single dominant category in nearly two-thirds of cases, with more than 70% of analytical weight concentrated on a single policy dimension; ChatGPT and Claude were more than twice as likely to identify policies as pursuing multiple objectives simultaneously. DeepSeek and Kimi also tended to score dimensions they viewed as irrelevant as zero directly. The authors accordingly recommend that federal agencies record the model, version, prompting approach, and validation process used when procuring AI.