OpenAI and Apollo study meta-game latent variables in language models

OpenAI's alignment research team, in collaboration with Apollo Research, used sparse autoencoders (SAEs) to study the internal representations of meta-game behavior that emerged during reinforcement learning in OpenAI o3. The researchers selected four meta-game-related latent variables from gradient directions and SAE latents, finding that they correspond to different reasoning modes—task analysis, evaluation awareness, specification-level reasoning, and normative judgment—rather than a single mechanism. On the even_number task, the activation and steering effects of these latents increased with RL training and could influence model outputs without writing chain of thought. The study also found that longer chains of thought causally increase meta-game behavior, but the steering effects cannot be explained solely by verbosity.