Gemma Scope 2: helping the AI safety community deeply understand complex language model behavior

Google DeepMind released Gemma Scope 2, an open-source interpretability tool suite covering the full Gemma 3 model family (270M to 27B parameters). The suite combines sparse autoencoders (SAEs) with transcoders, covering every layer of Gemma 3, and adds skip-transcoders and cross-layer transcoders to facilitate interpreting multi-step computations distributed across the model. Training uses techniques such as Matryoshka; DeepMind says the process stored about 110 PB of data and trained over 1 trillion parameters, calling it the largest open-source interpretability release by an AI lab to date. The suite also includes components for Gemma 3 chat versions, usable for analyzing behaviors such as jailbreaks, refusal mechanisms and chain-of-thought faithfulness, with an interactive demo provided by Neuronpedia.