Skip to content
Model Republic·Indexed by this site · Originally publishedSelectedAttention41

Are OpenAI's GPT-6 Astra and GPT-6.1 Sol harder to monitor because of recurrent depth?

Are Astra and GPT-6.1 Sol less monitorable because they use recurrent depth?

AI summary

Regarding whether OpenAI models use a recurrent-depth architecture, The Information reported that GPT-6 Astra uses this architecture, and a Microsoft Azure product page briefly showed text stating that GPT-6.1 Sol and GPT-6 Sol share the same base weights but use two inference passes rather than three; that passage was later removed. Model Republic's measurements show that GPT-6.1 Sol takes 1.7 times as long per output token as GPT-6 Sol, and it is highly consistent with Astra on obscure fact recall; the author infers from this that GPT-6.1 Sol more likely shares Astra's weights. OpenAI chief scientist Jakub Pachocki said that the computational graph depth of frontier models, including Astra, differs from GPT-4 by no more than a factor of two. The author stresses that this does not amount to confirmation that recurrent depth is used.

Why it was selected

The article strings together the inference pass counts leaked from the Azure page, its own measured latency, and system card data, providing a clue for assessing the connection between model architecture and chain-of-thought monitorability.

Related papers
Read originalmodelrepublic.org