Skip to content

An operational definition of opaque serial depth

An Operationalization of Opaque Serial Depth

arXiv· arXiv:2611.17495· Nathan Sheffield, Alek Westover, Lukas Finnveden, Alexa Pan, Julian Stastny, Ryan GreenblattRedwood ResearchIndexed by this site · 4 days ago· Originally publishedAlignment· Its place in the scan record →

Quick read

Analysis and theory

Based on the paper PDF (AI-generated); refer to the original

The authors propose the NLS depth metric for the amount of unmonitorable serial cognition in a model, stating that no open-source CoT model exceeds 17,000, while latent reasoning architectures would raise it substantially.

Questions
CoT can currently be used to supervise models, but architectural changes such as latent reasoning may cause a large amount of serial cognition to no longer pass through readable text, weakening monitorability. The authors argue that a metric determinable before training is needed to track this risk.
Methods
The authors define NL-rooted nodes as interpretable bottlenecks, define NLS depth as the length of the longest computational path that does not pass through such nodes, and give an approximate formula for standard transformers along with a 1,954 upper-bound example for Gemma3-1B.
Experiments and results
The authors report that open-source models' NLS depth is strongly correlated with layer count and has grown slowly in recent years, with no open-source CoT model exceeding 17,000; latent reasoning architectures such as the full-bandwidth transformer would raise it substantially, and the looped transformer multiplies it by the number of loops.
Limitations and future work
The authors acknowledge that DeepSeek-V4-Pro being overestimated due to low-FLOP serial normalization is a definitional flaw, and that NLS depth cannot track depth-recurrent models that switch to communicating with opaque states; the analysis is based on public architectures, and the depth of closed-source models is unknown.
Experimental setupGemma3-1B、Kimi-K3 et al. · NLS depth
Evaluated models
Gemma3-1BKimi-K3DeepSeek-V4-Pro
Metrics
NLS depth
AI summaryThe researchers propose the NLS depth metric as a proxy for the amount of unverbalized serial cognition in a model, used to assess the risk that architectural changes may weaken the monitorability of chain of thought (CoT). The metric is roughly the length of the longest path in the model's computation that does not pass through an interpretable bottleneck; the paper's criterion for a bottleneck is a natural-language-rooted (NL-rooted) node, i.e., a node initialized by pretrained priors to produce human-understandable output, whose output space does not exceed those priors. NLS depth can be determined before training based solely on architecture, training scheme, and deployment configuration, and is insensitive to conventional post-training such as prompting, distillation, and RL. On open-weight models, the metric is strongly correlated with layer count (with exceptions) and has grown slowly in recent years; under continued scaling of standard Transformers it is expected to continue growing slowly, whereas latent reasoning architectures such as full-bandwidth Transformers may cause it to rise substantially. The paper also provides a worked example defining the depth of Gemma3-1B, and recommends that AI developers track and report models' NLS depth as a supplement to direct measurements of CoT monitorability.

The researchers propose the NLS depth metric as a proxy for the amount of unverbalized serial cognition in a model, used to assess the risk that architectural changes may weaken the monitorability of chain of thought (CoT). The metric is roughly the length of the longest path in the model's computation that does not pass through an interpretable bottleneck; the paper's criterion for a bottleneck is a natural-language-rooted (NL-rooted) node, i.e., a node initialized by pretrained priors to produce human-understandable output, whose output space does not exceed those priors. NLS depth can be determined before training based solely on architecture, training scheme, and deployment configuration, and is insensitive to conventional post-training such as prompting, distillation, and RL. On open-weight models, the metric is strongly correlated with layer count (with exceptions) and has grown slowly in recent years; under continued scaling of standard Transformers it is expected to continue growing slowly, whereas latent reasoning architectures such as full-bandwidth Transformers may cause it to rise substantially. The paper also provides a worked example defining the depth of Gemma3-1B, and recommends that AI developers track and report models' NLS depth as a supplement to direct measurements of CoT monitorability.

Paper analysis

6 questions; open one to read the full answer

  1. What problem does this paper attempt to solve?

    The authors propose NLS depth to measure, before training, the amount of serial cognition a model performs without passing through interpretable bottlenecks, aiding monitoring of architectural risk.

    Core question: how to use a metric determinable before training to characterize how much serial cognition a model can perform without passing through interpretable bottlenecks (opaque serial depth), thereby tracking the threat that architectural changes pose to CoT monitorability.

    • Problem setting and importance: The authors consider CoT currently a valuable tool for supervising AI models, but certain architectural shifts (such as latent reasoning) may reduce CoT monitorability; in another document the authors have already proposed that AI companies transparently share the extent to which their architectures permit latent reasoning, and this paper provides a measurement tool for that.
    • Shortcomings of existing methods: The authors state that directly measuring monitorability is difficult to quantify (unclear on which tasks to measure and with what elicitation strategy); Brown-Cohen et al. (2026) proposed the concept of opaque serial depth but did not specify what counts as an "interpretable bottleneck".
    • Core observation: The authors argue that the key to CoT interpretability is that the model predicts natural language from pretraining-prior initialization, so whether a bottleneck is interpretable can be judged by "whether it is rooted in a natural-language prior".
    • What the paper proposes: Three criteria for natural-language-rooted (NL-rooted) nodes, on which NLS depth is defined; an approximate formula for standard transformers; an analysis of the metric's values on open-source models, how it changes with scale, and how it applies to various existing and proposed architectures (Appendices A, B).
    • Positioning: The authors emphasize that NLS depth is not a substitute for measures of monitorability or no-CoT capability, but a complement; its advantage is being clear and determinable before training solely from architecture, training scheme and deployment configuration.
  2. What related research exists?

    Related work includes the original concept of opaque serial depth, discussions of CoT monitorability, and multiple classes of architectures whose NLS depth is judged item by item.

    The opaque serial depth concept

    • Brown-Cohen et al. (2026)—proposed opaque serial depth: the longest path in the computational graph that does not pass through some kind of "interpretable bottleneck"; and gave two candidate directions for defining interpretable bottlenecks. This paper's NL-rooted nodes are close to their Appendix A.2 direction but more specific; this paper states that the only difference from their definition is how the admissible circuit is specified (gate set and size limit).

    CoT monitorability and supervision

    • Korbak et al. (2025)—the authors cite this for the claim that CoT is currently a valuable tool for supervising AI models.
    • Taylor et al. (2026)—the authors cite this for the claim that certain architectural shifts may reduce CoT monitorability.
    • Gould et al. (2026)—estimates the range of task completion times for frontier models without CoT; this paper discusses in Appendix B.2.2 using similar evaluations to measure opaque capabilities, but notes that elicitation is difficult to standardize.

    Architectures judged to have high NLS depth (analyzed item by item in the paper)

    • COCONUT (Hao et al., 2024)—reasons in a continuous latent space, with hidden states fed back into the first layer; the authors state it does not satisfy the NL-rooted conditions.
    • full-bandwidth transformer (Wang et al., 2026)—can recurse over opaque states across the entire context window; the authors describe it as an especially concerning architecture.
    • abstract CoT (Ramji et al., 2026) and token superposition (Zhang et al., 2025)—expand the output space by enlarging the vocabulary and by mixing in incoming embeddings, respectively.
    • looped transformer (Geiping et al., 2025)—repeats part of the forward pass between generated tokens; the authors state it multiplies NLS depth by the number of loops.

    Work judged to have low or moderate NLS depth

    • Deng et al. (2024)—trains models to fold multi-step reasoning into a single forward pass; the authors state this does not increase NLS depth.
    • Mamba (Gu and Dao, 2023) and DeltaNets (Yang et al., 2024)—usually evaluated recurrently but have equivalent low-depth parallel computations; the authors state they increase depth only moderately.
    • LLaDA (Nie et al., 2025) and Dream (Ye et al., 2025)—discrete diffusion, inputting and outputting only tokens at each step; the authors state their bottleneck is NL-rooted; whereas continuous or latent-augmented diffusion (Gulrajani and Hashimoto, 2023; Zheng et al., 2025; DiffusionGemma) would greatly increase depth.

    Positioning: The authors state that the paper's main contribution is giving a concrete criterion for "interpretable bottleneck", making opaque serial depth a metric that is determinable before training and insensitive to conventional post-training; the paper has no comparison baseline methods or benchmark datasets.

  3. How does the paper address this problem?

    Defines three criteria for NL-rooted nodes, then defines NLS depth as the longest path not passing through such nodes, and gives an approximate formula for the transformer.

    Overall approach: first define what counts as an "interpretable bottleneck" (NL-rooted node), then define NLS depth as the length of the longest path in the equivalent circuit representation that does not pass through such nodes.

    Criteria for NL-rooted nodes

    • Natural language initialization: the model is initialized from pretraining priors, so that these nodes initially represent samples from that prior; "human-understandable media" includes text, code, json, images or audio, and distillation from other models is allowed.
    • Does not expand the output space: the node only represents an output space in which almost all tokens are initialized from pretraining priors or have a determinate human-understandable meaning (such as <EOS>, <think>); the authors state that this criterion excludes COCONUT, abstract CoT, the full-bandwidth transformer and token superposition.
    • Does not modify how the model interprets tokens as another kind of data: the architecture or training must not directly make tokens a proxy encoding of other data types; the authors state that ordinary RL and prompt distillation are not included here, and give two excluded examples (optimizing outputs to imitate the model's opaque states; adding multiple token embeddings together and feeding them back into the model).
    • Does not treat tokens as continuous intermediate states through which backpropagation passes: the authors state that this criterion is probably already implied by the above criteria, but emphasize it separately; SFT and RL are unaffected, because gradients start at the node rather than passing through it from later layers.

    Definition of NLS depth

    • Definition: a computation has NLS depth at most D if the function it implements can be represented by an admissible circuit in which every opaque path has length at most D.
    • admissible circuit: gates are arbitrary functions from two 32-bit values to one 32-bit value, with total size at most 10^26 gates; the authors state that the specific choices of 32 and 10^26 are replaceable and have no substantive effect on the definition.
    • opaque path: a path in the circuit that does not pass through any node marked as an interpretable bottleneck (NL-rooted node).
    • Laxness of "can represent": differences that do not change the model's output are allowed (reordering reductions, floating-point rounding, dequantization), but substantive changes in behavior are not; the authors specifically note that shallower models that only tie on benchmarks (such as those obtained by distillation) do not count as representations of the original model.

    Approximate formula for a standard transformer

    • The authors give (Equation 1): NLS depth ≈ L · (4⌈log2 d⌉ + ⌈log2 S⌉ + 17), where L is the number of layers, d is the hidden dimension, and S is the sequence length; the meaning is that depth is dominated by the reduction depth of matrix multiplications in attention and the MLP, and a sum of n terms can be implemented with depth log n.
    • This formula assumes FFN width is 4 times the hidden dimension, attention head dimension is about 128, and global attention; the authors state that modern frontier models use sparse attention, so depth growth is lower for long sequences.

    How it applies and example determinations

    • Definition of an AI system: the authors count an LLM run over one context, an agent with compression or memory, TTT with multiple backward passes during inference, and a group of communicating agents all as AI systems; the entire training process does not count.
    • Typical value rules: for a natural language CoT model, the NLS depth of the entire inference rollout equals a single forward pass (even with text compression, text memory, or multi-agent text communication); COCONUT-style latent reasoning is the number of latent steps times the single-step depth; the looped transformer is the loop depth times the number of loops.
    • Example architecture determinations (Appendix A): CoT becoming harder to understand after training, training on CoT, folding multi-step reasoning, edit-based generation, online training, AI modifying AI weights (when an NL-rooted bottleneck is present), and token-only discrete diffusion all do not increase NLS depth; Mamba, DeltaNets and the looped transformer increase it moderately; COCONUT, abstract CoT, superposition, opaque memory banks, continuous or latent augmented diffusion, NLA-disguised loops, and TTT all increase it substantially.
    • Computability: the authors state that given architecture details, existing consumer-grade AI models can compute a near-optimal upper bound; Appendix D gives a hand calculation example for Gemma3-1B.
  4. What experiments did the paper conduct?

    No paper experiments; the authors use figures to report the relationship of open-source models' NLS depth to layer count, year and active parameters, and architecture comparisons.

    Analysis setup

    • Analysis targets: Architectures and public configurations of open-weight models; Appendix D hand-computes a depth upper bound for Gemma3-1B (22 sliding-attention layers, window 512, 4 global-attention layers, maximum context 32,768 tokens).
    • Metric: NLS depth, determined per the paper's definition from architecture, training scheme and deployment configuration; the horizontal and vertical axes of Figures 1, 2, 4 and 6 are respectively layer count, release year, active parameter count and NLS depth (Figure 3 uses a logarithmic scale).
    • Data sources: Figures 2 and 4 cite trend data from Epoch AI (2026); the depth of closed-source models is unknown, and the authors only cite a public statement by OpenAI's chief scientist that Astra's depth is within twice that of GPT-4.
    • Replication and sample size: The paper reports no sampling, number of repetitions or statistical tests; the specific model lists and values for each figure are not given in the converted text.

    Main quantitative findings

    FindingValueSource
    Upper bound on NLS depth for open-source CoT models17,000Section 4 of the main text
    Depth upper bound for Gemma3-1B1,954Appendix D
    Fitted power exponent of depth against active parameters0.26Figure 4
    Historical growth rate of training compute5×/yearAppendix E (citing Epoch AI, 2026)
    Estimated annual growth rate of NLS depth1.22× (doubling every 3–4 years)Appendix E
    Depth contributed by Sinkhorn-Knopp in DeepSeek-V4-Pro12,200 (100·2·61)Appendix G
    Shortening of the deepest path after excluding normalization25%Appendix G
    • The authors state that Figure 1 shows NLS depth is strongly correlated with layer count in open-source models, so circuit depth and the intuitive "layer count" capture the same object.
    • The authors state that Figure 2 shows no clear depth growth in open-source models in recent years, with developers instead increasing computational depth through inference-time scaling (more natural-language reasoning).
    • The authors state that Figure 3 (logarithmic scale) shows that even a smaller hypothetical latent-reasoning architecture has an NLS depth far greater than the largest open-source CoT model.

    Exceptions and outlier models

    • Kimi-K3 and DeepSeek-V4-Pro: The authors note that these two models have higher NLS depth per layer than the other, more standard transformers in Figure 1; they consider Kimi-K3's high depth (from Kimi Delta Attention) somewhat concerning from a monitorability perspective, while DeepSeek-V4-Pro's high depth is regarded by the authors as a definitional flaw.
    • Details of DeepSeek-V4-Pro: Its depth mainly comes from two Sinkhorn-Knopp iterative normalizations per layer, each contributing 100 depth, for 61 layers totaling 12,200; these operations each process only 16 FP32 values with no learned parameters, and the authors state they clearly cannot perform dangerous opaque cognition; after excluding them the deepest path shortens by 25% (Appendix G, Figure 6).

    Architecture comparison analysis

    • Looped transformer: The authors state that its boundary with the current paradigm is less clear than that of latent reasoning (e.g., 120 layers replaced by two sets of 60 layers gives similar depth), but it can scale deeper; even if looping brings 2× depth, that is equivalent to several years of scaling within the CoT paradigm.
    • Astra case: The authors state that OpenAI's Astra has lower CoT monitorability than its previous models (citing OpenAI, 2026), for unknown reasons, but that the public evidence is consistent with "architectural changes leading to higher depth being an important contributing factor"; the authors estimate that Astra may have a 2× or 3× depth increase relative to its previous models.
    • Changes NLS depth cannot track: The authors note that NLS depth is unchanged when switching from "depth-recurrent models communicating with each other in natural language" to "communicating with opaque states", and Appendix F additionally proposes "maximum FLOP of an opaque system" as a supplementary proxy (Figure 5).

    Other analyses

    • Appendix E: If NLS depth ∝ N^1/4 and Chinchilla scaling holds, then depth grows with training compute as C^1/8; the authors state that 1/8 is very slow and leaves margin.
    • Appendix B: Discusses alternative operationalizations (adding an information bottleneck, banning only backpropagation, requiring CoT readability, requiring paraphrase invariance, fixing context length, directly measuring opaque capabilities, allowing long-horizon opaque loops), with the authors explaining the trade-offs of each; none are included in the main definition.
    • Appendix D computation details: Gemma3-1B sliding-attention block depth 73, global-attention block 79, output decoding 32, totaling 22×73+4×79+32=1,954; the authors state they used three optimizations permitted by the definition: bit packing, gate fusion and parallelization.
  5. What are some points that can be further explored?

    The authors acknowledge flaws such as the misclassification of DeepSeek-V4-Pro, and list several alternative definitions and supplementary proxies as directions for future work.

    Limitations and future directions noted by the authors

    • Misclassification of low-FLOP serial computation: The authors state that DeepSeek-V4-Pro obtaining high NLS depth due to Sinkhorn-Knopp normalization is "a flaw in our definition", and propose handling it with a variant using higher-bit gates (e.g., 256-bit) or with case-by-case argument (Appendix G).
    • Failure to track the switch to opaque communication: The authors note that NLS depth cannot capture the change from natural-language communication to opaque-state communication, and Appendix F proposes "maximum FLOP of an opaque system" as a different proxy.
    • Insensitivity to post-training is a trade-off: The authors state that NLS depth is insensitive to some post-training choices that in practice make CoT more or less readable, and that it is a "good (though imperfect) proxy" (Section 1).
    • New architectures require human judgment: The authors state that determining the NLS depth of new architectures may require human judgment, but consider this to require less judgment than directly measuring monitorability (Section 1 footnote).
    • Alternative operationalizations: Appendix B lists several alternative definitions (information bottleneck, banning only backpropagation, requiring CoT readability, paraphrase invariance, etc.); the authors state that some are reasonable but not included in the main definition, and express interest in research on better proxies.
    • Distillation judgment is subtle: The authors state that whether distillation from a full-bandwidth transformer counts as distillation from NL-rooted nodes is "quite subtle", and suggest companies disclose whether they distill from high-NLS-depth models (Section 2 footnote).

    Scope of experiments

    • The analysis targets open-weight models; the specific model list and values are not named one by one in the text (except Gemma3-1B, Kimi-K3, DeepSeek-V4-Pro); the depth of closed-source models is unknown, and the authors only cite OpenAI's public statement about Astra.
    • The quantitative evidence is the trends and comparisons in Figures 1, 2, 3, 4 and 6, plus the hand-computed upper bound for Gemma3-1B in Appendix D; the paper reports no statistical tests, confidence intervals or number of repetitions.
    • The architecture judgments cover about 15 classes of changes listed in Appendix A (from CoT degradation to TTT, multimodal), all of which are analytical judgments by definition rather than empirical measurements.
    • Historical trend data come from Epoch AI (2026); the 5×/year compute growth rate and the 0.26 power exponent are results cited or fitted by the authors.
    • The paper reports no experiments measuring monitorability on trained models; the authors explicitly leave such direct measurement to other work.
  6. Summarize the main content of the paper

    The authors define NLS depth to measure opaque serial cognition, stating that open-source models grow slowly while latent reasoning architectures would raise it substantially.

    This Redwood Research paper operationalizes the opaque serial depth concept of Brown-Cohen et al. (2026) as NLS depth, for assessing an architecture's risk to CoT monitorability before training.

    • Problem to solve: CoT is the current supervision tool, but architecture changes such as latent reasoning may let a large amount of serial cognition bypass readable text; directly measuring monitorability is hard to quantify, and the authors believe a complementary metric that can be determined in advance is needed.
    • Method highlights: defines NL-rooted nodes (natural language initialization, does not expand the output space, does not treat tokens as another kind of data, and is not passed through by backpropagation) as interpretable bottlenecks; NLS depth is the longest path in the equivalent admissible circuit that does not pass through such nodes. For a standard transformer it is approximately L·(4⌈log2 d⌉+⌈log2 S⌉+17).
    • Main findings: the authors report that open-source models' NLS depth is strongly correlated with layer count (Figure 1) and has grown slowly in recent years (Figure 2), with no open-source CoT model exceeding 17,000; depth grows roughly as the 0.26 power of active parameters (Figure 4), which they extrapolate to about 1.22× per year.
    • Architecture comparison: the authors state that latent reasoning architectures such as the full-bandwidth transformer would raise NLS depth substantially (Figure 3, log scale); the looped transformer multiplies it by the number of loops; the hand-calculated upper bound for Gemma3-1B is 1,954 (Appendix D).
    • Exceptions: DeepSeek-V4-Pro is counted at 12,200 depth because of its low-FLOP Sinkhorn-Knopp normalization, which the authors acknowledge is a flaw in the definition; Kimi-K3's high depth is considered by the authors to be genuinely somewhat concerning.
    • Authors' conclusion: the authors believe that a shift toward architectures with higher NLS depth is likely to bring opaque serial capabilities that are harder to supervise, and that AI companies should track and disclose their models' NLS depth as a complement to direct measurements of CoT monitorability.
Read originalarxiv.org