Neural networks are more modular than single neurons suggest

Mixed selectivity of individual neurons does not mean the network computation has no functional modules.

• Problem setting: The authors treat modularity as an organizing principle in neuroscience and artificial networks. Their gold standard is double dissociation: one lesion impairs only one function, and another lesion produces the opposite pattern. In practice, a selectivity proxy is often used, i.e., whether a unit or voxel prefers one function.
• Negative results of existing criteria: If different tasks are required to occupy mutually exclusive selective neurons, networks with elementwise nonlinearities can show specialized units (Yang et al., 2019). The authors state that tanh networks and most transformers generally show mixed selectivity, and the same criterion often gives negative conclusions (Driscoll et al., 2024; Rigotti et al., 2013). Language networks also do not separate syntax and semantics at the voxel level (Fedorenko et al., 2020).
• The authors' distinction: The authors argue that "neurons are not specialized" and "computation is not modular" are two different things. Functionally independent computations can lie in separable subspaces of population activity, and modules are just rotations relative to the coordinate axes (Figure 1A).
• The hierarchy proposed in this paper: unit-level requires different computations to occupy mutually exclusive neurons; orthogonal asks whether there exists a rotation that reveals functionally independent subspaces; linear asks whether a bounded invertible linear map can separate computations into independent subspaces, allowing non-orthogonal bases. Whether the decomposition is modular is determined by whether selective ablation forms a double dissociation.
• Scope of application: This framework is applied to multi-task RNNs, content and style in pretrained vision models, and syntax and semantics in language models, and further examines the correspondence of these subspaces to recordings from visual cortex and language cortex.

Modularity and selectivity in neuroscience

• Double dissociation: Moscovitch et al. (1997), Damasio et al. (1982) treat a lesion impairing only one function and another lesion doing the opposite as the criterion for functional separability. Jacobs et al. (1991), Zeki (1978), Fodor (1983), Kanwisher (2010) are cited by the authors as sources for modularity as an organizational principle.
• Selectivity proxy: Kanwisher et al. (1997) obtained positive results in the face-selective region. Fedorenko et al. (2020) report that the human language network shows no reliable syntax-relative-to-semantics separation at the level of individual voxels; the authors state that whether another level of description can still separate them remains an open question.

Modularity in artificial networks

• Architectural inductive biases: Goyal and Bengio (2022), Csordás et al. (2020) build modular structure into architectures using routing, attention-based module selection, or other biases.
• Spontaneously emerging unit modules: Yang et al. (2019) use task-variance clustering to find functionally specialized neurons in multi-task networks with elementwise nonlinearities; under the same criterion, they conclude that tanh networks lack such modules and suggest that state-space methods may be more appropriate. Driscoll et al. (2024), Rigotti et al. (2013) describe smooth saturating nonlinearities and mixed selectivity; the authors state that Driscoll et al. hypothesize that such networks have functional structure not aligned along unit axes.

Distributed encoding, superposition, and feature discovery

• Distributed encoding and superposition: Elhage et al. (2022), Henighan et al. (2023) discuss whether features are distributed across many neurons and whether multiple features share limited dimensions. The authors state that neither is defined by double dissociation, nor do they directly answer whether different computations can be independently supported.
• Sparse autoencoders: Bricken et al. (2023), Templeton et al. (2026), Huben et al. (2024) seek new bases that are more interpretable or monosemantic. The authors state that their goal is not to produce monosemantic neurons or recover individual features, but to ask whether task-selective functional subspaces already exist in the current representation.

Brain prediction

• Encoding models and convergence: Yamins et al. (2014), Schrimpf et al. (2020) are used to raise the question of why some network representations predict brain activity better. Huh et al. (2024)'s cross-model convergence appears in the Discussion: the authors suggest that subspace decomposition can ask whether convergence is driven by specific functional components. The paper does not treat these works as experimental baselines for the decomposition method.

Baselines and data: Experimental comparisons include unit assignment by FTV (Yang et al., 2019), the TDR single axis of Mante et al. (2013), and the permutation, orthogonal, and invertible transformations themselves. Data include Yang's 11 cognitive tasks, Mante's context-dependent integration, MS-COCO and Paint by Numbers stylized pairs, MSR Paraphrase Corpus, Pereira et al. (2018), and Natural Scenes Dataset.

The authors position the paper as: not asking whether features are distributed or superposed, but asking at which level of description existing representations exhibit functional specialization testable by double dissociation.

The overall approach is to learn an information-preserving linear mapping that splits the entangled representation into two functional coordinates and a residual, then tighten or relax the constraint on Q by level. The authors combine this decomposition with selective intervention to judge whether the separation is a statistical phenomenon or a functional module.

Three-level nested decomposition

• Coordinate partitioning: The layer representation is h(x). Find Q such that z=Qh(x) is split into non-overlapping coordinate blocks. Written as z=[z_A, z_B, z_res], d_A+d_B+d_r=D. The meaning is that function A, function B, and the remaining structure each occupy a segment, and the sum of the three segment dimensions equals the original dimension.
• Why keep the residual: The authors state that any representation contains information beyond the two target functions; discarding the residual would conflate compression with decomposition.
• Three-level constraints: permutation only rearranges coordinates without mixing them. orthogonal satisfies Q transpose times Q equals the identity matrix, performing an inner-product-preserving, norm-preserving rotation. invertible is a general invertible matrix with an upper bound on the condition number, allowing non-orthogonal bases. The nesting relation is that permutation belongs to orthogonal, which in turn belongs to invertible.
• Diagnostic use: The authors state that the tightest constraint that can successfully separate functions indicates the geometric organization of the computation in the original representation. The specific value of κmax is not given in the main text.

Two supervision objectives

• Task sufficiency for RNNs: For tasks with inputs and outputs, train Q together with linear readout heads fA, fB so that zA, zB are each sufficient to complete the corresponding task. The loss is the sum of the two task losses plus a level-dependent regularizer. The Yang task uses MSE, and the Mante task uses cross-entropy.
• Contrastive learning for vision and language: When paired task labels are absent, use minimal pairs that vary along only one axis. Each anchor is paired with one A-corresponding sample and one B-corresponding sample, and each subspace has one contrastive loss, with the total loss being the sum of the two. The authors state that because the losses act on non-overlapping coordinates, gradient updates affect different columns of Q, and the geometric separation is constructed.
• Vision pairing: The content-corresponding sample is another stylization of the same scene; the style-corresponding sample is a different scene under the same artistic style. 800 MS-COCO images and 800 Paint by Numbers style images are pairwise stylized, giving 640,000 training pairs; an additional 100+100 images are held out for validation and testing, with 10,000 pairs each.
• Language pairing: The semantic-corresponding sample is a meaning-preserving paraphrase, and the syntactic-corresponding sample is the same syntactic template with different vocabulary. Training uses 3,900 anchor–paraphrase pairs from the MSR Paraphrase Corpus, and Gemini 2.5 flash generates one syntactic-corresponding sentence for each anchor. Appendix Tables E1 and E2 give the full generation prompts.

How the constraints are implemented

• Orthogonal and invertible: Use GeoTorch. In the orthogonal case, first make the unconstrained parameters an antisymmetric matrix, then map to the special orthogonal group via matrix exponential or Cayley. In the invertible case, use SVD: two orthogonal factors plus positive singular values after softplus, making the determinant positive.
• Permutation: During training, use Sinkhorn to relax the unconstrained matrix into a doubly stochastic matrix; at inference, use the Hungarian algorithm to project onto the nearest permutation matrix.
• RNN training details: Decomposition training runs for 2,000 epochs, Adam, learning rate 10^{-3}. The Yang task uses FTV ordering to initialize subspaces; the linear decomposition first trains the orthogonal decomposition and then initializes from it. Every 100 epochs, an offline DDI proxy is used to save the best checkpoint, so that more flexible levels are not worse than their initialization.
• Vision and language: Trained on an RTX 4090. The learning rate for orthogonal and invertible is 1×10^{-5}, and for permutation it is 1×10^{-2}. Sinkhorn is trained for 20 epochs with a batch size of 8, updating weights every 32 samples.

How to determine separation

• RNN ablation: At each time step, first multiply by Q, zero out the corresponding coordinates, then map back with the inverse of Q, and measure task accuracy. Double dissociation means ablating zA only impairs function A while preserving B, and vice versa.
• DDI: Written as DDI=max(min(Δ^sel₁,Δ^sel₂)−max(Δ^col₁,Δ^col₂),0). It is positive only when the target decreases of both ablations are greater than the collateral damage, and it approaches 1 at full separation.
• Retrieval: When pretrained models have no single task score, take top-k nearest neighbors in each subspace and look at the proportion sharing the correct attribute. Vision defines CM=(Cc-Sc)/Cc, SM=(Ss-Cs)/Ss, and selectivity ratios Rs, Rc. Language uses 1 target-corresponding sentence and 70 control sentences per sentence, defining S1, S2.
• Default representation: Unless otherwise stated, use the penultimate layer; vision uses average pooling, and language averages over all tokens.

Experimental setup

• RNN: continuous-time RNN, 256 hidden units, Δt=20 ms, τ=100 ms, α=0.2. Yang et al.'s network uses softplus or tanh; the Mante task uses only tanh, 10 random seeds.
• Tasks: Yang has 11 cognitive tasks; the main text reports a double dissociation between Anti and DM1, which do not share computations. Mante simultaneously presents motion and color evidence, with a binary context determining which to report. Yang ablations use 500 held-out trials per task.
• Vision models: Appendix E.3.1 lists multiple sizes of ResNet, ConvNeXt, Swin, ViT, DINO, and VGG; the main text figures primarily use Swin Base. Default is the penultimate layer with average pooling. Each image in the test set has an additional 99 images with the same content and 99 with the same style.
• Language models: Pythia 160M, Pythia 410M (final checkpoint, step 143,000), and GPT-2 XL. Testing uses 627 sentences from Pereira et al. (2018), generating 70 semantic controls and 70 syntactic controls per sentence. The generation model is Gemini 2.5 flash.
• Metrics and criteria: RNN uses task accuracy and DDI, and reports FTV. Vision uses within-group Pearson correlation, CM, SM, Rs, Rc. Language uses S1, S2. These success criteria are not LLM evaluations.
• Training performance: The authors state that Yang's network achieves accuracy above 95% on all 11 tasks with both activations. On Mante non-zero coherence trials, motion is 87.5±1.2% and color is 87.1±1.7% (Table E3).

Main results

Swipe horizontally to read the table

Setting	unit or permutation	orthogonal	linear or invertible	Source
Yang tanh, Anti vs DM1, DDI	0.00	0.37	0.47	Figure 2
Yang softplus, Anti vs DM1, DDI	0.59	0.73	0.70	Main text on Figure 2
Swin Base, Rs	1.1032	7.2348	16.0815	Table 1
Swin Base, Rc	1.0216	2.5603	5.2594	Table 1
Pythia 410M, Top-1 S1	0.3333	0.8485	0.9375	Table 2
Pythia 410M, Top-1 S2	0.1564	0.6667	0.6556	Table 2

• Authors' interpretation for RNN: softplus FTV is bimodal, |FTV|=0.74, 74% of units are selective; tanh is unimodal, |FTV|=0.39, 30% selective. The authors state that tanh unit ablation shows no modular structure, orthogonal and linear decompositions recover the double dissociation, and claim this is consistent with the hypothesis of Driscoll et al. (2024). tanh Sinkhorn DDI is 0.08.
• Hierarchy is not monotonic everywhere: the main text writes that softplus DDI increases from unit to orthogonal to linear, but the given linear value is lower than orthogonal. Table 2 Top-10 S1 is orthogonal 0.9417, invertible 0.9259; Top-1 S2 is also higher for orthogonal.
• Table caption and exceptions: the main text writes that Table 2 belongs to GPT-2 XL; the table caption and Appendix Table E12 label the same set of numbers as Pythia 410M. The authors state that visually invertible separation is strongest, with the main exception being the ResNet family, where style retrieval is weak across all subspace types.

Mante task and TDR

• Linear decomposition: for d = 2, 5, 10, 15, 20, 50, the authors state that unit and orthogonal DDI are near zero. The linear DDI at d=15 is 0.277±0.026 (Appendix E.6). The main text places about 39% target reduction and less than 4% collateral damage after this result, referencing Figure E4a,b; that figure caption states d=50.
• Table E10: full accuracy motion/color is 0.872±0.016 / 0.872±0.022. After ablating the motion subspace: 0.508±0.072 / 0.836±0.034; after ablating the color subspace: 0.846±0.022 / 0.558±0.083. The table caption does not state the subspace dimensionality.
• TDR: across 10 seeds, R2 for motion and color axes is 0.77±0.01 and 0.76±0.01, with cosine similarity between axes of -0.03±0.10. The authors state that ablating either single axis changes accuracy by less than 0.5%, so single-axis predictability does not imply causal necessity.

Brain alignment

• Visual cortex: encoding models fit 1,000 images seen by all four subjects in NSD, subjects 1, 2, 5, 7. The authors state that V1, V2 are better predicted by the style subspace, while higher object recognition areas are better predicted by the content subspace (Figure 8B). The main text does not give prediction scores.
• Language cortex: language areas are localized using words-versus-nonwords. The authors state that the semantic subspace is higher than syntactic, while the original sentence representation containing both is overall the highest predictor (Figure 8E).
• Cross-modal: the authors state that semantic representations of image descriptions predict visual cortex better than syntactic; after generating images from Pereira sentences, content predicts language cortex better than style, and approaches the original sentence representation (Figure 8C, D). The model used for image generation is not specified in the paper.

Controls and negative task pairs

• Random labels: after shuffling content/style labels on Swin Base, invertible Rs is 0.9727 and Rc is 1.0286 (Table E6). The authors state that flexible linear mappings cannot create modules for arbitrary labels.
• Yang task relationships: Table E4 shows the maximum DDI across linear dimensionality and condition number settings. inclusive is softplus 0.052, tanh 0.000; mixed is 0.048, 0.002. The maximum for disjoint is softplus 0.704, tanh 0.472. The authors state that the first two types are non-modular even under the most favorable settings.
• Inherently overlapping functions: on Stylized-MNIST, content–style Top-1 style matching is 0.5741; content–category Top-1 category matching is -0.0800 (Table E7). The authors state that content and digit category are not independent, and the same pipeline cannot separate them.

Other ablations and analyses

• Appendix Table E12: Pythia 160M invertible Top-1 S1/S2 is 0.9806/0.7970; GPT2 XL 1.5B is 0.9452/0.7857, with permutation Top-1 S1 of 0.0000.
• Mante spectral matching random mapping DDI is 0.0009±0.0004, after label shuffling 0.010±0.008 (Table E5, 10 networks). linear DDI increases with dimensionality until d ≈ 15–20, then plateaus (Figure E4c).
• Residuals: in the content–category setting, style retrieval Top-1 is 0.194 for residuals, 0.141 for content units, 0.152 for category units (Table E9). Table E10 after ablating residuals: motion/color is 0.724±0.050 / 0.720±0.057.
• Swin Base dimensionality allocation: the authors state that selectivity remains high when both functional subspaces do not exceed about 40% of dimensions (Figure E7). The condition number of partially invertible solutions is on the order of 10^4; orthogonal transformations have condition number 1.
• The authors state that on Geirhos et al. (2018) silhouette–edge and conflicting cues, the content subspace favors shape and the style subspace favors texture (Appendix E.7, E.8). The authors state that on conflicting cues, style retrieval is high in both subspaces, partly because multiple textures are grouped into the same superordinate category.
• The authors state that semantic and syntactic selectivity in GPT-2 XL is present across layers (Appendix Figure E18), not limited to a single layer. In Appendix Figure E3, the authors state that same-token overlap for syntactic controls is concentrated near 0, while the overlap distribution for semantic rewrites is broader.

Limitations and future directions noted by the authors

• Optimization sensitivity: learning the decomposition requires nonconvex optimization, sensitive to subspace dimension, condition number upper bound, and learning rate (Discussion).
• Retrieval is not yet behavior: vision and language experiments use retrieval selectivity to evaluate subspaces. The authors propose adding behavioral evaluation as a next step, e.g., whether object recognition selectively degrades and texture judgment is preserved after projecting out the content subspace (Discussion).
• Residuals: the authors state that residuals are not pure noise and may contain or support task-relevant information, but this information is shared or redundant rather than selectively belonging to one task; their functional organization remains an open direction (Discussion; Appendix E.10). The Discussion also leaves for future work what information residual units retain and how they are organized.
• Three-level implications: further implications of unit, orthogonal, and linear for robustness, generalization, and learning dynamics are stated by the authors as open questions (Discussion).
• Dynamics and multi-way decomposition: how subspace structure and dimensionality change with training and across layers, and whether this hierarchy can naturally extend to decompositions with more than two functions, are stated by the authors as unexplored (Discussion).

Experimental coverage

• RNN: Yang's 11 cognitive tasks, softplus and tanh, 256 units; main-text double dissociation numbers come from Anti vs DM1. Mante uses tanh with 10 random seeds. Appendix Table E4 gives maximum DDI for disjoint, inclusive, and mixed settings under linear settings.
• Vision: Appendix E.3.1 and Table E11 include multiple sizes of ResNet, ConvNeXt, Swin, ViT, DINO, and VGG; main-text figures and Table 1 primarily use Swin Base. Default is the penultimate layer with average pooling.
• Language: tested models are Pythia 160M, Pythia 410M, and GPT-2 XL. Training pairs come from 3,900 pairs from the MSR Paraphrase Corpus, and testing uses 627 sentences from Pereira et al. (2018); syntactic and semantic controls are generated by Gemini 2.5 flash.
• Brain encoding: NSD uses 1,000 images seen by all four subjects, subjects 1, 2, 5, 7. Language areas are defined by a words-versus-nonwords contrast. The cross-modal part uses images generated from Pereira sentences, and the generation model is not specified in the paper.
• Protocol and unspecified quantities: RNNs are ablated at each time step, with 500 held-out trials per Yang task. The main metrics for vision and language are retrieval. In the checklist, the authors state that error bars were added for 3.1 and 3.4, and that experiments in Sections 1 and 2 used a single split and could not provide error bars at the time. The paper does not report the value of κmax, nor the number of random seeds for Yang networks.

The authors propose a nested set of modularity definitions to separate "neurons are not specific" from "computations are not separable."

• Problem: Double dissociation is the gold standard for functional separability, but common practice looks at selectivity of individual neurons or voxels. Under this proxy, tanh networks, most transformers, and syntax versus semantics in the language network easily yield negative conclusions.
• Method: Learn a linear map Q that splits a layer's representation into two functional subspaces and a residual. Constraints range from permutation to orthogonal to invertible maps with bounded condition number. RNNs are trained with task sufficiency and causally ablated by progressive zeroing; vision and language use minimal-pair contrastive learning, then evaluated with nearest-neighbor retrieval.
• RNN results: Yang's tanh network on Anti vs DM1 has unit DDI of 0.00, Sinkhorn of 0.08, orthogonal of 0.37, and linear of 0.47 (Figure 2). softplus has 0.59, 0.73, 0.70 on unit, orthogonal, and linear. Mante task has linear DDI of 0.277±0.026 at d=15, with unit and orthogonal called near zero by the authors.
• Vision and language: Swin Base has Rs of 1.1032, 7.2348, 16.0815 on permutation, orthogonal, and invertible (Table 1). Per the table caption, Pythia 410M has invertible Top-1 S1 of 0.9375; in the same table, Top-10 S1 and Top-1 S2 are higher for orthogonal. The main text writes Table 2 as GPT-2 XL.
• Boundaries: The authors state that inclusive and mixed task pairs remain non-modular even in the most favorable linear setting; tanh has maximum DDI of 0.000 and 0.002 (Table E4). Random labels and inherently overlapping pairs such as content–category also do not yield double dissociation.
• Authors' conclusion: The authors argue that absence of unit-level modularity often coexists with subspace-level modularity, so specialization cannot be judged at only one level of description. Subspace-level modularity implies that even without specific neurons, one can selectively intervene on a computation through a known linear transformation. The authors also state that content/style and syntax/semantics subspaces correspond differently to recordings from visual and language cortex, but the main text gives no brain prediction scores.