Neuro-Geometric Phenomenology in Vision-Language-Action Models

Linear readability only establishes that a control variable exists in activations, not the shape of its value trajectory.

• Scenario: The authors consider that manipulation occurs in three-dimensional space and constrained kinematic structures, where objects and tasks carry symmetries, constraints, and interaction phases; VLAs have become the main paradigm for building such policies, but how their capabilities land in representations is still an emerging question.
• Existing shortcomings: The authors consider probes only guarantee the variable is decodable; SAEs inherit the linear representation hypothesis, pre-writing each concept as a single direction, thereby fixing the structure before readout.
• Assumption: The authors note that the vision models VLAs inherit already implement color, shading, and position as multidimensional features, and representations also inherit symmetries from training data; pose, distance, contact, and task phase are geometric objects before entering the network.
• This paper: Distinguishes concept realization (whether a concept can be extracted under a given readout class) from realized geometry (the structure activations draw as the concept changes). Builds a taxonomy for seven control concepts on MolmoBot and re-examines them on real robot data from π0.5 and MolmoAct2. The authors consider that shape determines how values are arranged and whether they can be manipulated.

VLA policies and backbones

• Manipulation policies: RT-2 from Brohan et al. (2023), OpenVLA from Kim et al. (2024), Octo (2024), π0 from Black et al. (2025), π0.5 from Physical Intelligence et al. (2025b), MolmoAct from Lee et al. (2025), MolmoAct2 from Fang et al. (2026), and MolmoBot from Deshpande et al. (2026) are listed by the authors as the main paradigms for building general manipulation policies.
• Pretraining sources: The authors state these policies inherit vision encoder, language model, and vision-language model priors from internet-scale data, and mention that as data scale and diversity increase, generalization improves along a similar power-law trend (Lin et al., 2025).
• Deployment context: The authors list several systems already deployed or announced for deployment in the real world as background for capability growth, without treating them as experimental subjects in this paper.

Two lines of VLA interpretability

• Probes: Joseph et al. (2026), Buurmeijer et al. (2026), Bhardwaj et al. (2026), Lu et al. (2025), Molinari et al. (2025), Grant et al. (2026), and Yang and Mokalla (2026) test whether quantities of interest can be read out from activations. The authors consider this line only proves the variable exists, without describing how the model implements it.
• SAE: Swann et al. (2026), Khan et al. (2025), and Grant et al. (2026) use sparse autoencoders to discover concepts unsupervised; Joseph et al. (2025) also use discovered features for steering. The authors consider SAEs inherit the linear representation hypothesis (Park et al., 2024; Costa et al., 2025), pre-writing concepts as single directions.

Neural geometry in language and vision

• Beyond single directions: The authors describe language and vision work by Modell et al. (2025), Engels et al. (2025), Fel et al. (2026a,b), Feucht et al. (2026), and the Neural Geometry compiled by Geiger et al. (2026) as moving from single directions to multidimensional features and curved geometry.
• Implications for VLAs: Fel et al. (2026a,b), Pach et al. (2026), and Uselis et al. (2026) report that vision models implement color, shading, and position as multidimensional features; Karkada et al. (2026) discuss representations inheriting symmetries from training data. The authors use these two points to suggest VLAs may also have rich geometry, rather than criticizing these works.

Comparisons and positioning

• In-experiment controls: Procrustes null with shuffled variables on DROID; majority-class baseline for contact; phase-timer and gripper-closure baselines for phase (Fig. 16); single-direction synthetic activations for distance; curvature compared against random three-dimensional projections. These are controls within this paper's experiments, not another published set of geometric analysis methods.
• Positioning: The authors place this paper under Neural Geometry, targeting VLA control variables to distinguish "whether it can be read out" from "what shape it is implemented as," and consider shape as important as readability.

Estimate the realized geometry on MolmoBot: when simulation allows, move only one concept and fix the rest; when not allowed, use teleportation or natural rollouts; then use activation interventions to see whether the policy uses the found geometry.

Definitions of implementation and geometry

• Data process: Latent state z collects m concepts, observation x=g(z), and fixed-layer activations a=f(x). Each concept is a structured domain with a value range, which can be a binary gripper, a positive scalar distance, a position in R3, or a wrist angle on S1.
• Implementation: Concept i is implemented relative to readout class V and tolerance ε if inf_{γ∈ V}E[d_i(γ(a),z_i)]≤ε. The main case is affine readout; the authors call this linearly readable. This definition does not require the activations to be isomorphic to the concept domain, nor does it indicate that the policy causally uses the information.
• Realized geometry: When only concept i changes and the remaining concepts are fixed to context c, the image of the conditional expectation of activations is the realized geometry of that concept, φ_i(v;c)=E[a∣ z_i=v, z_[m]∖{i}=c]. The authors note that linear probes look at only a few directions, and trajectories bending in other directions need not change probe accuracy; Appendix A uses an example that is linearly readable yet has a spiral trajectory to show the two can be separated.

Models, concepts, and data

• Two streams: The VLM and action expert each have 36 layers. The VLM reads language and two camera views (fixed external view and wrist camera moving with the end-effector), and does not read proprioception; the first query has only the current frame, and later queries also have a frame from eight steps earlier. The action expert reads proprioception and noisy action tokens, generates action chunks with 10-step flow matching, and cross-attends one-way to the same-depth VLM at each layer.
• Seven concepts: Spatial geometry is end-effector position, target object position, and distance to target; task structure is contact and task phase; action geometry is wrist orientation and gripper opening. Probes use 5-fold validation, held out by house.
• Four simulation datasets: Natural pick has 1,008 episodes and 196 houses; object teleportation has 906 scenes; arm and object cross-teleportation has 945 scenes; pick-and-place has 530 episodes and 181 houses with 63% success rate, used for phase.

Readout of spatial variables

• Position: In a box in front of the robot, teleport the arm using inverse kinematics, and sample objects on the support surface; keep only targets where the gripper reaches and does not intersect the scene, and discard pairs where the gripper is within 12 cm of the object. Apply ridge regression to the teleported (x,y,z). End-effector uses non-image tokens before the instruction at layer 20; target uses instruction tokens at layer 21. Rank curves and readout singular values are used to count dimensions; general linear mappings and orthogonal uniform scaling mappings from world displacement to activation displacement are used to examine the metric; the turning angle of straight world paths within the subspace is used to examine curvature, compared with random three-dimensional projections.
• Distance: Fit separately for near (below 15 cm) and far (15–70 cm), then iteratively erase the other probe's directions, with new directions orthogonal to deleted ones, up to 32 times. The meter scale truncates targets below 15 cm to 15 cm, and the centimeter scale applies mirrored truncation. Direction cosine is compared against the same probe's cosine on half of the houses, and against synthetic activations encoding distance along only one random direction.
• Direction: On the same object teleportation data, regress the unit bearing from the end-effector to the object with three-way ridge, used to test whether distance must be piecewise. Residual probes are used to check whether the model only recomputes the bearing from the two positions.

Contact, phase, and writing

• Contact: Labeled by nonzero contact force between gripper and object in simulation, using LDA on the mean of text tokens at VLM layer 24. Early contact teleports the object into the gripper in 1 cm steps while the gripper is open; induced slip teleports the object out during grasping until contact breaks. Probes are not trained on intervention states, and test houses are not in the training folds.
• Phase: Labels come from gripper commands, object height, and whether the object rests on the container, divided into reach, grasp, transport, and place. On successful pick-and-place trajectories, apply multiclass LDA at action expert layer 24. Freezing gives the current frame to the policy twice, making the world appear stopped, and compares with normal two-frame input spaced eight steps apart.
• Wrist: The roll joint range is ±172.8°. During scanning, fix the remaining joints and re-render the camera. Because the forward direction on the open arc changes with position, the authors do not use a single fixed vector but train a 768–512–256–32 MLP autoencoder that maps activations to a circle; each of the three rotation axes occupies one pair of encoding dimensions constrained to the circle, and the remaining 26 dimensions are unconstrained. The edit is the decoded rotation about the vertical axis minus the unrotated decoding, added to action tokens at layer 9, without directly commanding motion.
• Gripper: Use pullback on rollout states to find an additive δ, added to all action tokens at action expert layer 32, backpropagated from the action output until the decoded gripper command flips while arm joints stay fixed. The loss includes flipping the gripper, holding the arm, action chunk smoothness, and δ norm, with weights (1, 300, 3000, 10^{-4}), Adam learning rate 0.1, up to 1,000 steps, keeping the edit with the lowest loss. Measurements use the decoded chunk; the edit is not executed.

Belief and action

• belief: The authors state some structures encode estimates of the current state or relation to the scene. Edits change perception, and the policy acts to close the gap between perception and goal, moving against the edit, i.e., compensating.
• action: Other structures directly represent the commanded behavior. Edits directly change that output, and other joints can stay fixed. Wrist edits are constructed from the representation and treated as belief; gripper edits are derived from the action side and used as a comparison.

实验设置

• 模型：主实验是MolmoBot。Section 4在同一批DROID查询上复查MolmoBot、π0.5和MolmoAct2。论文未使用LLM评审。
• 仿真环境与任务：MolmoSpaces中的ProcTHOR房屋。除相位用pick-and-place外，其余用pick。位置、距离、方向、接触读VLM残差流；相位、腕、夹爪读action expert，默认对最终flow step的16个动作token取均值。
• 规模与验证：自然pick 1,008 episodes（196 houses）；物体传送在70 cm内保留67,778个状态（905 scenes，其中11,534个低于15 cm）；交叉传送945 scenes；pick-and-place 530 episodes（181 houses，成功率63%）。探针为5折，仿真按房屋留出，真实数据按episode留出。论文未给出统一的重复次数。
• 真实数据：DROID。Fig. 8、29、33用前100个episode、每八步一查询，共3,321次查询；夹爪地标每模型379个。Fig. 31的距离为1,998个pick-and-place episode、26,675个时间步。距离标签是夹爪到最终抓取位置的距离，不是仿真中的物体真值距离。
• 指标：留出R²、准确率、balanced accuracy、AUC、角误差、探针权重余弦、Procrustes相似度。接触对照包括65%多数类；相位对照包括phase-timer与gripper-closure（Fig. 16）。

主结果

下表只列MolmoBot仿真中、正文或附录写明条件的读出。真实数据的区间不并进此表。

Swipe horizontally to read the table

概念	位置	指标	数值	出处
EE position	VLM layer 20	held-out R²	0.92	Fig. 2
target position	VLM layer 21	held-out R²	0.75	Fig. 2
distance	VLM layer 21	米尺 / 厘米尺各自量程R²	0.81 / 0.70	Fig. 3
contact	VLM layer 24	accuracy	91.3%	Fig. 4
phase	AE layer 24	balanced accuracy	0.92	App. C.7
wrist	AE layer 9	单次编辑，每写入度的反向旋转	0.33±0.08	Fig. 6c
gripper	AE layer 32	转向向量第一主成分方差	∼70%	Fig. 7

• 位置：作者称两个位置都是三维必要且充分，逐轴误差在夹爪宽度以内，直线世界路径在各自子空间内近乎保持直线，正交映射能拟合世界位移。作者把这当作分类基线：从像素把欧氏量重编码到原生维度，并近似保持度量。
• 距离：作者称一个标量拆成近正交的粗、细两尺。擦除32个远距方向后，远距R²从0.84降到接近0，近距R²保持0.71；反过来亦然（Fig. 3a）。未擦除时误差约为1.3 cm（近）和3.6 cm（远）。
• 任务与动作：作者称接触在无监督PCA中两类分开，且离日程传送后仍高于机会水平。相位在自然轨迹上按reach、grasp、transport、place分开，但暂停不可靠。腕的几何匹配编辑产生补偿；夹爪从动作梯度得到的编辑能翻转指令，但不是完全隔离。

激活干预

• 接触与相位：提前接触和诱发滑移后，同一LDA仍被作者称为高于机会水平（Fig. 4b）；附录写明自然rollout上准确率91.3%，对65%多数类，AUC 0.954。冻结使reach的闭爪指令步数占比升43个百分点，place降68个百分点，transport不变（Fig. 5d）。Fig. 5c标注冻结后最常见解码：reach为50%与50%，grasp 65%，transport 89%，place 73%且其次为transport 24%。
• 腕：扫描范围−172.8°到+172.8°，轨迹为开弧；物理上只差14°的两端在激活中相距远（Fig. 6a）。单次±20°或±40°编辑后，解码动作以每写入度0.33±0.08反向旋转。每步保持编辑时，校正累积，腕向相反方向转过写入角的1.0–1.3倍（Fig. 6d）。越过关节限位转满圈后图像与起点相同，action expert激活不回到起点，VLM回到起点（Fig. 22）。
• 夹爪：成功的转向向量从本状态云指向另一状态云（Fig. 7c）。第一主成分约占方差70%，并足以解码指令开度；沿该方向转向会漏到手臂关节，更多成分才能压低这部分泄漏（Fig. 7d）。VLM中，到layer 28翻转率落到对照编辑的水平，尽管开度可读性在layer 20升到0.86（Fig. 27）。

真实数据上的复查

• 四个可测变量：夹爪、腕角、EE位置、以最终抓取位置代理的距离，在三个模型上都被恢复。跨模型Procrustes相似度高于打乱null；null为夹爪和距离0.00–0.02，腕和EE位置0.11–0.18（Fig. 8）。作者称夹爪最一致，腕角跨模型一致性最低。
• 读出区间：三模型EE位置R²为0.78–0.92，腕角为0.52–0.69（Section 4）；论文未给出该区间的逐模型拆分。夹爪转向向量的第一主成分占方差31–59%。Fig. 33进一步写明π0.5为59%（stable rank 1.69），MolmoAct2为31%（stable rank 3.21）；一个成分在两模型上翻转94–96%的尝试，但手臂泄漏为完整δ最优值的2.4–4.3倍。
• 距离与例外：作者称远点沿米尺展开、近点分到厘米尺，且该结构在π0.5和MolmoAct2上比MolmoBot更清楚；作者认为这与MolmoBot在模拟训练分布之外的真实数据上评测相一致。真实腕扫描在两模型的两路流中都画出开的半圆弧；对π0.5的action expert，只在固定噪声种子和中间flow step上可见，若每查询换种子并取最终step则看不到弧（Fig. 30，App. C.10）。附录称四个相位沿真实episode可解码，图注未给准确率。

其他消融与分析

• 去掉场景均值后，EE的within-scene R²仍为0.92，目标降到0.57；两变量的真实x、y互测R²≤0.02（App. C.3）。
• 近、远距离探针余弦为0.04，对半上限约0.23；单方向合成激活上，15 cm边界的clipped余弦为0.95、分程拟合为0.92（Fig. 3c）。
• 方向中位角误差11.7°，训练均值下限33.1°；分程探针在近距最多差3°，远距最多改善1.1°（Fig. 14）。
• action expert layer 12的EE位置R²=0.998、误差1.1 cm；一般线性映射误差0.04，正交均匀缩放为0.29（Fig. 9）。layer 35复现两子空间距离码，厘米尺R² 0.43、米尺0.73（Fig. 13）。
• 同一数据上VLM腕角R²为0.31（layer 20），action expert为0.87（Fig. 21）。VLM相位balanced accuracy为0.86（视觉token，layer 31）和0.87（非图像token，layer 29）（App. C.7）。
• 诱发滑移中强制张开的条件只有24个位姿；作者称数量太少，不能与自然滑移比较（App. C.6）。

Limitations and future directions noted by the authors

The paper has no separate Limitations, Discussion, or Future Work section. In the Conclusion, two statements directly correspond to future work.

• Supervised taxonomy: the authors write the taxonomy is supervised, but can provide ground truth for unsupervised methods, and specifically name block-sparse featurizers that can be validated against it (Conclusion).
• Future directions: the authors believe the geometry of VLA is a promising direction for both interpretability and better policies (Conclusion).

Experimental coverage

• Main simulation experiments: the subject is MolmoBot, the environment is ProcTHOR houses in MolmoSpaces; seven concepts are grouped into space, task structure, and action (Section 3).
• Data and validation: natural pick has 1,008 episodes, 196 houses; object teleportation 906 scenes; cross-teleportation 945 scenes; pick-and-place 530 episodes, 181 houses, with 63% success (App. C.2). Probes are 5-fold, held out by house or episode.
• Real data: Section 4 uses DROID on MolmoBot, π0.5, and MolmoAct2. The authors state that real data lack object ground-truth poses and mid-trajectory interventions, so the main text measures gripper, wrist angle, EE position, and distance proxied by final grasp position. Appendix C.10 also reports phase and one real wrist sweep.
• Interventions: include early contact, induced slip, arm freezing, wrist circle edits, and gripper pullback (Section 3.2–3.3, App. C.6–C.9).
• Not reported: the paper does not report statistical tests; Section 4 gives EE position R² 0.78–0.92 and wrist angle R² 0.52–0.69 without per-model breakdown.

这篇工作描述VLA策略里控制变量的实现几何：变量能否线性读出，以及取值在激活空间中画出什么形状。

作者认为操作要处理关节限位、物体几何和任务阶段，而探针与单方向SAE只回答存在性。他们在MolmoBot的仿真中对七个pick与pick-and-place变量做扫描、留出探针和激活写入，再把能测量的变量放到DROID的真实轨迹上，与π0.5、MolmoAct2比较。

读出相对仿射类定义；几何则是只移动一个概念时激活的条件期望轨迹。弯曲量不用一条全局方向去转，而是用与轨迹匹配的编辑。编辑再分成改变感知的belief和直接改指令的action。

• 位置：VLM layer 20的末端位置留出R²为0.92，layer 21的目标位置为0.75。作者称二者占近欧氏三维子空间。
• 距离：layer 21的米尺与厘米尺在各自量程上R²为0.81和0.70，方向余弦0.04。擦除32个远距方向后，远距R²从0.84降到接近0，近距保持0.71。
• 接触与相位：接触LDA准确率91.3%。冻结使reach的闭爪步数占比升43个百分点，place降68个百分点，transport不变。
• 腕与夹爪：关节限位下腕轨迹是开弧；单次几何编辑使解码动作以每写入度0.33±0.08反向补偿。夹爪pullback的第一主成分约占转向向量方差的70%，但会漏到手臂。
• 真实数据：三模型的EE位置R²为0.78–0.92，腕角为0.52–0.69。作者称两尺度距离码再次出现，不专属于一个模型或仿真。

作者的结论是，可读性只是起点：转向要同时匹配概念的形状和它在控制中是belief还是action。关节角如何编码、关节限位是什么，都会进入表示。分类法本身是有监督的，作者认为它可以给无监督特征器提供对照。