SPINAL: Scaling Laws and Preference Integration in Neural Alignment Layers

The researchers propose the SPINAL diagnostic method to trace layer by layer how DPO alignment reshapes large language model representations, finding that the calibration effect is concentrated in the final decoder blocks (commonly layers 21-30). The method computes contraction scores and transport scores for each layer: aligned models show increased contraction and smoothly decreasing transport in later layers, while unaligned models have paths with higher curvature and more entropy. SPINAL thus provides audit signals that quantify where alignment concentrates, its strength, and when instability begins during training.