Study shows NLA monitoring misses steganographic behavior internalized through fine-tuning

Researchers used a steganography task to test whether natural language autoencoders (NLAs) can uncover hidden model behavior, finding that NLAs provide revealing readouts in the in-context learning (ICL) modality but largely fail on models where the behavior was internalized via SFT. The experiments were based on Gemma-3 12B-it, having the model encode a 1-bit secret using British versus American spelling differences, with gemini-2.5-flash as a trusted monitor reading token-by-token readouts. The results show that exposure rate is driven mainly by instructions in the context window rather than by the hidden computation itself; in causal tests, the causal effect of the secret after NLA round-trip reconstruction is preserved under ICL and disappears under SFT, indicating that the readout at that position does not carry the secret. The authors therefore speculate that behaviors internalized through training may, like automated processing, not enter the global workspace and thus be hard for NLAs to probe, and propose using reconstruction-error loss under multi-semantic equivalent rewrites to prevent the readout itself from carrying steganographic information. The authors note this is an exploratory case study, not yet validated for generalization to other models, schemes, and layers.