Study: NLA can identify activations but struggles to distinguish true from false descriptions

1. 01:05
   arXiv: Alignment, Deception & Oversight

   Monitorability of natural language autoencoders (NLAs): identification and discrimination

   The paper formalizes NLA training as a generative reference game and supervised deployment as a discriminative multimodal partially observable reference game, and infers that NLA explanations can only identify which activation they describe, not distinguish the true description of that activation from highly similar false descriptions. The authors use the NLA's own AR as a surrogate monitoring model and validate on a released checkpoint: the true description wins over descriptions of other activations in all 320 comparisons, but wins only 60.2% against a fluent adversary that changes a single statement (standard error 6.7%, chance level 50%, n=35). Meaning-preserving rewrites by an unrelated model (Qwen2.5-7B-Instruct) cause a loss equivalent to only 2.1% of the distance to another activation's description, indicating no private convention between AV and AR. The paper notes these adversaries involve only statements decidable at the conversation-record level, so the results cover only the easiest cases, and recommends that NLAs be evaluated with adversaries matched in specificity and that a discriminative term be added to the training objective.

Current attention 40·Peak in the comparable period 40 (Nov 6 03:00)·Change over the comparable past 24 hours 0%