Skip to content
Redwood Research· Caleb Biddulph·Indexed by this site · Originally publishedSelectedAttention48

Redwood Research proposes an untrusted advice protocol: 16-character advice can recover about 67% of the capability gap

Untrusted advice for AI control: Short, strong advice significantly uplifts weak LLMs

AI summary

Redwood Research proposes an untrusted advice protocol, where a trusted execution model T performs all actions, and an untrusted model U can only send short advice through a restricted channel, thereby compressing U's influence into a monitorable narrow channel. On SWE-bench Verified, with U using Claude Sonnet 4.6 and T using Gemini 3.1 Flash Lite and gpt-oss-120b, with a limit of 16 characters per advice, T can recover about 67% of the capability gap between T and U, averaging about 320 characters per task. The authors re-measure information content using LLM surprisal, which can reduce bit usage by about 2.5 to 30 times compared to a naive upper bound of 8 bits per character; in BashArena, they also try having U select from a fixed advice menu, reducing per-step influence to 5 to 6 bits.

Why it was selected

Redwood Research proposes letting an untrusted model advise a trusted model only through an extremely narrow information channel, recovering about 67% of the capability gap on SWE-bench with 16 characters, offering a quantifiable approach to AI control.

Read originalblog.redwoodresearch.org