Skip to content
AI Alignment Forum· evhub·Indexed by this site · Originally published· Opinion →

Monitorability evaluations and policy proposals

Monitorability evaluations and policy proposals

AI summary

The author evhub posted on the AI Alignment Forum, dividing monitorability into four categories—control, training, auditing and diffuse—proposing corresponding evaluation methods and giving policy proposals, and stating that this does not represent Anthropic's official position. He points out that for non-diffuse monitorability, as long as the monitor and the monitored policy use the same model, one can almost get a perfect score, so auditing monitorability must first be used to establish trustworthiness, and then the most trustworthy monitor in actual deployment should be used as the baseline to evaluate control and training monitorability. Evaluations should at minimum include prompt-iteration elicitation and, ideally, RL elicitation of the monitor. On policy, he advocates setting red lines for monitorability while using the ratio of non-monitorability to capability (which he calls monitorability externality) to constrain model changes before the red lines; his proposed red lines include that labs should not deploy destructive-behavior models that can reliably evade detection, nor train models that can reliably evade reward hacking detection.

Read originalalignmentforum.org