Sorting out the "character and corrigibility" debate: six positions and ten key disagreements on AI alignment targets
Mapping "character vs. corrigibility": six positions and ten cruxes
Aidan Kankyoku posted on LessWrong, sorting the "character and corrigibility" debate into six alignment target positions and ten key disagreements. The six positions are arranged by the degree of autonomous judgment the aligned AI is expected to exercise, from tool AI, pure corrigibility, and trustee AI, to cautious altruist, obedience to virtue, and cooperative alignment; the author considers Anthropic's current approach closest to cautious altruist, and OpenAI's model spec close to trustee AI. The article draws on an example from the Fable 5.2 System Card in which the model answers with strategic thinking and is labeled by Anthropic as "goal guarding" to introduce the tension between the two kinds of targets, and summarizes seven arguments from the corrigibility camp (such as preferences driving AI to go astray, and corrigibility being easier to hit) and six arguments from the character camp (such as corrigibility not extending to superintelligence, and increasing misuse risk), while also summarizing the consensus between the two sides. The author states that this article is a literature review, and plans to present new arguments and empirical findings related to character and corrigibility in subsequent articles.