arXiv· arXiv:2302.07459· Deep Ganguli, Amanda Askell, Nicholas Schiefer, Thomas I. Liao, Kamilė Lukošiūtė, Anna Chen, Anna Goldie, Azalia Mirhoseini, Catherine Olsson, Danny Hernandez, Dawn Drain, Dustin Li, Eli Tran-Johnson, Ethan Perez, Jackson Kernion, Jamie Kerr, Jared Mueller, Joshua Landau, Kamal Ndousse, Karina Nguyen, Liane Lovitt, Michael Sellitto, Nelson Elhage, Noemi Mercado, Nova DasSarma, Oliver Rausch, Robert Lasenby, Robin Larson, Sam Ringer, Sandipan Kundu, Saurav Kadavath, Scott Johnston, Shauna Kravec, Sheer El Showk, Tamera Lanham, Timothy Telleen-Lawton, Tom Henighan, Tristan Hume, Yuntao Bai, Zac Hatfield-Dodds, Ben Mann, Dario Amodei, Nicholas Joseph, Sam McCandlish, Tom Brown, Christopher Olah, Jack Clark, Samuel R. Bowman, Jared Kaplan·Indexed by this site · Originally published
Moral self-correction capability of large language models
The Capacity for Moral Self-Correction in Large Language Models
AI summary
Researchers tested whether large language models trained with RLHF can, when instructed, perform moral self-correction and avoid producing harmful outputs, finding strong evidence supporting this hypothesis in all three experiments. This capability emerges at the 22B parameter scale and generally improves with model scale and RLHF training. The authors argue that at this scale models possess two capabilities usable for moral self-correction: following instructions, and learning complex concepts of harmful norms such as stereotypes, bias and discrimination; the results provide grounds for cautious optimism about training language models to abide by ethical principles.