FAR.AI releases 2023 alignment research progress: from KataGo adversarial attacks to robustness scaling laws

FAR.AI announced its alignment research progress since its founding over a year ago, with its research divided into three categories: robustness science, value alignment, and model evaluation. The team found catastrophic failure modes in superhuman Go AIs such as KataGo and AlphaGo, is using mechanistic interpretability methods to analyze their adversarial attack vulnerabilities, and is exploring scaling laws for language model robustness. The organization believes that existing alignment methods such as RLHF cannot provide provable safety guarantees, and its goal is to incubate early-stage safety research agendas.