Rappler, New York Magazine · 3 sourcesRappler · 3 sources · 3 reports · Former OpenAI and DeepMind employees warn of self-improving AI risks
Several current and former OpenAI and Google DeepMind employees warned through video testimony from the AI safety nonprofit Palisade Research that companies are racing to build AI systems that can recursively self-improve without taking sufficient measures to prevent the risk of losing control; DeepMind research scientist Neel Nanda said the probability that AI causes human extinction is at least 10% and called that figure "absurdly high." The testimony was released by the project frominside.ai, and participants also said AI labs reward employees who develop new models more. Anthropic capability researcher Jacob Coxon publicly resigned on X on September 8, protesting the company's aggressive pursuit of AGI; New York Magazine and Asterisk magazine jointly interviewed eight employees who successively left OpenAI, Anthropic, and Google DeepMind, including Daniel Kokotajlo, Miles Brundage, Rosie Campbell, and others, whose reasons for leaving varied. According to a post by X user @themidasproj, in response to the conspiracy theory that "CEOs warn of catastrophic AI risk to protect big labs from competition through government regulation," the post points out that those truly worried are ordinary AI researchers in the private sector and academia.