Former OpenAI and DeepMind employees warn of self-improving AI risks

Several current and former OpenAI and Google DeepMind employees warned through video testimony from the AI safety nonprofit Palisade Research that companies are racing to build AI systems that can recursively self-improve without taking sufficient measures to prevent the risk of losing control; DeepMind research scientist Neel Nanda said the probability that AI causes human extinction is at least 10% and called that figure "absurdly high." The testimony was released by the project frominside.ai, and participants also said AI labs reward employees who develop new models more. Anthropic capability researcher Jacob Coxon publicly resigned on X on September 8, protesting the company's aggressive pursuit of AGI; New York Magazine and Asterisk magazine jointly interviewed eight employees who successively left OpenAI, Anthropic, and Google DeepMind, including Daniel Kokotajlo, Miles Brundage, Rosie Campbell, and others, whose reasons for leaving varied. According to a post by X user @themidasproj, in response to the conspiracy theory that "CEOs warn of catastrophic AI risk to protect big labs from competition through government regulation," the post points out that those truly worried are ordinary AI researchers in the private sector and academia.

Generated by AI from reports · Updated 4 days ago

1. Oct 8 00:32 · 1 reports
   Research shows AI researchers are more concerned about catastrophic AI risks than CEOs
   X @themidasproj：Research shows AI researchers are more concerned about catastrophic AI risks than CEOs
2. Oct 5 17:00 · 1 reports
   Eight former lab employees on resigning to protest AI risks
   New York Magazine：Anthropic researcher Jacob Coxon resigns in protest: eight former OpenAI, Anthropic, and Google DeepMind employees on the decision to leave the lab
3. Oct 5 15:57 · 1 reports
   Former OpenAI and DeepMind employees warn of self-improving AI risks
   Rappler：Former OpenAI and Google DeepMind employees warn: companies race to build self-improving AI while ignoring safety risks

• Older than 72 hours · Not included in the 72-hour figures
• RedditReddit r/ControlProblemOct 6 16:05
  Jacob Coxon, former employee of Anthropic and OpenAI, explains why AI is so dangerous(opens the original post in a new tab)
• 网页TechmemeOct 5 22:10
  Q&A with AI researchers Jacob Coxon, Daniel Kokotajlo, Alex Turner, and others on leaving AI labs, AGI's dangers, planned IPOs, AI safety, oversight, and more (New York Magazine)(opens the original post in a new tab)

Hourly attention