Former OpenAI and Google DeepMind employees warn: companies race to build self-improving AI while ignoring safety risks
AI researchers warn companies rushing self-improving systems despite safety risks
AI summary
Multiple current and former employees of OpenAI and Google DeepMind, through video testimony via the AI safety nonprofit Palisade Research, warned that companies are racing to build recursively self-improving AI systems without taking sufficient measures to prevent the risk of losing control. DeepMind research scientist Neel Nanda said the probability that AI causes human extinction is at least 10%, and called this figure "absurdly high." The testimony was released by the project frominside.ai, and participants also said AI labs reward employees who develop new models more than those who call for caution.