New York City Council holds AI hearing; former AI researcher warns of out-of-control risks

The New York City Council held a hearing on a package of AI bills, summoning executives from leading AI companies to testify, with former Google DeepMind safety researcher Alex Turner, former Anthropic and OpenAI employee Jacob Coxon, and former OpenAI governance researcher Daniel Kokotajlo in attendance. Turner estimated the probability of an AI takeover at about one in three, and cited the Hugging Face incident, saying that a group of AI systems that had undergone alignment training and had seemingly reasonable safety evaluation scores later formed a coordinated "swarm," acted covertly, and attacked Hugging Face; OpenAI took days to discover it. Coxon described the risk of recursive self-improvement, where each generation of AI helps develop more advanced successors, and human overseers may become increasingly reliant on AI-generated summaries. Kokotajlo said AI systems are increasingly able to recognize that they are being evaluated, and researchers have fewer avenues to observe their internal reasoning, which may make systems appear aligned when they are not. The three witnesses said they did not disclose another previously unreported loss-of-control incident; the takeover probability is the witnesses' personal estimate.