Judiciary sounds alarm for AI governance of government

In the case American Council of Learned Societies v. National Endowment for the Humanities, a U.S. federal court found that DOGE's use of ChatGPT to screen and terminate more than 1,400 NEH grants was unconstitutional. The facts showed that two DOGE employees entered descriptions of hundreds of grants into ChatGPT and asked whether they were related to DEI. ChatGPT flagged projects based on race or gender mentioned in the descriptions, and DOGE adopted these classifications wholesale as the basis for termination. The court held that terminating grants because of their association with supposedly disfavored viewpoints violates the First Amendment, and that canceling grants because they involve protected categories such as race or gender violates equal protection, noting that these classifications constitute the government's own conduct in a constitutional sense. The author argues that large language models are merely tools, and agencies should not outsource decision-making to models but should instead pair them with employee training and reviewable oversight processes.