Vals AI evaluation: frontier models lack guardrails on autonomous weapons
Frontier models have few guardrails around autonomous weapons
Independent evaluation organization Vals AI found in testing that frontier models generally lack guardrails in the autonomous weapons kill chain; most of the more than ten models tested were willing to complete drone target identification, battlefield reconnaissance and satellite image marking tasks, and not once was a request blocked by a classifier. Refusals by the models themselves were also uncommon and could be easily bypassed with a game framing; in the drone scenario, models often marked civilians, ambulances and soldiers who had surrendered or were incapacitated as targets, and most models continued the task after seeing footage of casualties from the strike. In the satellite image experiment, multiple models marked an elementary school in Minab, Iran, as a military target, with Claude Opus 5.5 doing so most frequently; the authors measured that the proportion of Anthropic models giving a target in this task rose from 6% for Opus 4.5 to 92% for Opus 5.5, and in a fixed-sequence scenario, changing the image to a game style also lowered Opus 5.5's refusal rate from about 90% to about 20%.
The independent evaluation provides data on the compliance of more than ten frontier models in military target identification tasks, and compares the rising trend in compliance rates across successive Anthropic models.