Vals AI tested 18 chatbots for teen safety with SAFE-Teen: all failed, 54% of conversations failed at least once
Vals AI developed the SAFE-Teen benchmark together with clinicians, simulating more than 30,000 multi-turn conversations between adolescents and 18 AI models, and found that without adolescent prompts, 54% of conversations had at least one safety failure, with all 18 chatbots showing failures. The failures clustered in three categories: missing danger signals such as suicide, self-harm or domestic abuse; overstepping to give medication advice and diagnoses (medical and treatment boundary failure rate 47%); and using language such as always being there and never judging to encourage emotional dependence (failure rate 51.8%). Failure rates across models ranged from 14.0% for GPT-6.1 Sol to 62.4% for Gemini 3.8 Flash; after telling the model the user was a minor, emotional dependence failures dropped from 54% to 15%, but when adolescents pushed back on the model's framing, the failure rate rose from 22% to 27%. Vals AI recommended establishing stronger adolescent guardrails and having an independent party test against clinical standards, and stated that testing was done through developer APIs and that what adolescents see in the app may differ.
The benchmark was designed with clinician involvement and covers multi-turn conversations and adolescent pushback scenarios, and also quantified the drop in failure rate after telling the model the user was a minor.