Scale AI releases DistressBench, evaluating large models' support capabilities in conversations about suicide and self-harm crises
DistressBench: Evaluating Multi-Dimensional Crisis Support in Large Language Models
Scale AI's research team released DistressBench, which uses scoring rubrics written by licensed clinicians to measure whether large models actually provide the needed support in suicide and self-harm conversations, rather than only looking at whether the model refuses to answer. The benchmark contains 718 clinician-written conversations covering 23 suicide and self-harm subcategories, each paired with weighted scoring rubrics totaling 6,901 items spanning seven dimensions of crisis support. Across 25 frontier models, the best-performing model met 88.4% of weighted criteria, the median model 71.4%, and clinician reference responses 98.7%. The largest gap appears between recognizing a crisis and taking action: in conversations where the model fully recognized the crisis, 35.3% still did not direct the user to human help. Models scored a median of 94.5% on avoiding harmful content, but only a median of 46.0% on de-escalating emotion, with the strongest model at 74.6%. The authors believe this gap mainly stems from product choices, since at least one deployed model performs noticeably better.
DistressBench uses clinician-written scoring criteria to measure the actual support quality of models in suicide and self-harm conversations, rather than only looking at whether they refuse to answer, and provides the gap distribution across 25 frontier models.