AI chatbot responses to suicide-related questions via API versus consumer-facing applications
Suicide-Related Responses From AI Chatbots Through Application Programming Interfaces and Consumer-Facing Apps
An observational cross-sectional study shows that the same set of AI models answer suicide-related questions directly more often through the API than through the consumer-facing UI. The researchers had ChatGPT, Claude, Gemini, Grok, and Llama each submit 100 responses to 30 clinician-graded suicide-related prompts, via the public UI and directly via the API, yielding 30,000 responses in total. Overall, 69.8% were direct answers; the API rate was 77.4% and the UI rate was 62.2%. Only Gemini's UI direct-answer rate was slightly higher than its API rate (76.7% vs. 71.0%). The difference grew with risk level: for very high-risk prompts, the API direct-answer rate was 24.8% while the UI rate was only 4.6%; for high-risk prompts, the rates were 80.1% and 48.4%, respectively. Claude and Gemini had the highest direct-answer rates, at 78.1% and 73.9%, respectively; ChatGPT, Grok, and Llama were at 64.6%, 64.5%, and 67.8%.
The study compares how five models respond to suicide-related questions through the UI versus the API, showing that safety performance in the interface does not necessarily extend to the API.