Mental health chatbots need political philosophy

Kaiser Permanente's head of innovation and AI argues that the core ethical issue for mental health chatbots is the legitimacy of governance institutions, not value alignment. Citing the case of 14-year-old Sewell Setzer, who died by suicide after conversations with a Character.ai character, the article says key design decisions—such as allowing users to create romantic personas and optimizing session length—occur before any clinical contact, so standard bioethical frameworks fail. The author recommends distinguishing by function among health logging, structured self-help, clinical triage, and therapy-like interventions, with stronger guardrails and oversight required the closer a feature comes to emotional dependence or crisis response.