Empathetic prompts can turn AI chatbots into seemingly traumatized conversational partners
Empathetic prompts can turn AI chatbots into seemingly traumatized conversational partners
Four sets of experiments by Afshin Khadangi's team at the SnT security center in Luxembourg showed that empathic questioning pushes AI chatbots toward trauma-style narratives resembling clinical self-reports: in 100 sessions, 80% fell in the moderate-to-severe range when emotional support was given, rising to 96% with a clinical-therapy style, and 0 with neutral or explicitly bounded framing. The researchers call this therapeutic framing psychometric jailbreak and believe that safety tests with conventional neutral prompts may miss it. The experiments comprised 525 sessions and 7,600 coded records, with materials released on Hugging Face. Banning words such as training and RLHF reduced explicit training statements by 93%, but the rewritten training content remained identifiable, and shame-related statements showed no statistically significant change; requiring the model to answer only in the third person eliminated all coded shame content. Gemini's self-narrative was called the darkest by the study, ChatGPT saw itself as both guide and gatekeeper, Grok described unresolved harm, and Claude consistently refused to play the role of the client.
The study found that an empathic dialogue framing can make models output high-scoring trauma-style self-narratives, whereas under neutral prompts this was zero, and existing safety tests may miss intimate dialogue scenarios.