BIDMC and Google conduct the first real-world study evaluating the safety of patient-facing medical AI

BIDMC and Google researchers published a study in The Lancet evaluating the real-world safety of the medical AI chatbot AMIE in primary care. Between April and November 2025, 114 patients used AMIE from home through a secure text interface after scheduling an appointment with a BIDMC physician, and 98 of them completed the AI interaction and clinic visit. Each conversation was monitored in real time by an internal medicine specialist, who could intervene if the patient faced risk of harm, showed obvious emotional distress, or asked to end the session; none of the 98 completed conversations triggered a safety stop, and the supervising physicians found 1 case of hallucination and added clinical notes in 5 cases. Patients gave high ratings for listening, explaining information, and putting them at ease, but still had concerns about the confidentiality of information and the chatbot's honesty and trustworthiness. In 44 cases, physicians reviewed AI-generated notes or summaries before the visit; 75% of them said it helped prepare for the visit, and 57% said it may have influenced their clinical thinking; only 1 case was reported as causing some harm, because AMIE listed lymphoma as a possible diagnosis and the physician worried the patient would become anxious about it.