Prospective clinical feasibility study of the conversational diagnostic AI AMIE in an outpatient primary care clinic

The research team reports a prospective single-arm feasibility study of the large language model-based conversational diagnostic AI AMIE, in which the system conducted history-taking for urgent care patients at a leading academic medical center and presented potential diagnoses for patients and physicians to discuss. 100 adult patients completed text conversations with AMIE up to 5 days before their visit, and human safety supervisors monitored all interactions in real time, with no intervention required to terminate according to prespecified criteria. Patient satisfaction was high, and attitudes toward AI improved after the interaction (p < 0.001); primary care physicians found AMIE's output useful and helpful for visit preparation. According to medical record review 8 weeks after the visit, AMIE's differential diagnosis included the final diagnosis in 90% of cases, with a top-three accuracy of 75%; blinded evaluation showed that the overall quality of its differential diagnosis and management plans was comparable to that of primary care physicians (DDx p = 0.6, Mx appropriateness and safety p = 0.1 and 1.0), but physicians were better on plan practicality and cost-effectiveness (p = 0.003 and 0.004).