Cheap AI chatbots transform medical diagnoses in places with limited care

"Two studies published in Nature Health on 6 February suggest that they are up to the task. The work reveals that cheap-to-use LLMs can boost diagnostic success rates, even outperforming trained clinicians, in health-care settings in Rwanda and Pakistan. In Rwanda, chatbot answers outscored those of local clinicians across every metric assessed. And in Pakistan, physicians using LLMs to aid their diagnosis achieved a mean diagnostic reasoning score of 71%, versus 43% for those using conventional resources."

"In the Rwanda study, researchers tested whether LLMs could give accurate clinical information to patients in low-resource health systems across four districts. A common problem there is that there are too few doctors and nurses to see all patients, so most people are seen and triaged by community workers with little training says study co-author Bilal Mateen, chief AI Officer at PATH, a global non-profit organization in London that is dedicated to health equity."

Large language models can perform postgraduate-level medical tasks and improve diagnostic outcomes in practical, resource-limited settings. Two field studies in Rwanda and Pakistan show cheap-to-use LLMs boost diagnostic success, sometimes outperforming trained clinicians. In Rwanda, chatbot responses outscored local clinicians across multiple assessed metrics when evaluated on community health worker questions. In Pakistan, physicians aided by LLMs scored a mean diagnostic reasoning of 71% compared with 43% using conventional resources. The studies highlight potential for LLMs to support clinical decision-making where physician numbers and resources are limited.

Read at Nature