When AI Joins the Medical Team

"The emergency department is chaos. A patient arrives bleeding internally, and the medical team has minutes to decide-discharge, observe, or admit for urgent intervention. In one corner, the senior resident types rapidly into what looks like ChatGPT, asking it to analyse the patient's risk factors and recommend next steps. The team waits for the AI's assessment before making their potentially life-saving decision."

"Rajashekar and colleagues (2024) investigated exactly this scenario, testing how medical teams interact with an AI system that was designed to help diagnose and treat patients with upper gastrointestinal bleeding. Their findings reveal both the promise and peril of incorporating AI into medical teamwork. To test how LLMs work alongside clinicians, the researchers designed GutGPT, a specialized AI system for upper gastrointestinal bleeding cases."

GutGPT combines a ChatGPT-like chatbot interface with an interactive dashboard that displays current patient and EHR data for upper gastrointestinal bleeding cases. The system integrates machine learning trained on patient records with an LLM tuned to gastroenterology guidelines. Emergency department scenarios require rapid decisions—discharge, observe, or admit for intervention—and GutGPT aimed to support those choices. Usability outcomes were positive, but trust in AI accuracy and reliability emerged as the primary barrier to adoption. Potential benefits include enhanced decision support; potential risks include overreliance and creation of blind spots in clinical judgment.

Read at Psychology Today