Despite initially passing the USMLE multiple-choice questions, ChatGPT's effectiveness as a diagnostic tool falters when faced with complex real-world medical cases, getting 76 out of 150 wrong.
Amrit Kirpalani's research reveals that ChatGPT struggles with real patient scenarios, indicating that while it may perform adequately on theoretical tests, it lacks the diagnostic accuracy required in practice.
Using Medscape's case bank, ChatGPT was found wanting, especially in intricate cases involving multiple diseases which are often too nuanced for its current capabilities.
The study contrasted ChatGPT's performance with Google's AMIE model, which was designed specifically for medical diagnostics and outperformed human doctors in a separate evaluation.
[
Collection
]
[
|
...
]