In Harvard study, AI offered more accurate diagnoses than emergency room doctors | TechCrunch

"At each diagnostic touchpoint, o1 either performed nominally better than or on par with the two attending physicians and 4o, especially pronounced at the first diagnostic touchpoint."

"With that information, the o1 model managed to offer 'the exact or very close diagnosis' in 67% of triage cases, compared to one physician who had the exact or close diagnosis 55% of the time."

"We tested the AI model against virtually every benchmark, and it eclipsed both prior models and our physician baselines."

A study evaluated the performance of OpenAI's language models in medical contexts, particularly in emergency room cases. Researchers compared diagnoses from two attending physicians with those generated by the o1 and 4o models. The o1 model performed better or on par with the physicians, especially during initial triage. In triage cases, the o1 model achieved a close diagnosis in 67% of instances, while the physicians had rates of 55% and 50%. The study emphasized that no data preprocessing was done, ensuring a fair comparison.

Read at TechCrunch