Report: “ChatGPT is Truly Awful at Diagnosing Medical Conditions”

From LiveScience:

ChatGPT’s medical diagnoses are accurate less than half of the time, a new study reveals.

Scientists asked the artificial intelligence (AI) chatbot to assess 150 case studies from the medical website Medscape and found that GPT 3.5 (which powered ChatGPT when it launched in 2022) only gave a correct diagnosis 49% of the time. 

Previous research showed that the chatbot could scrape a pass in the United States Medical Licensing Exam (USMLE) — a finding hailed by its authors as “a notable milestone in AI maturation.” 

But in the new study, published Jul. 31 in the journal PLOS ONE, scientists cautioned against relying on the chatbot for complex medical cases that require human discernment.

“If people are scared, confused, or just unable to access care, they may be reliant on a tool that seems to deliver medical advice that’s ‘tailor-made’ for them,” senior study author Dr. Amrit Kirpalani, a doctor in pediatric nephrology at the Schulich School of Medicine and Dentistry at Western University, Ontario, told Live Science. “I think as a medical community (and among the larger scientific community) we need to be proactive about educating the general population about the limitations of these tools in this respect. They should not replace your doctor yet.”

Learn More, Read the Complete Article

See Also: Evaluation of ChatGPT as a Diagnostic Tool For Medical Learners snd Clinicians (via PLOS One)

Filed under: News, PLOS