Research Article (Preprint): “Artificial Intelligence’s Contribution to Biomedical Literature Search: Revolutionizing or Complicating?”

The article (preprint) linked below was recently shared on bioRxiv.

Title

Artificial Intelligence’s Contribution to Biomedical Literature Search: Revolutionizing or Complicating?

Authors

Rui Yip
Stanford University

Young Joo Sun
Stanford University

Alexander G. Bassuk
University of Iowa

Vinit B. Mahajan
Stanford University
University of Iowa
Palo Alto Health Care System

Source

via bioRxiv

DOI: 10.1101/2024.10.07.617112

Abstract

There is a growing number of articles about conversational AI (i.e., ChatGPT) for generating scientific literature reviews and summaries. Yet, comparative evidence lags its wide adoption by many clinicians and researchers. We explored ChatGPT’s utility for literature search from an end-user perspective through the lens of clinicians and biomedical researchers. We quantitatively compared basic versions of ChatGPT’s utility against conventional search methods such as Google and PubMed. We further tested whether ChatGPT user-support tools (i.e., plugins, web-browsing function, prompt-engineering, and custom-GPTs) could improve its response across four common and practical literature search scenarios: (1) high-interest topics with an abundance of information, (2) niche topics with limited information, (3) scientific hypothesis generation, and (4) for newly emerging clinical practices questions. Our results demonstrated that basic ChatGPT functions had limitations in consistency, accuracy, and relevancy. User-support tools showed improvements, but the limitations persisted. Interestingly, each literature search scenario posed different challenges: an abundance of secondary information sources in high interest topics, and uncompelling literatures for new/niche topics. This study tested practical examples highlighting both the potential and the pitfalls of integrating conversational AI into literature search processes, and underscores the necessity for rigorous comparative assessments of AI tools in scientific research.

Direct to Full Text Article
21 pages; PDF.

Filed under: News