Journal Article: “Can ChatGPT Be Used To Predict Citation Counts, Readership, and Social Media Interaction? An Exploration Among 2222 Scientific Abstracts”
The article (full text) linked below was recently published by Scientometrics.
Title
Can ChatGPT Be Used To Predict Citation Counts, Readership, and Social Media Interaction? An Exploration Among 2222 Scientific Abstracts
Authors
Joost de Winter
Delft University of Technology
Source
Scientometrics
February 15, 2024
DOI: 10.1007/s11192-024-04939-y
Abstract
This study explores the potential of ChatGPT, a large language model, in scientometrics by assessing its ability to predict citation counts, Mendeley readers, and social media engagement. In this study, 2222 abstracts from PLOS ONE articles published during the initial months of 2022 were analyzed using ChatGPT-4, which used a set of 60 criteria to assess each abstract. Using a principal component analysis, three components were identified: Quality and Reliability, Accessibility and Understandability, and Novelty and Engagement. The Accessibility and Understandability of the abstracts correlated with higher Mendeley readership, while Novelty and Engagement and Accessibility and Understandability were linked to citation counts (Dimensions, Scopus, Google Scholar) and social media attention. Quality and Reliability showed minimal correlation with citation and altmetrics outcomes. Finally, it was found that the predictive correlations of ChatGPT-based assessments surpassed traditional readability metrics. The findings highlight the potential of large language models in scientometrics and possibly pave the way for AI-assisted peer review.
Direct to Full Text Article
About Gary Price
Gary Price (gprice@gmail.com) is a librarian, writer, consultant, and frequent conference speaker based in the Washington D.C. metro area. He earned his MLIS degree from Wayne State University in Detroit. Price has won several awards including the SLA Innovations in Technology Award and Alumnus of the Year from the Wayne St. University Library and Information Science Program. From 2006-2009 he was Director of Online Information Services at Ask.com.