October 22, 2021

Journal Article: “Meta-Research: Large-Scale Language Analysis of Peer Review Reports”

The article linked (accepted manuscript) to below was recently published by eLife.

Title

Meta-Research: Large-Scale Language Analysis of Peer Review Reports

Authors

Ivan Bulija
University of Split School of Medicine, Croatia

Daniel Garcia-Costa
Universitat de València, Spain

Francisco Grimaldo
Universitat de València, Spain

Flaminio Squazzoni
University of Milan, Italy

Ana Marušić
University of Split School of Medicine, Croatia

Source

eLife 2020;9:e5324
DOI: 10.7554/eLife.53249

Abstract

Peer review is often criticized for being flawed, subjective and biased, but research into peer review has been hindered by a lack of access to peer review reports. Here we report the results of a study in which text-analysis software was used to determine the linguistic characteristics of 472,449 peer review reports. A range of characteristics (including analytical tone, authenticity, clout, three measures of sentiment, and morality) were studied as a function of reviewer recommendation, area of research, type of peer review and reviewer gender. We found that reviewer recommendation had the biggest impact on the linguistic characteristics of reports, and that area of research, type of peer review and reviewer gender had little or no impact. The lack of influence of research area, type of review or reviewer gender on the linguistic characteristics is a sign of the robustness of peer review.

Direct to Full Text Article
27 pages; PDF.

About Gary Price

Gary Price (gprice@mediasourceinc.com) is a librarian, writer, consultant, and frequent conference speaker based in the Washington D.C. metro area. Before launching INFOdocket, Price and Shirl Kennedy were the founders and senior editors at ResourceShelf and DocuTicker for 10 years. From 2006-2009 he was Director of Online Information Services at Ask.com, and is currently a contributing editor at Search Engine Land.

Share