Aliakbar Akbaritabar Max Planck Institute for Demographic Research
Dimity Stephen German Centre for Higher Education Research and Science Studies
Source
via arXiv
DOI
Abstract
This paper provides insight into the changes manuscripts undergo during peer review, the potential reasons for these changes, and the differences between scientific fields. A growing body of literature is assessing the effect of peer review on manuscripts, however much of this research currently focuses on the social and medical sciences. We matched more than 6,000 preprint-publication pairs across multiple fields and quantified the changes in their reference lists.
We also quantified the change in references per full-text section for 565 pairs from PLOS journals. In addition, we conducted manual checks of a randomly chosen sample of 98 pairs to validate our results, and undertook a qualitative analysis based on the context of the reference to investigate the potential reasons for reference changes.
We found 10 disciplines, mostly in the natural sciences with high levels of removed references. Methods sections undergo the most relative change in the natural sciences, while in the medical and health sciences, the results and discussion sections underwent the most changes. Our qualitative analysis identified issues with our results due to incomplete preprint reference lists.
In addition, we deduced 10 themes for changing references during peer review. This analysis suggested that manuscripts in the natural and medical sciences undergo more extensive reframing of the literature used to situate and interpret the results of studies than the social and agricultural sciences, which are further embedded in the existing literature through peer review. Peer review in engineering tends to focus on methodological details. Our results are useful to the body of literature examining the effectiveness of peer review in fulfilling its intended purposes.
Gary Price (gprice@gmail.com) is a librarian, writer, consultant, and frequent conference speaker based in the Washington D.C. metro area.
He earned his MLIS degree from Wayne State University in Detroit.
Price has won several awards including the SLA Innovations in Technology Award and Alumnus of the Year from the Wayne St. University Library and Information Science Program. From 2006-2009 he was Director of Online Information Services at Ask.com.
