Scientific Data Volume 9, Article number: 60 (2022)
DOI:
Abstract
This article presents a study on the quality and execution of research code from publicly-available replication datasets at the Harvard Dataverse repository. Research code is typically created by a group of scientists and published together with academic papers to facilitate research transparency and reproducibility. For this study, we define ten questions to address aspects impacting research reproducibility and reuse. First, we retrieve and analyze more than 2000 replication datasets with over 9000 unique R files published from 2010 to 2020.
Source: 10.1038/s41597-022-01143-6
Second, we execute the code in a clean runtime environment to assess its ease of reuse. Common coding errors were identified, and some of them were solved with automatic code cleaning to aid code execution. We find that 74% of R files failed to complete without error in the initial execution, while 56% failed when code cleaning was applied, showing that many errors can be prevented with good coding practices. We also analyze the replication datasets from journals’ collections and discuss the impact of the journal policy strictness on the code re-execution rate. Finally, based on our results, we propose a set of recommendations for code dissemination aimed at researchers, journals, and repositories.
Gary Price (gprice@gmail.com) is a librarian, writer, consultant, and frequent conference speaker based in the Washington D.C. metro area.
He earned his MLIS degree from Wayne State University in Detroit.
Price has won several awards including the SLA Innovations in Technology Award and Alumnus of the Year from the Wayne St. University Library and Information Science Program. From 2006-2009 he was Director of Online Information Services at Ask.com. Gary is also the co-founder of infoDJ an innovation research consultancy supporting corporate product and business model teams with just-in-time fact and insight finding.
From Fox 17 (Grand Rapids): The folks over at the Grand Rapids Public Library made a fascinating discovery while digging through their massive archives back in March 2021, and are ...
The article linked below was recently published by the International Journal of Communication. Title Knowledge Work in Platform Fact-Checking Partnerships Authors Valérie Bélair-Gagnon University of Minnesota-Twin Cities, USA Rebekah Larsen ...
A Guide to Communicating With Others: Messaging Apps (via Privacy International) De Gruyter Acquires Mercury Learning and Information Report by the French Committee for Open Science Working Group on Electronic ...
From an Internet Archive Blog Post by Jason Scott: It’s time to add another family of emulated older technology to the Internet Archive. The vast majority of platforms within what ...
The article linked below was recently published by Quantitative Science Studies. Title Crossref as a Bibliographic Discovery Tool in the Arts and Humanities Authors Ángel Borrego Universitat de Barcelona, Melcior ...
Colorado: Suspensions Increase at Pikes Peak Library District Under New Security Protocols (via The Gazette) Montana: ImagineIF Trustees Hold Special Meeting on Library Security Concerns (via Daily Inter Mountain) North ...
From the Associated Press: A roundup of some of the most popular but completely untrue stories and visuals of the week. None of these are legit, even though they were ...
From The Sydney Morning Herald: Authors, illustrators, and editors will be compensated for e-book and audiobook library borrowings for the first time, in a move by the federal government to ...
From the National Archives and Records Administration (NARA): A draft Customer Research Agenda was open for public review and comment in October 2022. “We’re grateful for the feedback we received ...
From MIT Technology Review: Hidden patterns purposely buried in AI-generated texts could help identify them as such, allowing us to tell whether the words we’re reading are written by a ...
From the Congressional Research Service: Nearly one in four Americans has a disability, according to 2018 estimates from the U.S. Census Bureau. Congress has recognized that in addition to making ...
From The NY Times: When [Joan] Didion died in 2021 at age 87, the news set off an outpouring of tributes to a writer who fused penetrating insight and idiosyncratic personal voice, ...
