The Data Refuge Stories project has really taken off and doing great work (check out the revamped website!). But a question we get asked frequently is what happened to the data collected during Data Rescue events?
From January to May 2017 over 400 datasets from 33 agencies were collected at about 50 data rescue events. The workflow* for harvesting and preparing data for datarefuge.org was developed mostly by Delphine Khanna and Rachel Appel from Temple University Libraries, Laurie Allen from UPenn Libraries, and Justin Schell from University of Michigan Libraries. Many others at DataRescue Philly also contributed. The issue of trust was very important to us in archiving this data. While our workflow isn’t the best method there are multiple validation points and a documented chain of custody.
So far, our student workers have unpacked about 100 datasets. They’ve uncovered just a few duplicate datasets, datasets that were too messy to be usable, or datasets that were no longer available at their original location. We do not have metrics on how many, if any, of these datasets are being used. We only know about data that goes missing on government website if someone reports it to us or the media (EDGI is doing some great work on monitoring changes to websites though). What we’ve heard has gone missing has most often been moved to an archival sitethrough the work of the National Archives and Records Administration or by some other means. Other missing data was retained by the Internet Archive. There are of course a few things we know have been taken down or are planned to be removed and are unfortunately not in our repository or rescued by any other group we know of.
Data Refuge Project Shares Statistics, Update in New Blog Post
Filed by August 3, 2018on