SUBSCRIBE
SUBSCRIBE
EXPLORE +
  • About infoDOCKET
  • Academic Libraries on LJ
  • Research on LJ
  • News on LJ
  • Skip to primary navigation
  • Skip to main content
  • Skip to primary sidebar
  • Libraries
    • Academic Libraries
    • Government Libraries
    • National Libraries
    • Public Libraries
  • Companies (Publishers/Vendors)
    • EBSCO
    • Elsevier
    • Ex Libris
    • Frontiers
    • Gale
    • PLOS
    • Scholastic
  • New Resources
    • Dashboards
    • Data Files
    • Digital Collections
    • Digital Preservation
    • Interactive Tools
    • Maps
    • Other
    • Podcasts
    • Productivity
  • New Research
    • Conference Presentations
    • Journal Articles
    • Lecture
    • New Issue
    • Reports
  • Topics
    • Archives & Special Collections
    • Associations & Organizations
    • Awards
    • Funding
    • Interviews
    • Jobs
    • Management & Leadership
    • News
    • Patrons & Users
    • Preservation
    • Profiles
    • Publishing
    • Roundup
    • Scholarly Communications
      • Open Access

February 5, 2019 by Gary Price

Center for Open Science (COS) Receives Investment by DARPA (Defense Advanced Research Projects Agency) to Improve Assessment Of Research Credibility

February 5, 2019 by Gary Price

From COS:

The Center for Open Science (COS) has been selected to participate in DARPA’s new program Systematizing Confidence in Open Research and Evidence (SCORE) with a 3-year cooperative agreement potentially totaling more than $7.6 million.  This program represents an investment by DARPA to assess and improve the credibility of social and behavioral science research.

DARPA identifies the purpose of SCORE is “to develop and deploy automated tools to assign ‘confidence scores’ to different SBS research results and claims. Confidence scores are quantitative measures that should enable a DoD consumer of SBS research to understand the degree to which a particular claim or result is likely to be reproducible or replicable.”  If successful, consumers of scientific evidence–researchers, funders, policymakers, etc.–would have readily available information about the uncertainty associated with that evidence.

[Clip]

To achieve this aspirational objective, the SCORE program will produce four primary artifacts:

  1. A massive database of approximately 30,000 claims from published papers in the social-behavioral sciences. The database will be enhanced with evidence about those claims that is automatically and manually extracted from the paper itself and merged into the database from other sources, such as how often the work has been cited and whether the data are openly accessible or the research was preregistered.
  2. Experts will review and score about 3,000 of those claims in surveys, panels, or prediction markets for their likelihood of being reproducible findings.
  3. Teams will use the database of information about the claims to generate algorithms (artificial intelligence) to score the same claims as the experts.
  4. Hundreds of researchers will conduct replications on a sample of the claims to test the experts’ and algorithms’ ability to predict reproducibility.

The COS team, in collaboration with partners from the University of Pennsylvania and Syracuse University, is responsible for creating the database.  Other teams will recruit experts and create the algorithms to evaluate the claims. And, COS will coordinate a massive collaboration of researchers from every area of the social and behavioral sciences to conduct replication and reproducibility studies.

“DARPA’s investment signals the onset of the next phase of the reformation that is underway in the social and behavioral sciences,” said Brian Nosek, Executive Director of COS and Principal Investigator of the COS team. “For the last eight years, the research community has been scrutinizing the reproducibility of its findings and the quality of its research practices.  Now, that learning is being translated into opportunities to improve research practices to accelerate the pace of discovery.”

[Clip]

Coordination of such a large-scale collaboration will be aided by the Open Science Framework (https://osf.io/) an open-source collaborative management tool that facilitates rigorous, reproducible research that is maintained by COS.  “OSF is essential for us to be able to manage such a complex program with so many contributors, projects, and associated data,” noted Beatrix Arendt, Program Manager at COS.  “We are committed to transparency of process and outcomes so that we are accountable to the research community to do the best job that we can, and so that all of our work can be scrutinized and reproduced for future research that will build on this work.”

[Clip]

Researchers interested in potentially joining this program to conduct replication or reproduction studies are encouraged to review the Call for Collaborators.

Direct to Complete Announcement

Filed under: Data Files, Journal Articles, Management and Leadership, News, Open Access, Reports

SHARE:

About Gary Price

Gary Price (gprice@gmail.com) is a librarian, writer, consultant, and frequent conference speaker based in the Washington D.C. metro area. He earned his MLIS degree from Wayne State University in Detroit. Price has won several awards including the SLA Innovations in Technology Award and Alumnus of the Year from the Wayne St. University Library and Information Science Program. From 2006-2009 he was Director of Online Information Services at Ask.com. Gary is also the co-founder of infoDJ an innovation research consultancy supporting corporate product and business model teams with just-in-time fact and insight finding.

ADVERTISEMENT

Archives

Job Zone

ADVERTISEMENT

Related Infodocket Posts

Not Real News: An Associated Press Roundup of Untrue Stories Shared Widely on Social Media This Week

From the Associated Press: A roundup of some of the most popular but completely untrue stories and visuals of the week. None of these are legit, even though they were ...

Report: "Australian Authors to Receive Compensation for E-Book Loans for First Time"

From The Sydney Morning Herald: Authors, illustrators, and editors will be compensated for e-book and audiobook library borrowings for the first time, in a move by the federal government to ...

National Archives and Records Administration (NARA) Publishes Customer Research Agenda

From the National Archives and Records Administration (NARA): A draft Customer Research Agenda was open for public review and comment in October 2022. “We’re grateful for the feedback we received ...

Report: "A Watermark for Chatbots Can Expose Text Written by an AI"

From MIT Technology Review: Hidden patterns purposely buried in AI-generated texts could help identify them as such, allowing us to tell whether the words we’re reading are written by a ...

The Accessibility of Federal Information and Data: A Brief Overview of Section 508 of the Rehabilitation Act (Updated...

From the Congressional Research Service: Nearly one in four Americans has a disability, according to 2018 estimates from the U.S. Census Bureau. Congress has recognized that in addition to making ...

NY Times: "New York Public Library Acquires Joan Didion’s Papers"

From The NY Times: When [Joan] Didion died in 2021 at age 87, the news set off an outpouring of tributes to a writer who fused penetrating insight and idiosyncratic personal voice, ...

University of North Carolina at Chapel Hill: María Estorino Named Vice Provost for University Libraries and University Librarian

Below, Find the Full Text of a Letter Sent to the Carolina Community From Kevin M. Guskiewicz University of North Carolina at Chapel Hill Chancellor Kevin M. Guskiewicz and J. ...

Boston Public Library Celebrates Black History Month with Annual “Black Is…” Booklist & Special Events

From the Boston Public Library: The Boston Public Library is proud to contribute to the celebration of Black History Month with its annual “Black Is…” booklist. The booklist aims to commemorate ...

Research Resources: New Online Tool Provides Health Snapshot of All 435 U.S. Congressional Districts (Congressional District Health Dashboard)

From NYU Langone: Researchers at NYU Grossman School of Medicine, in partnership with the Robert Wood Johnson Foundation, unveiled the Congressional District Health Dashboard (CDHD), a new online tool that ...

Report: "cOAlition S Confirms the End of Its Financial Support for Open Access Publishing Under Transformative Arrangements After...

From a cOAlition S  Announcement: Transformative arrangements – including Transformative Agreements and Transformative Journals – were developed to encourage subscription journals to transition to full and immediate open access within a defined timeframe (31st December 2024, ...

Library of Congress: Hannah Sommers Appointed New Associate Librarian for Researcher and Collections Services

From the Library of Congress: The Library of Congress announced today the appointment of Hannah Sommers as the new Associate Librarian for Researcher and Collections Services in the Library Collections and Services Group. In this role, Sommers will lead the future of the Library’s collections and the services it delivers to researchers and users. She will be central ...

Virginia Tech: University Libraries Dean Tyler Walters Appointed Board Chair of Academic Preservation Trust; IEEE Computer Society 2023...

As Book Bans Increase Across the Country, a Boston University Scholar is Fighting Back Core’s Library Resources & Technical Services Journal Goes Fully Open Access Digital Image Processing: It’s All ...

ADVERTISEMENT

FOLLOW US ON TWITTER

Tweets by infoDOCKET

ADVERTISEMENT

This coverage is free for all visitors. Your support makes this possible.

This coverage is free for all visitors. Your support makes this possible.

Primary Sidebar

  • News
  • Reviews+
  • Technology
  • Programs+
  • Design
  • Leadership
  • People
  • COVID-19
  • Advocacy
  • Opinion
  • INFOdocket
  • Job Zone

Reviews+

  • Booklists
  • Prepub Alert
  • Book Pulse
  • Media
  • Readers' Advisory
  • Self-Published Books
  • Review Submissions
  • Review for LJ

Awards

  • Library of the Year
  • Librarian of the Year
  • Movers & Shakers 2022
  • Paralibrarian of the Year
  • Best Small Library
  • Marketer of the Year
  • All Awards Guidelines
  • Community Impact Prize

Resources

  • LJ Index/Star Libraries
  • Research
  • White Papers / Case Studies

Events & PD

  • Online Courses
  • In-Person Events
  • Virtual Events
  • Webcasts
  • About Us
  • Contact Us
  • Advertise
  • Subscribe
  • Media Inquiries
  • Newsletter Sign Up
  • Submit Features/News
  • Data Privacy
  • Terms of Use
  • Terms of Sale
  • FAQs
  • Careers at MSI


© 2023 Library Journal. All rights reserved.


© 2022 Library Journal. All rights reserved.