SUBSCRIBE
SUBSCRIBE
EXPLORE +
  • About infoDOCKET
  • Academic Libraries on LJ
  • Research on LJ
  • News on LJ
  • Skip to primary navigation
  • Skip to main content
  • Skip to primary sidebar
  • Libraries
    • Academic Libraries
    • Government Libraries
    • National Libraries
    • Public Libraries
  • Companies (Publishers/Vendors)
    • EBSCO
    • Elsevier
    • Ex Libris
    • Frontiers
    • Gale
    • PLOS
    • Scholastic
  • New Resources
    • Dashboards
    • Data Files
    • Digital Collections
    • Digital Preservation
    • Interactive Tools
    • Maps
    • Other
    • Podcasts
    • Productivity
  • New Research
    • Conference Presentations
    • Journal Articles
    • Lecture
    • New Issue
    • Reports
  • Topics
    • Archives & Special Collections
    • Associations & Organizations
    • Awards
    • Funding
    • Interviews
    • Jobs
    • Management & Leadership
    • News
    • Patrons & Users
    • Preservation
    • Profiles
    • Publishing
    • Roundup
    • Scholarly Communications
      • Open Access

November 30, 2021 by Gary Price

Library of Congress: LC Labs Releases Report on Humans-in-the-Loop Machine Learning Research Framework

November 30, 2021 by Gary Price

From the Library of Congress:

Library of Congress innovation specialists examining the role of human expertise and experience in developing machine-powered research tools today released a report detailing their findings. The “Humans in the Loop” recommendation report from LC Labs details the potential and responsibility of the Library of Congress in its ongoing work to deepen access to its vast collections and share knowledge with other institutions.

The Library’s digital experiments have resulted in popular public initiatives such as By the People, the crowdsourcing platform powered by volunteer transcription, Citizen DJ, a music discovery and mixing app, and Newspaper Navigator, a machine learning algorithm that uncovered more than a million images in the Chronicling America newspaper collection. To discover the combined power of machine learning and crowdsourcing, the “Humans in the Loop” experiment investigated each step of creating a machine learning algorithm, building an engaging crowdsourcing program, and launching a prototype web experience for potential users. Together these approaches could transform access and discovery of the Library’s vast resources by combining human expertise with machine learning outputs.

“As the cultural heritage community has used more digital approaches to help our users access and discover large collections, people have wondered about the role of real humans in the future study of humanities,” said Kate Zwaard, director of Digital Strategy at the Library of Congress. “We wanted to answer that question in a way that promises to engage people, remain mindful of ethical and privacy impacts, and make our collections useful. We want to offer this report as a resource for other scholars and institutions who share these goals.”

The Library’s popular U.S. Telephone Directory collection, with its consistent layouts and fonts and unique snapshots of American communities over time, provided the ideal test sample for “Humans in the Loop.” LC Labs staff, Library subject matter experts, and partners from AVP, a data solutions provider, designed an experiment based on machine learning and crowdsourcing processes that could be created with the telephone directory’s contents. Using bounding boxes drawn around business listings and addresses in the phone books and transcriptions of these segments, the experiment team created training data to teach an algorithm to keep drawing. Wireframe mockups of sample web presentations were created for testing with potential users and for showcasing how volunteers might engage with and learn more about the collection.

Though the telephone directories are organized alphabetically with businesses categorized by industry, the research team quickly found that machine learning catalyzed a workflow for identifying the specific business name and address data that can enable flexible searching, incorporating geographic data and other Library collections. The experiment revealed the value of human expertise from volunteers and staff alike in every step of the experiment. Validating contributions and feedback on workflow design set the stage to not only improve the discovery of related information and context, but also to return exponential dividends. The humans following manual workflows processed 119 Directory listings; through this careful analysis this work seeded the machine learning workflow that generated 15,000 listings in just four days.

While the complete findings of the “Humans in the Loop” report can be found here, two major themes emerged: designing flexible and informed approaches and major investment in staffing and resources will enable sustained success. No two collections are exactly the same, so the processes outlined in “Humans in the Loop” are not a one-size-fits-all solution, and there is no substitute for human enthusiasm for problem solving.

Direct to Humans in the Loop Website/Resources

Direct to Final Recommendations Document
(97 pages; PDF)

Learn More: Visit LC Labs

Filed under: Data Files, Libraries, News, Patrons and Users

SHARE:

About Gary Price

Gary Price (gprice@gmail.com) is a librarian, writer, consultant, and frequent conference speaker based in the Washington D.C. metro area. He earned his MLIS degree from Wayne State University in Detroit. Price has won several awards including the SLA Innovations in Technology Award and Alumnus of the Year from the Wayne St. University Library and Information Science Program. From 2006-2009 he was Director of Online Information Services at Ask.com. Gary is also the co-founder of infoDJ an innovation research consultancy supporting corporate product and business model teams with just-in-time fact and insight finding.

ADVERTISEMENT

Archives

Job Zone

ADVERTISEMENT

Related Infodocket Posts

Indiana School Librarians Worry New Law Banning 'Obscene' Books Will Harm Their Work and Students; Chicago Sun-Times Introduces...

Bloomsbury: Survival of Publishers Points to AI Prophecy Overkill (via FT, Subs Only) ||| Archived Version Chicago Sun-Times Introduces a ‘Right to Be Forgotten’ Policy Indiana School Librarians Worry New ...

Journal Article: "Global Trends in Digital Preservation: Outsourcing Versus In-House Practices"

The article linked below was recently published by the Journal of Librarianship and Information Science (JOLIS). Title Global Trends in Digital Preservation: Outsourcing Versus In-House Practices Authors Rafiq AhmadBacha Khan ...

Not Real News: An Associated Press Roundup of Untrue Stories Shared Widely on Social Media This Week

From the Associated Press: A roundup of some of the most popular but completely untrue stories and visuals of the week. None of these are legit, even though they were ...

Report: Lawsuit Challenges Arkansas Law Allowing Librarians to Be Criminally Charged Over ‘Harmful’ Materials; Freedom to Read Foundation...

From the Arkansas Times A group of public libraries and supporters filed a federal lawsuit Friday to challenge a new state law that aims to censor what books children can get to ...

Yale Launches LUX, A Powerful New Search Tool For Cross-Collection Exploration

From the Yale Library: LUX: Collection Discovery—a new cross-collection search tool—provides users worldwide with online access to more than 17 million items within Yale University’s museums, libraries, and archives. “The ...

Five New or Recently Updated Reports From the Congressional Research Service (CRS)

A small selection of new or recently updated reports from the Congressional Research Service. Is That Climate Change? The Science of Extreme Event Attribution Juneteenth: Fact Sheet Montana’s TikTok Ban ...

Gavin Newsom Warns California Schools That Ban Books Will Answer to the Attorney General

From The Sacramento Bee: Gov. Gavin Newsom sent a stern message Thursday to school leaders across California — any attempt to ban books from classrooms or libraries may require them ...

Joint Statement: Massachusetts Library Organizations Stand with Librarians Against Censorship and Intolerance

Here’s the Full Text of a Statement From: The Massachusetts Board of Library Commissioners (MBLC) The Massachusetts Library Association (MLA) The Massachusetts Library System (MLS) The Massachusetts School Library Association ...

Library and Archives Canada Announces 1931 Census of Canada is Now Available Online

From a Library and Archives Canada Library and Archives Canada (LAC) is proud and excited to offer access to the digitized 1931 Census of Canada, 92 years after it was conducted. ...

Op/Ed: In Washington State, "State, Local Libraries Rebuilding Lives After Prison"

From an Everett Herald Commentary by Washington Sec. of State, Steve Hobbs and Sara Jones, Washington State Librarian: In 2016, Gov. Jay Inslee prompted state and local agencies to collaborate ...

Coordinating Research Data Services: Key Barriers and Questions; The DAISY Consortium Publishes 2022 Annual Report; and More Headlines

COAR Community Consultation on Managing Non-English And Multilingual Content In Repositories (via COAR) Coordinating Research Data Services: Key Barriers and Questions (via Ithaka S+R) The DAISY Consortium Publishes 2022 Annual ...

Association of American Publishers (AAP) Reports Publishing Revenues Totaled $28.10 Billion for 2022, Revenues Down 2.6% on Year-Over-Year...

Here’s the Full Text of an AAP StatShot Report Posted Today: The Association of American Publishers (AAP) today released the StatShot Annual report covering the calendar year 2022, estimating that ...

ADVERTISEMENT

FOLLOW US ON TWITTER

Tweets by infoDOCKET

ADVERTISEMENT

This coverage is free for all visitors. Your support makes this possible.

This coverage is free for all visitors. Your support makes this possible.

Primary Sidebar

  • News
  • Reviews+
  • Technology
  • Programs+
  • Design
  • Leadership
  • People
  • COVID-19
  • Advocacy
  • Opinion
  • INFOdocket
  • Job Zone

Reviews+

  • Booklists
  • Prepub Alert
  • Book Pulse
  • Media
  • Readers' Advisory
  • Self-Published Books
  • Review Submissions
  • Review for LJ

Awards

  • Library of the Year
  • Librarian of the Year
  • Movers & Shakers 2022
  • Paralibrarian of the Year
  • Best Small Library
  • Marketer of the Year
  • All Awards Guidelines
  • Community Impact Prize

Resources

  • LJ Index/Star Libraries
  • Research
  • White Papers / Case Studies

Events & PD

  • Online Courses
  • In-Person Events
  • Virtual Events
  • Webcasts
  • About Us
  • Contact Us
  • Advertise
  • Subscribe
  • Media Inquiries
  • Newsletter Sign Up
  • Submit Features/News
  • Data Privacy
  • Terms of Use
  • Terms of Sale
  • FAQs
  • Careers at MSI


© 2023 Library Journal. All rights reserved.


© 2022 Library Journal. All rights reserved.