SUBSCRIBE
SUBSCRIBE
EXPLORE +
  • About infoDOCKET
  • Academic Libraries on LJ
  • Research on LJ
  • News on LJ
  • Skip to primary navigation
  • Skip to main content
  • Skip to primary sidebar
  • Libraries
    • Academic Libraries
    • Government Libraries
    • National Libraries
    • Public Libraries
  • Companies (Publishers/Vendors)
    • EBSCO
    • Elsevier
    • Ex Libris
    • Frontiers
    • Gale
    • PLOS
    • Scholastic
  • New Resources
    • Dashboards
    • Data Files
    • Digital Collections
    • Digital Preservation
    • Interactive Tools
    • Maps
    • Other
    • Podcasts
    • Productivity
  • New Research
    • Conference Presentations
    • Journal Articles
    • Lecture
    • New Issue
    • Reports
  • Topics
    • Archives & Special Collections
    • Associations & Organizations
    • Awards
    • Funding
    • Interviews
    • Jobs
    • Management & Leadership
    • News
    • Patrons & Users
    • Preservation
    • Profiles
    • Publishing
    • Roundup
    • Scholarly Communications
      • Open Access

July 9, 2019 by Gary Price

Hate Spoilers? This AI Tool Spots Them For You

July 9, 2019 by Gary Price

From UC San Diego:

Did social media spoil the Avengers’ Endgame movie for you? Or maybe one of the Game of Thrones books? A team of researchers from the University of California San Diego is working to make sure that doesn’t happen again. They have developed an AI-based system that can flag spoilers in online reviews of books and TV shows.

“Spoilers are everywhere on the internet, and are very common on social media. As internet users, we understand the pain of spoilers, and how they can ruin one’s experience,” said Ndapa Nakashole, a professor of computer science at UC San Diego and one of the paper’s senior authors.

Some websites allow people to manually flag their posts with tags that serve as ‘spoiler ahead’ warning signs. But this doesn’t always happen. So researchers wanted to develop an artificial intelligence tool powered by neural networks to automatically detect spoilers. They named the tool SpoilerNet.

On a theoretical level, researchers want to better understand how people write spoilers and what kind of linguistic patterns and common knowledge mark a sentence as a spoiler.

Researchers will present their findings at the 2019 annual meeting of the Association for Computational Linguistics in Florence, Italy, July 28 to Aug. 2. The tool the researchers developed could be used to build a browser extension to shield people from spoilers.

To train and test SpoilerNet, the UC San Diego team went looking for large datasets of sentences containing spoilers. Spoiler alert! They found none. So they created their own by collecting more than 1.3 million book reviews annotated with spoiler tags by book reviewers. The tags encompass sentences that include spoilers and hide them behind a “view spoiler” link in the text. The reviews were collected from Goodreads, a social networking site that allows people to track what they read, and share thoughts and reviews with other readers.

[Clip]

Researchers trained SpoilerNet on 80 percent of the reviews on Goodreads, running the text through several layers of neural networks. The system could detect spoilers with 89 to 92 percent accuracy.

They also ran SpoilerNet on a dataset of more than 16,000 single-sentence reviews of about 880 TV shows. The accuracy of the tool to detect spoilers was 74 to 80 percent.

Most of the errors came from the system getting distracted by words that are usually loaded and revelatory–for example murder or killed.

Looking forward, the Goodreads dataset can be used as a powerful tool to train algorithms to detect spoilers in different types of content–say, tweets containing spoilers.

Direct to Complete Article

Direct to Full Text Conference Paper: Fine-Grained Spoiler Detection from Large-Scale Review Corpora (via arXiv)

Filed under: Data Files, Journal Articles, News, Patrons and Users

SHARE:

About Gary Price

Gary Price (gprice@gmail.com) is a librarian, writer, consultant, and frequent conference speaker based in the Washington D.C. metro area. He earned his MLIS degree from Wayne State University in Detroit. Price has won several awards including the SLA Innovations in Technology Award and Alumnus of the Year from the Wayne St. University Library and Information Science Program. From 2006-2009 he was Director of Online Information Services at Ask.com. Gary is also the co-founder of infoDJ an innovation research consultancy supporting corporate product and business model teams with just-in-time fact and insight finding.

ADVERTISEMENT

Archives

Job Zone

ADVERTISEMENT

Related Infodocket Posts

Report: "Australian Authors to Receive Compensation for E-Book Loans for First Time"

From The Sydney Morning Herald: Authors, illustrators, and editors will be compensated for e-book and audiobook library borrowings for the first time, in a move by the federal government to ...

National Archives and Records Administration (NARA) Publishes  Customer Research Agenda

From the National Archives and Records Administration (NARA): The National Archives and Records Administration (NARA) has posted its . A draft Customer Research Agenda was open for public review and comment ...

Report: "A Watermark for Chatbots Can Expose Text Written by an AI"

From MIT Technology Review: Hidden patterns purposely buried in AI-generated texts could help identify them as such, allowing us to tell whether the words we’re reading are written by a ...

The Accessibility of Federal Information and Data: A Brief Overview of Section 508 of the Rehabilitation Act (Updated...

From the Congressional Research Service: Nearly one in four Americans has a disability, according to 2018 estimates from the U.S. Census Bureau. Congress has recognized that in addition to making ...

NY Times: "New York Public Library Acquires Joan Didion’s Papers"

From The NY Times: When [Joan] Didion died in 2021 at age 87, the news set off an outpouring of tributes to a writer who fused penetrating insight and idiosyncratic personal voice, ...

University of North Carolina at Chapel Hill: María Estorino Named Vice Provost for University Libraries and University Librarian

Below, Find the Full Text of a Letter Sent to the Carolina Community From Kevin M. Guskiewicz University of North Carolina at Chapel Hill Chancellor Kevin M. Guskiewicz and J. ...

Boston Public Library Celebrates Black History Month with Annual “Black Is…” Booklist & Special Events

From the Boston Public Library: The Boston Public Library is proud to contribute to the celebration of Black History Month with its annual “Black Is…” booklist. The booklist aims to commemorate ...

Research Resources: New Online Tool Provides Health Snapshot of All 435 U.S. Congressional Districts (Congressional District Health Dashboard)

From NYU Langone: Researchers at NYU Grossman School of Medicine, in partnership with the Robert Wood Johnson Foundation, unveiled the Congressional District Health Dashboard (CDHD), a new online tool that ...

Report: "cOAlition S Confirms the End of Its Financial Support for Open Access Publishing Under Transformative Arrangements After...

From a cOAlition S  Announcement: Transformative arrangements – including Transformative Agreements and Transformative Journals – were developed to encourage subscription journals to transition to full and immediate open access within a defined timeframe (31st December 2024, ...

Library of Congress: Hannah Sommers Appointed New Associate Librarian for Researcher and Collections Services

From the Library of Congress: The Library of Congress announced today the appointment of Hannah Sommers as the new Associate Librarian for Researcher and Collections Services in the Library Collections and Services Group. In this role, Sommers will lead the future of the Library’s collections and the services it delivers to researchers and users. She will be central ...

Virginia Tech: University Libraries Dean Tyler Walters Appointed Board Chair of Academic Preservation Trust; IEEE Computer Society 2023...

As Book Bans Increase Across the Country, a Boston University Scholar is Fighting Back Core’s Library Resources & Technical Services Journal Goes Fully Open Access Digital Image Processing: It’s All ...

Funding: Library Freedom Project Receives $1 Million Grant Award From the Mellon Foundation to Advance Critical Privacy and...

Here’s the Full Text of the Library Freedom Project (LFP) Announcement:   Library Freedom Project (LFP) has been awarded $1,000,000 from the Mellon Foundation to expand the program’s work. For ...

ADVERTISEMENT

FOLLOW US ON TWITTER

Tweets by infoDOCKET

ADVERTISEMENT

This coverage is free for all visitors. Your support makes this possible.

This coverage is free for all visitors. Your support makes this possible.

Primary Sidebar

  • News
  • Reviews+
  • Technology
  • Programs+
  • Design
  • Leadership
  • People
  • COVID-19
  • Advocacy
  • Opinion
  • INFOdocket
  • Job Zone

Reviews+

  • Booklists
  • Prepub Alert
  • Book Pulse
  • Media
  • Readers' Advisory
  • Self-Published Books
  • Review Submissions
  • Review for LJ

Awards

  • Library of the Year
  • Librarian of the Year
  • Movers & Shakers 2022
  • Paralibrarian of the Year
  • Best Small Library
  • Marketer of the Year
  • All Awards Guidelines
  • Community Impact Prize

Resources

  • LJ Index/Star Libraries
  • Research
  • White Papers / Case Studies

Events & PD

  • Online Courses
  • In-Person Events
  • Virtual Events
  • Webcasts
  • About Us
  • Contact Us
  • Advertise
  • Subscribe
  • Media Inquiries
  • Newsletter Sign Up
  • Submit Features/News
  • Data Privacy
  • Terms of Use
  • Terms of Sale
  • FAQs
  • Careers at MSI


© 2023 Library Journal. All rights reserved.


© 2022 Library Journal. All rights reserved.