Journal Article: Search Engine Results and Confirmation Bias

Title

The Narrow Search Effect and How Broadening Search Promotes Belief Updating

Authors

Eugina Leung
Tulane University

Oleg Urminsky
University of Chicago

Source

Proceedings of the National Academy of Sciences
Vol. 122, Iss. 0

DOI: 10.1073/pnas.2408175122

Blurb

In an analysis of links between search engines and people’s beliefs, researchers examined data from 21 studies involving a total of 9,906 participants and multiple search engines and AI platforms, including Google and ChatGPT. The analysis found that users’ biased search behaviors and the narrowly focused optimization of results by search algorithms combined to support users’ existing preconceptions and beliefs. Modifying search algorithms to provide a broad range of search results could help update users’ beliefs, according to the authors.

Abstract

Information search platforms, from Google to AI-assisted search engines, have transformed information access but may fail to promote a shared factual foundation. We demonstrate that the combination of users’ prior beliefs influencing their search terms and the narrow scope of search algorithms can limit belief updating from search. We test this “narrow search effect” across 21 studies (14 preregistered) using various topics (e.g., health, financial, societal, political) and platforms (e.g., Google, ChatGPT, AI-powered Bing, our custom-designed search engine and AI chatbot interfaces). We then test user-based and algorithm-based interventions to counter the “narrow search effect” and promote belief updating. Studies 1 to 5 show that users’ prior beliefs influence the direction of the search terms, thereby generating narrow search results that limit belief updating. This effect persists across various domains (e.g., beliefs related to coronavirus, nuclear energy, gas prices, crime rates, bitcoin, caffeine, and general food or beverage health concerns; Studies 1a to 1b, 2a to 2g, 3, 4), platforms (e.g., Google—Studies 1a to 1b, 2a to 2g, 4, 5; ChatGPT, Study 3), and extends to consequential choices (Study 5). Studies 6 and 7 demonstrate the limited efficacy of prompting users to correct for the impact of narrow searches on their beliefs themselves. Using our custom-designed search engine and AI chatbot interfaces, Studies 8 and 9 show that modifying algorithms to provide broader results can encourage belief updating. These findings highlight the need for a behaviorally informed approach to the design of search algorithms.

Direct to Full Text Article

Filed under: Conference Presentations, Data Files, News, Patrons and Users