Research Article: “An Evidence-Based Method for Assessing the Value of a Search Tool: A Pilot Study”

The following article appears in the new issue of the Journal of the Medical Library Association (JMLA).
Title
An Evidence-Based Method for Assessing the Value of a Search Tool: A Pilot Study
Authors
Donald Stanley Pearson
The Ohio State University Comprehensive Cancer Center
Stevo Roksandic
Mount Carmel Health System
Jill Kilanowski
Mount Carmel College of Nursing
Source
Journal of the Medical Library Association (JMLA)
Vol 106, No 4 (2018)
DOI: 10.5195/jmla.2018.287
Abstract

Objective
The objective of this study was to develop an evidence-based method with a set of metrics that could be used to assess an information search tool.
Methods
This pilot study analyzed a two-group convenience sample of graduate nursing students and resident physicians. The intervention group received ten minutes of instruction on a familiar search tool (eSearcher). Each group was provided one prompt to search for clinical guidelines on a given topic within their scope of practice and asked to find the best result using only eSearcher (intervention group) or specifically excluding eSearcher (comparison group). Three measurements of search results were employed: time elapsed to complete the search, an accuracy score, and a participant-reported score of confidence in the result.
Results
Forty-two students participated in this study (23 graduate nursing students and 19 resident physicians). The intervention group consisted of 22 participants (12 graduate nursing students and 10 resident physicians), and the comparison group consisted of 20 participants (11 graduate nursing students and 9 resident physicians). The intervention group had lower mean ranks in both accuracy and confidence compared to the comparison (not eSearcher) group, although these differences were not significant. However, the intervention (eSearcher) group had significantly longer search times compared to the comparison (not eSearcher) group.
Discussion
These findings provided new insights into the performance of the search tool and how users felt about their search experience. The quantitative evidence gained from this study led directly to an informed decision to explore other options for search tools. The evidence-based methods and process developed in this pilot study will enable similar studies to test other student groups and other search tools, leading to better informed purchasing and instructional decisions.

Direct to Full Text ||| PDF Version (6 pages; PDF)

Filed under: Libraries, New Issue, News, Patrons and Users