New Article: “Google Scholar as Replacement for Systematic Literature Searches: Good Relative Recall and Precision are Not Enough”

The following article was published online today.

Title

Google Scholar as replacement for systematic literature searches: good relative recall and precision are not enough

Authors

Martin Boeker, Werner Vach, Edith Motschall
Institute of Medical Biometry and Medical Informatics, Department of Medical Biometry and Statistics, University Medical Center Freiburg 

 Source

BMC Medical Research Methodology 2013
13:131

From the Abstract

Recent research indicates a high recall in Google Scholar searches for systematic reviews. These reports raised high expectations of Google Scholar as a unified and easy to use search interface. However, studies on the coverage of Google Scholar rarely used the search interface in a realistic approach but instead merely checked for the existence of gold standard references. In addition, the severe limitations of the Google Search interface must be taken into consideration when comparing with professional literature retrieval tools.
We investigated Cochrane reviews with a number of included references between 11 and 70 with a total of 396 references. The Google Scholar searches resulted in sets between 4,320 and 67,800 and a total of 291,190 hits. The relative recall of the Google Scholar searches had a minimum of 76.2% and a maximum of 100% (7 searches). The precision of the Google Scholar searches had a minimum of 0.05% and a maximum of 0.92%. The overall relative recall for all searches was 92.9%, the overall precision was 0.13%.
The reported relative recall must be interpreted with care. It is a quality indicator of Google Scholar confined to an experimental setting which is unavailable in systematic retrieval due to the severe limitations of the Google Scholar search interface. Currently, Google Scholar does not provide necessary elements for systematic scientific literature retrieval such as tools for incremental query optimization, export of a large number of references, a visual search builder or a history function. Google Scholar is not ready as a professional searching tool for tasks where structured retrieval methodology is necessary.

Direct to Full Text ||| Provisional PDF Version
See Also: Research Paper: “The Google Scholar Experiment: How to Index False Papers and Manipulate Bibliometric Indicators”(September 11, 2013)

Filed under: Journal Articles, News, Reports