New From ScholCommLab: “Becoming Metrics Literate: New Study Examines Videos on the H-index”
From a ScholCommLab Post:
Scholarly metrics are widely applied to assess research quality and impact despite their known limitations. One of the most popular scholarly metrics is the h-index—which is defined as the “h number of papers with at least h number of citations.” This means that if a researcher has an h-index of 12, they have published 12 papers with at least 12 citations each.
Although simple, the h-index is flawed and often applied inappropriately. It’s been criticized due to inconsistencies, lack of field normalization, database-dependence, and time-dependence. Yet the h-index continues to be used for hiring, review, and tenure.
Lauren Maggio, professor of medicine at the Uniformed Services University of Health Sciences, has witnessed how academics misuse metrics in high-stakes decisions. Across her career, sitting on many hiring committees, she has “unfortunately seen candidates compared in relation to their h-index,” despite its shortcomings.
The project, led by Stefanie Haustein, co-director of the ScholCommLab, aims to improve the appropriate use of scholarly metrics in academia. The team is developing educational videos and evaluating their efficiency in a randomized controlled trial to teach academics “Metrics Literacies,” the skills and knowledge to recognize, interpret, critically assess, and effectively and ethically use scholarly metrics.
In their first study, recently published in PLOS ONE, the Metrics Literacies team examined the characteristics of available educational videos on the h-index. They identified more than 247 videos to get a sense of how the h-index is described and presented. “We watched a ton of YouTube,” laughed Lauren.
Only 31 videos met the inclusion criteria (English-speaking, less than 10 mins long, and actually about the h-index). At least two of the authors independently coded each of the videos based on their content, presentation style, and educational quality.
While most videos defined or calculated the h-index, 48% of videos did not mention any caveats. There were no mentions of disadvantages to early career researchers (77%), database differences (71%), or disciplinary differences (61%). In fact, Lauren saw that “There were presenters very much proposing and helping viewers to think about how to boost their h-index, which shouldn’t really have been a surprise.” The lack of critical reflection in the videos fails to provide transparency on the shortcomings of the h-index.
About Gary Price
Gary Price (email@example.com) is a librarian, writer, consultant, and frequent conference speaker based in the Washington D.C. metro area. He earned his MLIS degree from Wayne State University in Detroit. Price has won several awards including the SLA Innovations in Technology Award and Alumnus of the Year from the Wayne St. University Library and Information Science Program. From 2006-2009 he was Director of Online Information Services at Ask.com. Gary is also the co-founder of infoDJ an innovation research consultancy supporting corporate product and business model teams with just-in-time fact and insight finding.