New From ScholCommLab: “Becoming Metrics Literate: New Study Examines Videos on the H-index”

From a ScholCommLab Post:

Scholarly metrics are widely applied to assess research quality and impact despite their known limitations. One of the most popular scholarly metrics is the h-index—which is defined as the “h number of papers with at least h number of citations.” This means that if a researcher has an h-index of 12, they have published 12 papers with at least 12 citations each.

Although simple, the h-index is flawed and often applied inappropriately. It’s been criticized due to inconsistencies, lack of field normalization, database-dependence, and time-dependence. Yet the h-index continues to be used for hiring, review, and tenure.

Lauren Maggio, professor of medicine at the Uniformed Services University of Health Sciences, has witnessed how academics misuse metrics in high-stakes decisions. Across her career, sitting on many hiring committees, she has “unfortunately seen candidates compared in relation to their h-index,” despite its shortcomings.

[Clip]

The project, led by Stefanie Haustein, co-director of the ScholCommLab, aims to improve the appropriate use of scholarly metrics in academia. The team is developing educational videos and evaluating their efficiency in a randomized controlled trial to teach academics “Metrics Literacies,” the skills and knowledge to recognize, interpret, critically assess, and effectively and ethically use scholarly metrics.

[Clip]

In their first study, recently published in PLOS ONE, the Metrics Literacies team examined the characteristics of available educational videos on the h-index. They identified more than 247 videos to get a sense of how the h-index is described and presented. “We watched a ton of YouTube,” laughed Lauren.

Only 31 videos met the inclusion criteria (English-speaking, less than 10 mins long, and actually about the h-index). At least two of the authors independently coded each of the videos based on their content, presentation style, and educational quality.

[Clip]

While most videos defined or calculated the h-index, 48% of videos did not mention any caveats. There were no mentions of disadvantages to early career researchers (77%), database differences (71%), or disciplinary differences (61%). In fact, Lauren saw that “There were presenters very much proposing and helping viewers to think about how to boost their h-index, which shouldn’t really have been a surprise.” The lack of critical reflection in the videos fails to provide transparency on the shortcomings of the h-index.

Learn More, View Graphics in the Complete Post

Filed under: Journal Articles, News, PLOS, Video Recordings