Report/Backgrounder: Artificial Intelligence, Automation and Peer Review

A new backgrounder (7 pages; PDF) from JISC.  Mike Thelwall, University of Wolverhampton, is the author.

From the Summary:

This document summarises recent developments in the automation of the editorial and peer review process. It discusses the opportunities for AI to support editors and reviewers, as well as the ethical challenges in terms of the potential to cause bias in the publishing system. It focuses most on the challenges for adopting automated peer review and its potential to cause unintended bias, arguing that any type of artificial intelligence for overall article quality or future impact assessment could generate international biases against authors in non-English speaking countries, including most poorer countries. Essentially, this bias would occur because an AI system would learn features that associate with authors from high impact countries, such as the USA, UK and Australia, as markers of high quality, penalising articles from lower overall impact countries, irrespective of the quality of the submitted articles. Thus, many good researchers would suffer from publishing from the same country as less good researchers, despite the additional skill that is needed to produce high quality research in less well-resourced countries. This could be addressed by adding bias correction factors to counteract likely biases, if developers could demonstrate that such correction factors eliminated the problem overall.

The document is organized into the following sections:

• Summary
• Overview of Existing Tools
• Ethical Considerations and Algorithmic Accountability
• AI, Automation, Language and Second Order National Biases
• Ethical Considerations When Using Automation
• Predicting Article Quality Or Future Citation Impact with AI
• References

Direct to Full Text
7 pages; PDF.

Filed under: News, Publishing, Reports