Journal Article (preprint): “Chatbot Assessment: Best Practices for Artificial Intelligence in the Library”

The article (approved for publication, preprint) is scheduled to appear in portal: Libraries and the Academy (Volume 25, Number 4).

Title

Chatbot Assessment: Best Practices for Artificial Intelligence in the Library

Authors

Michelle Ehrenpreis
Lehman College, CUNY

John DeLooper
Lehman College, CUNY

Source

portal: Libraries and the Academy
Volume 25, Number 4) (preprint)

Abstract

In November 2019, the Leonard Lief Library implemented Ivy.ai, a proprietary chatbot on its website. This implementation was the first academic library installation of a vendor-supplied chatbot to be discussed in the professional literature. This chatbot functioned as a new tool that assisted users seeking information from the library website. User questions provided insight to the authors about the kinds of topics students searched for via the library website. In April 2023, the chatbot’s vendor began using OpenAI’s ChatGPT Application Programming Interface (API) to improve the chatbot’s functionality. This change, from a rules-based chatbot system to a transformer model, enhanced the chatbot’s ability to provide answers to patrons. To better understand this major change, the authors assessed the chatbot’s usage during the Spring 2023 semester. This assessment revealed the kinds of questions the chatbot struggled to answer, and possible reasons why. The assessment’s findings demonstrated how chatbots can successfully function as a enhancement to the library website. The article also presents best practices for libraries looking to implement or experiment with chatbots and contributes to the ongoing discussion of artificial intelligence in libraries.

Direct to Full Text (preprint)
34 pages; PDF. 

See Also: More Preprints From portal: Libraries and the Academy (Volume 25, Number 4)

Filed under: Academic Libraries, Libraries, News, Patrons and Users, Reports