Report: “Historians Use Data Science to Mine The Past”

From a Recently Published Article From PNAS (Proceedings of the National Academy of Sciences of the United States of America) by Carolyn Beans:

Historians today can perform computational tasks as they dig for insights—using, for example, computational models to reveal how often two words appear together in texts, launching network analyses to link individuals who appear in the same documents, or training computer vision models to recognize key features on digitized maps.

With these tools in hand, historians ask questions that would be difficult to answer with a lifetime of scrolling through microfiche. When and where did the originators of the anti-government expansion movement in the United States first meet? Who led the spy group that sent intel to the United States from Cuba in the late 1800s?

But practitioners of this field, known as digital history, understand that they are working with incomplete data. Most records have yet to be digitized, and many that are contain biases. And there are dangers in mining words from decades past without a deep understanding of their context.

[Clip]

One challenge is that databases to which scholars turn with these new methods may be incomplete in ways that aren’t obvious. For example, a newspaper might be included in a database, but if OCR didn’t capture its pages properly, the text wouldn’t be searchable.

Case in point: In 2023, research led by historians at The Alan Turing Institute in London, United Kingdom, revealed that searching the British Library’s digitized newspaper collection for information about life during the Industrial Revolution would return politically biased results. The reason: OCR was better at reading the fonts favored by more expensive and conservative papers than those used by less expensive, liberal ones (4).

Even when OCR works as planned, databases only capture a sliver of archival records worldwide, though those numbers are steadily growing, [historian Jo] Guldi says. In the United States, as of 2023, the Library of Congress had digitized more than 9 million records, including newspapers, manuscripts, and the personal papers of every president from George Washington to Calvin Coolidge (5).

Learn More, Read the Complete Article (about 2300 words)

Source: 10.1073/pnas.2508428122

Filed under: Conference Presentations, Data Files, Journal Articles, Libraries, Maps, News