November 28, 2020

Wikimedia/Wikipedia Begins Providing Access to Anonymized Search Log Files

UPDATE (9/20): Wikipedia Releases Search Data To Public But Pulls It After Privacy Concerns (via Search Engine Land)

From a Wikimedia Blog Post by Diederik van Liere:

I am very happy to announce the availability of anonymous search log files for Wikipedia and its sister projects, as of today. Collecting data about search queries is important for at least three reasons:

+ it provides valuable feedback to our editor community, who can use it to detect topics of interest that are currently insufficiently covered.
+ we can improve our search index by benchmarking improvements against real queries.
+ we give outside researchers the opportunity to discover gems in the data.

[Clip]

Every day from today, we will publish the search queries for the previous day at: http://dumps.wikimedia.org/other/search/ (we expect to have a 3 month rolling window of search data available).

Each line in the log files is tab separated and it contains the following fields:

Server hostname
Timestamp (UTC)
Wikimedia project
URL encoded search query
Total number of results
Lucene score of best match
Interwiki result
Namespace (coded as integer)
Namespace (human-readable)
Title of best matching article

[Clip]

We are making this data available under a CC0 license: this means that you can copy, modify, distribute and perform the work, even for commercial purposes, all without asking permission. But we do appreciate it if you cite us when you use this data source for your research, experimentation or product development.

Read the Complete Blog Post

Coverage

Wikimedia releases anonymous search log files for Wikipedia (via The Next Web)

About Gary Price

Gary Price (gprice@mediasourceinc.com) is a librarian, writer, consultant, and frequent conference speaker based in the Washington D.C. metro area. Before launching INFOdocket, Price and Shirl Kennedy were the founders and senior editors at ResourceShelf and DocuTicker for 10 years. From 2006-2009 he was Director of Online Information Services at Ask.com, and is currently a contributing editor at Search Engine Land.

Share