The federal government’s Defense Advanced Research Project Agency today made publicly available an open-source tool that could lead to an entire new ecosystem of search-engines. Seventeen organizations received early access to the project, called Memex.
The technology is designed to extend current search capabilities to the deep web, the dark web, multimedia, and other content. The 17 organizations have so far produced 40 different tools.
Though the tools are all open-source, that doesn’t mean they’re free. Licensing varies from project to project, and only three of the 40 so far — those developed by Georgetown — are part of the public domain. Each project was “partially funded” by DARPA.
Today’s web searches use a centralized, one-size-fits-all approach that searches the Internet with the same set of tools for all queries. While that model has been wildly successful commercially, it does not work well for many government use cases. To help overcome these challenges, DARPA launched the Memex program in September 2014.
Memex seeks to develop software that advances online search capabilities far beyond the current state of the art.
The goal is to invent better methods for interacting with and sharing information, so users can quickly and thoroughly organize and search subsets of information relevant to their individual interests. Creation of a new domain-specific indexing and search paradigm will provide mechanisms for improved content discovery, information extraction, information retrieval, user collaboration, and extension of current search capabilities to the deep web, the dark web, and nontraditional (e.g. multimedia) content.