NOAA Expands Public Access to Big Data via Collaboration with Amazon Web Services

From NOAA:

NOAA generates thousands of datasets as part of its mission to collect information on environments that span from the surface of the sun to the depths of the ocean floor.
More than 68,000 datasets are made publicly available from NOAA with the National Centers for Environmental Information (NCEI) providing access to more than half of those datasets. With such vast holdings, NOAA faces the challenge of providing access to the data in an efficient and scalable manner.
[Clip]
AWS currently hosts a variety of NOAA datasets, including both atmospheric and oceanic data. Available datasets can be discovered through the Registry of Open Data on AWS.
One of the first datasets to move to AWS through the BDP was the historical archive of Level-II Next Generation Weather Radar (NEXRAD), available through NCEI. The NEXRAD Level-II archive was among the top initial datasets of interest to move to AWS since it already has an active user community, and it has qualities that are of value to research, commercial, and federal users.
The BDP began in 2015 as a four-year business experiment to understand if the inherent value in NOAA’s data could be more fully realized if made available in the cloud. Some of NOAA’s datasets are extremely large and can be cumbersome to transfer through traditional data transfer methods, requiring substantial amounts of bandwidth and time. In some cases, there can be costs associated with the data acquisition, such as when a satellite receiver is required for satellite data.
[Clip]

In addition to NEXRAD Level-II, AWS also hosts datasets from sources such as the Geostationary Operational Environmental Satellite (GOES) 16 from NOAA National Environmental Satellite, Data, and Information Service (NESDIS), the Operational Forecast System from NOAA National Ocean Service (NOS), and the National Water Model (NWM) from NOAA National Weather Service (NWS), to name a few. These datasets highlight the variety of hosted data, ranging from focusing on satellite weather data, ocean data, to hydraulic data. When new datasets become available on the platform, NOAA subject matter experts and AWS host informational sessions to provide users an opportunity to learn more about the data.

Currently, the BDP is in its last year of the experimental phase and is working with AWS to continue hosting additional datasets. The BDP is also gathering feedback from AWS, the other collaborators, and end users that will help develop a sustainable operational phase of the project.

Learn More, Read the Complete Announcement

Filed under: Data Files, News, Patrons and Users