From RAND:
The full title of the report is:
From Summary Webpage
The National Science Foundation (NSF) created the National Science Digital Library/Distributed Learning (NSDL) program in 2000. Between 2000 and 2011, NSDL awarded more than 250 grants to support the development of resources, collections, and technical tools and services for teachers and learners in science, technology, engineering, and math (STEM) education. NSDL’s mission was to assemble these elements into a distributed digital library that would be widely available to the public through websites. In 2011, NSF discontinued grant funding for projects in order to build on the results of the NSDL program and to establish the next generation of cyberlearning programs.
This report documents the third and final phase in a series of evaluations of the NSDL program. The broad goal of the evaluation was to assess the sustainability of NSDL resources, tools, and services over time and across the changing technology landscape. The report also identifies lessons that could be useful to other programs, both within and outside NSF — specifically, the attributes of sustainable digital initiatives and the extent to which the NSDL program demonstrated these attributes.
Key Findings
Audience
NSDL and the pathways have shown considerable improvement in identifying the audience served.
The NSDL portal shows strong audience engagement in terms providing capabilities for users to organize, comment on, contribute, and share resources.
NSDL exceled in development of paradata, which document resource use in the aggregate, and in the collection and harvesting of annotation metadata, which consist of users’ comments and ratings for resources.
Organization
NSDL has a strong technical infrastructure, with a robust platform that is well suited to a metadata-based collection portal.
The NSDL portal and pathways have visionary leadership, but evidence was less clear for staff in operational roles to support a transition from research to sustainable business organizations.
Broad and diverse parent organizations and other stakeholders are a potential asset with respect to revenue for sustainability.
Collections
The NSDL portal and pathways showed improvements in the authoritativeness of their collections, professional oversight of resource accessions, and basic search services.
Accurate and complete metadata, alignment of educational standards to resources in metadata, and the expansion of usage metadata are significant contributions of the NSDL portal and some pathways, but few projects have shown how materials can be tailed to meet specific educational requirements or audiences.
Tools and Services
Funding supported the development of a diverse array of tools and services. Although many NSDL projects created innovative products, there were redundant efforts across some projects, as well as gaps in the provision of needed tools and services.
The degree to which products were integrated with the NSDL portal was variable.
Recommendations
Improve audience identification by enabling users to customize resources to their workflows, providing a compelling rationale for users to select each portal over competing alternatives, providing stronger evidence of accountability to the audience, and broadening opportunities for audience engagement.
Improve organization by allowing for integrated security beyond basic user identification and password protection, providing support for users in terms of training and personalized services, and staffing roles to support the transition from research projects to nonprofit businesses.
Improve collections by publishing a sound copyright strategy, documenting copyright and permitted uses in resource metadata, and enabling searching from the front page of each site.
Improve tools and services by fostering collaboration among projects to avoid redundancy in development and fill gaps in services and ensuring that projects are integrated with the NSDL portal.
Future evaluation efforts of the health and sustainability of digital initiatives should use a broader range of methods and data sources, such as data from clients and customers and findings about impact on learning outcomes.
Cyberlearning initiatives, particularly multiyear, collaborative initiatives, need a much greater focus on sustainability from the outset to balance the emphasis that NSF has traditionally placed on
innovation.Cyberlearning initiatives should use a combination of initial grant funding, followed by contracts or cooperative agreements, to foster a balance between innovation and sustainability.
NSF should incorporate concrete sustainability requirements into solicitations and annual reporting requirements — in particular, to target funding directly to products that have been identified as sustainable and worthy of ongoing development and support.
Direct to Full Text Report
229 pages; PDF.
Direct to Key Findings Web Page
About Gary Price
