March 7, 2018 — The National Science Foundation’s (NSF) Directorate for Education and Human Resources (EHR) has implemented new guidance for the scientists it supports, providing detailed directions on how to make their research results easier to replicate.
The new data management plan guidance, which took effect earlier this year, requires researchers seeking EHR funding to describe in their proposals how they plan to generate data from studies and related materials. The updated guidance aims to make validating research findings and teasing out new ones easier for the research community.
“This new guidance promotes scientific transparency and clarifies expectations for EHR-funded research,” said Evan Heit, director for EHR’s Division of Research on Learning in Formal and Informal Settings (DRL). “The goal is that researchers should be able to show their work so that others can check it, perhaps improve on it and take the project further.”
Researchers should indicate how they plan to share data, products and methods so that other investigators can understand, validate and replicate research findings.
Reproducing education studies has taken on added significance during the last few years as government agencies implement federally mandated public access policies.
In 2013, the White House Office of Science and Technology Policy called for executive departments and agencies that spend more than $100 million annually on research and development to make their research results available to the public, industry and the scientific community. NSF responded with a series of data management policies tied to the scientific disciplines it funds.
For EHR, public access means researchers can learn more from previous findings, and study how those findings apply to real-world situations.
For example, if a study identified an innovative method to enhance STEM education for one student population, other researchers could precisely repeat it with different populations elsewhere in the country, or in different socioeconomic conditions, to see if they achieve the same result.
“A commitment to data-sharing, replication and reproducibility at the publication stage can foster more rigorous peer-reviewed research that forms the foundation for high quality science,” Jim Lewis, NSF acting assistant director for EHR, said during an address to the American Educational Research Association workshop.
Certain EHR programs have led the way for more replicability in research studies. The Improving Undergraduate STEM Education: Education and Human Resources (IUSE: EHR) program was one of the first to specifically call for replicable studies of successful educational interventions. Previously, IUSE: EHR encouraged small-scale design projects that adapted and implemented strategies shown to be effective at other institutions. In an effort to support transparency and broad applicability, IUSE: EHR now requires that any new learning materials and computer software code should be sharable under an open license, allowing others to use and build upon that work.
“This is the first time we have been so definite about the need for these kinds of proposals, but it’s not the first time this call to the field has appeared,” said Myles Boylan, IUSE: EHR lead program said of the new guidance.
EHR’s guidance on data management also will be shared with proposal reviewers during NSF’s merit review process to help promote scientific transparency and the potential for replication.