Project Activities
This project will examine a sample of the efficacy and effectiveness trials funded by IES and completed by 2020. It will seek to understand and document the direct and indirect impact of each project on practice, fundamental understanding of education, human capital for education research, and research methods. Data sources will include project abstracts from the IES website, study proposals, and reports of those studies, as well as interviews with Principal Investigators. Researchers will use internet searches to identify products (such as papers) and citations to the studies. They will use web scraping to discover mentions of the studies in websites of state education agencies, school districts, regional education labs, and professional organizations. Data extracted from these sources will be systematically coded and reliability of the coding will be assessed.
The research team will document the products of each study (such as scientific papers and reports) as well as indications that they influenced practice either directly or indirectly. Scientific knowledge is not just created from single studies in isolation, but by studies building upon one another. They will build a database showing relations between each study and both prior and subsequent related studies. They will document the ways in which earlier studies influenced later studies through what they learned, though the personnel that they trained, and through the fundamental knowledge that they created. They will also document situations in which a study either created a methodological innovation that was later used in other studies or motivated other methodological research. They will also include a set of case studies designed to illustrate the ways in which particular studies have made contributions to knowledge about education.
Structured Abstract
Data analytic strategy
The data analysis will largely be descriptive. Researchers will identify each study's direct and indirect contributions to practice, to fundamental knowledge and understanding of education, contribution to human capital for education research, and impact on methodology (or lack thereof). They will summarize the data identify the network structure of relations among the studies and the network structure of personnel who worked on the studies.
Key outcomes
Key Outcomes: Researchers will describe:
- the number of efficacy and effectiveness trials ended with statistically significant outcomes, were not completed, or had non-significant results, along with the basic demographics of these trials
- the reasons the trials did not result in statistically significant outcomes, including those that experienced recruitment, methodological, implementation issues, or problems with the theory of change
- results that were used directly to improve practice, including inclusion in the WWC and IES practice guides, direct commercialization, or having influence on other products
- indirect effects of each study, such as contributing to a larger research agenda that expanded fundamental knowledge and understanding of education (e.g., design of future interventions, relationships with stakeholders)
- how each study affected the creation of human capital for education science, and improvements in scientific infrastructure
- how the results of trials led to the improvement of methods for designing interventions or the design, analysis, or interpretation of education research.
People and institutions involved
IES program contact(s)
Project contributors
Additional project information
This project is jointly funded by the National Center for Special Education Research and the National Center for Education Research.
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
