Evaluation of Investing in Innovation
Contract Information
Current Status:
Technical assistance is underway.
Duration:
September 2010 – June 2022
Cost:
$25,359,202
Contract Number:
ED-IES-10-C-0064
ED-IES-13-C-0005
ED-IES-14-C-0007
ED-IES-15-C-0002
Contractor(s):
Abt Associates
Contact:
Reports
Expanding both the use and evidence base of effective educational strategies is critical to improving student learning and closing equity gaps nationwide. The Investing in Innovation (i3) Fund is one key U.S. Department of Education program explicitly focused on these goals and the first to require adherence to the Department's standards for high quality, rigorous evaluation. Between 2010 and 2016, the i3 program invested $1.4 billion to help districts, nonprofit organizations, and schools implement educational strategies with strong prior evidence of effectiveness and develop and test innovative approaches to improve student outcomes. Importantly, all 172 i3 grantees were required to fund independent evaluations to measure the impact of their educational approach. To maximize what could be learned from these separate evaluations, the Department provided comprehensive technical assistance to each local evaluator and summarized both the quality of and findings from the evaluations. Lessons from these activities have informed similar efforts under the i3 program's successor, the Education and Innovation Research (EIR) Program.
- Did the i3 evaluations meet the Department's standards for high quality evidence of effectiveness (those developed by the What Works Clearinghouse (WWC))?
- Did the evaluations find that i3 grantees implemented their educational approaches as intended (i.e., with adequate fidelity)?
- Did the i3 evaluations find that the approaches improved student academic outcomes?
The contractor is providing comprehensive, customized technical assistance to the local evaluators of each of the i3 grants through the end of the grant program in 2021.
A separate and independent team assesses the quality of the evidence produced by the i3 evaluations, using the WWC evidence standards.
A report summarizing the first 67 i3 evaluations to be completed, titled The Investing in Innovation Fund: Summary of 67 Evaluations, was released in June 2018.
Data files containing publicly-available information on the impact and implementation findings from the i3 grants whose evaluations are included in the 2018 report are available for the purposes of secondary analysis.
Updates to the summary tables and exhibits in the June 2018 report incorporating the remaining i3 evaluations are expected in 2022.
The final report found that:
- Almost three-quarters of the i3 evaluations were implemented consistent with WWC evidence standards with or without reservations (49 of 67, or 73 percent).
- Sixty percent of the evaluations met all of the goals i3 set for evaluation quality. In addition to consistency with WWC evidence standards, these goals included: independence, high-quality implementation measurement, and including a sample that adequately represents those served under the grant.
- Eighteen percent (12 of 67) of the i3 evaluations found a statistically significant positive impact on at least one student academic outcome.
- Thirteen percent (9 of 67) of the i3 evaluations found evidence of both adequate implementation fidelity and positive impacts on student academic outcomes.