Skip Navigation

Evaluation of Investing in Innovation

Contract Information

Current Status:

Technical assistance is underway.

Duration:

September 2010 – December 2021

Cost:

$25,359,202

Contract Number:

ED-IES-10-C-0064
ED-IES-13-C-0005
ED-IES-14-C-0007
ED-IES-15-C-0002

Contractor(s):

Abt Associates

Contact:

Reports

The Investing in Innovation (i3) Fund was established by the American Recovery and Reinvestment Act of 2009 (ARRA) to provide funds to local educational agencies and nonprofits with a record of improving student achievement. The program awarded competitive grants to implement and evaluate educational interventions to improve student academic outcomes.

The i3 program awarded three different tiers of grants. The amount of funding awarded was aligned with the strength of the prior evidence supporting the proposed intervention as well as the proposed implementation scale. The smallest, or "Development," grants supported developing and testing interventions with limited or no prior evidence. Interventions with moderate evidence of effectiveness could receive a larger "Validation" grant, to implement and test the intervention in a broader population or in new contexts. The largest, or "Scale-up," grants supported interventions with strong prior evidence of effectiveness to be implemented and tested on a much larger scale.

The i3 program funded seven cohorts of grantees from 2010 through 2016. The program has awarded more than $1.4 billion to 172 grantees across the three tiers. One of the i3 program's key priorities was evaluating the impact of the funded interventions. Each grantee was required to fund an independent evaluation using local evaluators. To maximize what could be learned from these evaluations, the Department is conducting an Evaluation of i3, which (1) provides comprehensive technical assistance to local evaluators for all grantees, and (2) summarizes both the quality of and findings from the evaluations.

  • Were the i3 evaluations strong?
  • Did the i3 evaluations find the interventions to be implemented with adequate fidelity?
  • Did the i3 evaluations find that the interventions improved student academic outcomes?

The contractor provides comprehensive, customized technical assistance to all i3 independent local evaluators. The technical assistance is facilitated by a one-on-one relationship with a technical assistance provider.

In addition to providing technical assistance, the contractor characterizes whether or not the evaluations meet several criteria used to define a strong study and provides a report for the program office each year about the number of grantees meeting the i3 evaluation-related Government Performance and Results Act measures. The contractor also summarizes the results of the i3 evaluations.

The final report found that:

  • Almost three-quarters of the i3 evaluations were implemented consistent with WWC evidence standards with or without reservations (49 of 67, or 73 percent).
  • Sixty percent of the evaluations met all of the goals i3 set for evaluation quality. In addition to consistency with WWC evidence standards, these goals included: independence, high-quality implementation measurement, and including a sample that adequately represents those served under the grant.
  • Eighteen percent (12 of 67) of the i3 evaluations found a statistically significant positive impact on at least one student academic outcome.
  • Thirteen percent (9 of 67) of the i3 evaluations found evidence of both adequate implementation fidelity and positive impacts on student academic outcomes.

The final report of the project, titled The Investing in Innovation Fund: Summary of 67 Evaluations, was released in June 2018.