REL Midwest Ask A REL Response

Educator Effectiveness

August 2020

Carlson, D., & Lavertu, S. (2018). School improvement grants in Ohio: Effects on student achievement and school administration. Educational Evaluation and Policy Analysis, 40(3), 287–315. Retrieved from https://eric.ed.gov/?id=EJ1186801

From the ERIC abstract: “The federal School Improvement Grant (SIG) program allocated US$7 billion over nearly a decade in an effort to produce rapid and lasting improvements in schools identified as low performing. In this article, we use a regression discontinuity design to estimate the effect of Ohio’s SIG turnaround efforts on student achievement and school administration. The results indicate that Ohio’s SIG program significantly increased reading and math achievement, with effects in both subjects of up to 0.20 standard deviations in the second year after SIG eligibility identification. Estimates for the third year are somewhat larger, in the range of one quarter of a standard deviation. We provide evidence that these effects were primarily attributable to schools that implemented the SIG Turnaround model. We also show that SIG eligibility had a positive effect on per-pupil spending, but no average effect on administrative outcomes, including staff turnover, the number of staff members in the school, and school closure. These null overall effects mask heterogeneity across SIG models, however. Most notably, Turnaround schools experienced more turnover than they otherwise would have, whereas Transformation schools experienced less.”

Note: REL Midwest was unable to locate a link to the full-text version of this resource. Although REL Midwest tries to provide publicly available resources whenever possible, it was determined that this resource may be of interest to you. It may be found through university or public library systems.

Council of the Great City Schools. (2015). School Improvement Grants: Progress report from America’s great city schools. Washington, DC: Author. Retrieved from https://eric.ed.gov/?id=ED559377

From the ERIC abstract: “This report measures trends in performance among urban schools receiving federal School Improvement Grant (SIG) awards as part of the American Recovery and Reinvestment Act of 2009 (ARRA). The Council of the Great City Schools aims to document how member districts of the Council of the Great City Schools implemented SIG and specifically what effects the program had on student test scores and school ‘holding power’—the ability of high schools to move students through the system on a timely basis. Finally, based on interviews with district and school-based staff in several case study districts, common characteristics of successful and unsuccessful implementation of the SIG program in Council schools and districts are identified and described. Results of the analysis across states for grades three through eight in both math and reading indicate that the gaps in the percentages of students scoring at or above Proficient on state assessments between SIG-award schools and the two comparison groups (SIG-eligible schools that did not receive grants and non-SIG-eligible schools) appear to have narrowed steadily over the first two years of the grants, and then leveled off in the third year. The findings suggest that SIG-award schools also reduced the percentage of students in the lowest proficiency levels on state assessments. In many respects, this measure could be considered the most relevant assessment of the impact of the SIG investment, as more than one out of every three students in SIG-award schools were classified in the lowest performance level on state assessments.”

Dee, T. S. (2012). School turnarounds: Evidence from the 2009 stimulus. (PEPG 12-04). Cambridge, MA: Program on Education Policy and Governance, Kennedy School of Government, Harvard University. Retrieved from https://eric.ed.gov/?id=ED534653

From the ERIC abstract: “The American Recovery and Reinvestment Act of 2009 (ARRA) targeted substantial School Improvement Grants (SIGs) to the nation’s ‘persistently lowest achieving’ public schools (i.e., up to $2 million per school annually over 3 years) but required schools accepting these awards to implement a federally prescribed school-reform model. Schools that met the ‘lowest-achieving’ and ‘lack of progress’ thresholds within their state had prioritized eligibility for these SIG-funded interventions. Using data from California, this study leverages these two discontinuous eligibility rules to identify the effects of SIG-funded whole-school reforms. The results based on these ‘fuzzy’ regression-discontinuity designs indicate that there were significant improvements in the test-based performance of schools on the ‘lowest-achieving’ margin but not among schools on the ‘lack of progress’ margin. Complementary panel-based estimates suggest that these improvements were largely concentrated among schools adopting the federal ‘turnaround’ model, which compels more dramatic staff turnover.”

Dragoset, L., Thomas, J., Herrmann, M., Deke, J., James-Burdumy, S., Graczewski, C., et al. (2017). School Improvement Grants: Implementation and effectiveness (NCEE 2017-4013). Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance. Retrieved from https://eric.ed.gov/?id=ED572215

From the ERIC abstract: “In response to the recession that began in 2007, the U.S. Congress passed, and President Barack Obama signed into law, the American Recovery and Reinvestment Act of 2009 (Pub. Law 111-5). At an estimated cost of $831 billion, this economic stimulus package sought to save and create jobs, provide temporary relief to those adversely affected by the recession, and invest in education, health, infrastructure, and renewable energy. States and school districts received $100 billion to secure teachers’ jobs and promote innovation in schools. This funding included $3 billion for School Improvement Grants (SIG), one of the Obama administration’s signature programs and one of the largest federal government investments in an education grant program. The SIG program awarded grants to states that agreed to implement one of four school intervention models—transformation, turnaround, restart, or closure-in their lowest-performing schools. Each of the models prescribed specific practices designed to improve student outcomes, including outcomes for high-need students such as English language learners (ELLs) (U.S. Department of Education 2010a). Given the importance of the SIG program and sizable investment in it, the Institute of Education Sciences (IES) commissioned this evaluation to focus on four primary questions: (1) Did schools implementing a SIG-funded model use the improvement practices promoted by SIG, and how did that compare to use of those practices by schools not implementing a SIG-funded model?; (2) Did use of SIG-promoted practices include a focus on ELLs, and did that focus on ELLs differ between schools implementing a SIG-funded model and schools not implementing one?; (3) Did receipt of SIG funding to implement a school intervention model have an impact on outcomes for low-performing schools?; and (4) Was the type of school intervention model implemented related to improvement in outcomes for low-performing schools? This is the final report for this evaluation of SIG. This final report builds on the earlier briefs and report by including an additional year of data (spring 2013) and by examining whether receipt of SIG funding had an impact on student outcomes. The findings in this report suggest that the SIG program did not have an impact on the use of practices promoted by the program or on student outcomes (including math or reading test scores, high school graduation, or college enrollment), at least for schools near the SIG eligibility cutoff.”

Dragoset, L., Thomas, J., Herrmann, M., Deke, J., James-Burdumy, S., & Lee Luca, D. (2019). The impact of School Improvement Grants on student outcomes: Findings from a national evaluation using a regression discontinuity design. Journal of Research on Educational Effectiveness, 12(2), 215–250. Retrieved from https://eric.ed.gov/?id=EJ1215569

From the ERIC abstract: “As part of the American Recovery and Reinvestment Act of 2009, the School Improvement Grants (SIG) program received more than $3 billion with the goal of substantially improving student achievement. The SIG program’s funding was to help states turn around the nation’s persistently lowest-achieving schools using one of four school intervention models—transformation, turnaround, restart, or closure. We used a regression discontinuity design and a large sample of schools from more than 20 states to evaluate the impact of implementing a SIG-funded intervention model on use of SIG-promoted practices and student outcomes. Our regression discontinuity design exploited cutoff values on the continuous variables used to define the SIG eligibility tiers to compare outcomes in schools that just met the eligibility cutoff for receiving SIG funding to outcomes in schools that just missed it. We found that SIG had no impact on any of the outcomes we examined, including math and reading test scores, high school graduation rates, and college enrollment rates. Using a correlational analysis, we found the turnaround model was associated with larger student achievement gains in math than the transformation model for grades 6 through 12.”

Note: REL Midwest was unable to locate a link to the full-text version of this resource. Although REL Midwest tries to provide publicly available resources whenever possible, it was determined that this resource may be of interest to you. It may be found through university or public library systems.

Le Floch, K. C., O’Day, J., Birman, B., Hurlburt, S., Nayfack, M., Halloran, C., et al. (2016). Case studies of schools receiving School Improvement Grants. Final report. (NCEE 2016-4002). Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance. Retrieved from https://eric.ed.gov/?id=ED565615

From the ERIC abstract: “The Study of School Turnaround (SST) examines the change process in a diverse, purposive sample of schools receiving federal School Improvement Grants (SIG) from 2010-11 to 2012-13. With the passage of the American Recovery and Reinvestment Act of 2009 (ARRA), the SIG program underwent three major shifts. First, ARRA boosted total SIG funding in fiscal year 2009 to approximately 6.5 times the original 2009 appropriation through Title I, section 1003(g) of the Elementary and Secondary Education Act (ESEA). SIG funds were distributed to states by formula based on each state’s Title I share. States then had to competitively make SIG awards to districts with eligible schools. Second, ARRA targeted funds at only the very worst schools—those that were in the bottom 5 percent of performance and had been low performing for an extended period of time. Third, schools receiving SIG were now required to implement one of four prescriptive intervention models believed to be more aggressive and comprehensive than those generally adopted under prior policies. By increasing the level of funding, better targeting these funds to the persistently lowest-achieving schools, and requiring that schools adopt specific intervention models, the revamped SIG program aimed to catalyze more aggressive efforts to turn around student performance. This report focuses on a small sample of schools receiving SIG over the first three years of the revamped SIG program, from 2010-11 to 2012-13. It presents findings from the study’s 25 core sample schools, which were the focus of data collection in spring 2011 and spring 2012, and a subsample of 12 of the 25 schools (collectively referred to as the core subsample), which were selected for data collection in spring 2013 and are the focus of more in-depth analyses looking across all three years of SIG.”

Perlman, C. L., & Redding, S. (Eds.). (2011). Handbook on effective implementation of School Improvement Grants. Lincoln, IL: Academic Development Institute. Retrieved from https://eric.ed.gov/?id=ED565866

From the ERIC abstract: “The purpose of this ‘Handbook’ is to bolster the effective implementation of the intervention models and strategies outlined in the ‘2010 School Improvement Grant’ (SIG) program—section 1003(g) of the Elementary and Secondary Education Act (ESEA)—in order to achieve the program’s clear goal—rapid improvement of persistently low-achieving schools. The ‘Handbook’ offers succinct and practical explanations of the SIG’s required and recommended models and strategies, references to the underlying research, and connections to useful resources. The intended audience for this ‘Handbook’ includes: (1) state education agencies (SEAs); (2) local education agencies (LEAs); (3) charter management organizations (CMOs); (4) education management organizations (EMOs); (5) organizational partners engaged in school improvement; and (6) schools engaged in rapid improvement. The ‘Handbook’ is organized into two parts. Part I frames the purposes of the School Improvement Grants, to classify schools within performance strata and identify the ‘persistently low-achieving’ schools, and offers a framework for diagnosing a school’s performance and practice in order to target interventions and supports for rapid improvement. Part II itemizes more than 50 strategies relevant to the School Improvement Grants, connects the strategies with research, cites available resources, and offers action principles for the SEA, the LEA, and the school.”

Rosenberg, L., Christianson, M. D., & Hague Angus, M. (2015). Improvement efforts in rural schools: Experiences of nine schools receiving School Improvement Grants. Peabody Journal of Education, 90(2), 194–210. Retrieved from https://eric.ed.gov/?id=EJ1059583

From the ERIC abstract: “Low-performing schools in rural settings can face challenges common to all struggling schools, such as low student motivation and maintaining a qualified teaching staff. However, aspects of rural schools’ settings, such as the distance from urban areas and the commute between the schools and the students’ and teachers’ homes, can exacerbate the challenges that rural schools face. This article focuses on the experiences of nine rural schools that were part of a study for the U.S. Department of Education on the school improvement process in a purposive sample of 35 schools receiving federal funds through the School Improvement Grants (SIG) program. In particular, it examines how respondents in the rural schools perceived that the schools’ rural setting influenced the schools’ challenges and improvement actions and presents findings on the recruitment and retention of teaching staff, an activity integral to school improvement efforts under SIG.”

Note: REL Midwest was unable to locate a link to the full-text version of this resource. Although REL Midwest tries to provide publicly available resources whenever possible, it was determined that this resource may be of interest to you. It may be found through university or public library systems.

Stein, L., LiCalsi, C., Kistner, A., Garcia-Piriz, D., & Melchior, K. (2017, April). Measuring impact and understanding implementation of School Improvement Grants in Massachusetts. Paper presented at the Annual Meeting of the American Educational Research Association, San Antonio, TX. Retrieved from https://eric.ed.gov/?id=ED593310

From the ERIC abstract: “Despite largely mixed reports about the impact of School Improvement Grants (SIGs) on school improvement nationwide, Massachusetts has experienced proven success with these grants, termed School Redesign Grants (SRGs) in Massachusetts, as evidenced by comparative interrupted time series analyses, which show that SRGs have a significant impact on student achievement in both English language arts and mathematics one, two, and three years after grant receipt. This study builds upon previous impact studies of SRGs in Massachusetts, by incorporating additional schools and additional cohorts, and utilizes qualitative analyses to identify specific turnaround strategies or activities that distinguish schools able to improve student outcomes from schools struggling to do so, despite receiving the same grant and being afforded the same autonomies.”

Sun, M., Penner, E. K., & Loeb, S. (2017). Resource- and approach-driven multidimensional change: Three-year effects of school improvement grants. American Educational Research Journal, 54(4), 607–643. Retrieved from: https://eric.ed.gov/?id=ED586405

From the ERIC abstract: “Hoping to spur dramatic school turnaround, the federal government channeled resources to the country’s lowest-performing schools through School Improvement Grants (SIG). However, prior research on SIG effectiveness is limited and focuses primarily on student achievement. This study uses a difference-in-differences strategy to estimate program impacts on multiple dimensions across the 3-year duration of the SIG award in one urban school district. Following 2 years of modest improvement, we find pronounced, positive effects of SIG interventions on student achievement in Year 3, consistent with prior literature indicating that improvements from comprehensive school turnarounds emerge gradually. We also identify improvements indicating the process through which change occurred, including reduced unexcused absences, increased family preference for SIG schools, improved retention of effective teachers, and greater development of teacher professional capacity.”

U.S. Department of Education, Office of State Support. (2015). School Improvement Grants national summary: School Year 2012-13. Washington, DC: Author. Retrieved from https://www2.ed.gov/programs/sif/index.html

From the abstract: “School Improvement Grants National Summary: School Year 2012-13 analyzes achievement, graduation rates, and leading indicator data from the 2012-13 school year for schools that began implementation of a SIG model in the 2010–11, 2011–12, and 2012–13 school years.”

Wei, T., & Johnson, E. (2020). How states and districts support evidence use in school improvement. Study snapshot. (NCEE 2020-004). Washington, DC: U.S. Department of Education, National Center for Education Evaluation and Regional Assistance. Retrieved from https://eric.ed.gov/?id=ED605885

From the ERIC abstract: “‘The Every Student Succeeds Act’ encourages educators to use school improvement strategies backed by rigorous research. This snapshot, based on national surveys administered in 2018, describes what guidance states provided on improvement strategies and how districts selected such strategies in lowest-performing schools. Most states pointed districts and schools to evidence on improvement strategies, but few required schools to choose from a list of approved strategies. In turn, most districts reported that evidence of effectiveness was ‘very important’ when choosing improvement strategies, but the evidence districts relied on probably varies in quality.”

Yatsko, S., Lake, R., Bowen, M., & Cooley Nelson, E. (2015). Federal School Improvement Grants (SIGs): How capacity and local conditions matter. Peabody Journal of Education, 90(1), 27–52. Retrieved from https://eric.ed.gov/?id=EJ1050835

From the ERIC abstract: “In 2009, the federal government committed over $3 billion nationwide to help states and districts turn around their worst-performing schools. The U.S. Department of Education intended for the School Improvement Grants (SIGs) to spur dramatic change. This report looks at the results of a field study of the first-year implementation of those grants in Washington State, which received $50 million in SIG funding over three years. Researchers hoped to see what school-level changes were underway, how they compared to the intent of the grants, and the early role that districts played in SIG implementation. The report provides findings from the state, district, and school level. Researchers found that, with some exceptions, districts and schools in Washington State are approaching the turnaround work in ways only marginally different from past school improvement efforts. Despite the hard work of administrators, principals, and especially teachers, the majority of schools studied show little evidence of the type of bold and transformative changes the SIGs were intended to produce. The report offers recommendations regarding the roles that federal, state, and local education agencies should play in support of school turnaround work. Those administering future grants targeted at the nation’s lowest-performing schools could avoid the problems described here and improve their chances of affecting dramatic, not incremental, change.”

Note: REL Midwest was unable to locate a link to the full-text version of this resource. Although REL Midwest tries to provide publicly available resources whenever possible, it was determined that this resource may be of interest to you. It may be found through university or public library systems.