Evaluation of the Impact of Charter School Strategies

Contractor: Mathematica Policy Research, Inc., University of Washington

Background/Research Questions:

The Public Charter Schools Program (Title V, Part B, Subpart 1 of ESEA) supports the planning, development, and initial implementation of charter schools in states and communities across the U.S, with funding of $216 million in FY 2009. A key component of both the federal program and the charters that schools sign with authorizers in their states is a promise to reach milestones for students' academic achievement.

There remains ongoing debate about the extent to which charter schools improve students' academic achievement, with much of the evidence based on descriptive or quasi-experimental studies that cannot fully separate the effects of charter enrollment from the characteristics of students who choose to attend those schools. Policy interest has shifted from a sole focus on "are they effective" to the policy levers (e.g., the level of autonomy, types of authorizers) and school characteristics that might make them more effective. This study of charter middle schools examined:

• What are the impacts of charter schools on student achievement, other indicators of performance, and parent and student satisfaction?
• To what extent does the degree of autonomy or policy environment under which charter schools operate seem to influence their effectiveness?
• In what ways are charter schools and the sending regular public schools different? What role do these school factors or characteristics play in determining student outcomes?

Design:

NCEE conducted a large-scale, multi-state randomized controlled trial of charter schools. About 40 charter middle schools in 15 states that had more applicants than space available participated. Among applicants to each charter school, about 30 students in the entry grade were randomly assigned through lotteries to be admitted to the school (treatment group) or to attend another school of their own choosing (the control group). Schools and students were recruited in early 2005 and in early 2006. Two years of data were collected for students on both groups, including student records, annual test scores, and surveys of students, principals, and parents.

Cost/Duration: $6,174,723 over 6.5 years (September 2003–July 2010)

Current Status:

The only (final) report was published in June 2010 (http://ies.ed.gov/ncee/pubs/20104029/index.asp).

Key Findings:

• On average, participating charter middle schools were neither more nor less successful than traditional public schools in improving math or reading test scores, attendance, grade promotion, or student conduct within or outside of school. Being admitted to a study charter school did significantly improve both students' and parents' satisfaction with school.
• Charter middle schools' impact on student achievement varied significantly across schools, with those in urban areas and serving higher proportions of low-income and low achieving students more effective (relative to their nearby traditional public schools) than were other charter schools in improving math test scores.

There was no significant relationship between achievement impacts and the charter schools' policy environment. However, enrollments and the use of ability grouping in math or English classes were associated with less negative impacts on achievement.