Setting
The experimental portion of the study was conducted in 13 KIPP schools located in California,
the District of Columbia, Georgia, Massachusetts, New York, and Texas. The quasi-experimental
portion of the study was conducted in 41 KIPP schools located in Arkansas, California, Colorado, the District of Columbia, Georgia, Indiana, Louisiana, Massachusetts, New York, North Carolina, Oklahoma, Pennsylvania, Tennessee, and Texas.
Study sample
For the experimental portion of the study, each of 13 KIPP schools used a lottery to randomly
assign students either to receive an offer to attend the KIPP school or to not receive an offer.
The schools included in the analyses varied depending on the availability of outcome data,
and a list of schools used in each analysis was provided by the authors after an inquiry by the
WWC. Specifically, the analysis of state assessments in reading and mathematics included
725 students (260 intervention and 465 comparison) entering fifth or sixth grade who applied
to attend 10 of the KIPP schools that used a lottery. The follow-up sample in these schools
included 536 students (202 intervention and 334 comparison) 1 year after random assignment
and 441 students (181 intervention and 260 comparison) 2 years after random assignment.
The experimental portion of the study also included the administration of the TerraNova
reading and mathematics assessment at 10 schools, some of which were different than the 10
schools included in the analysis of state assessments. The sample for this portion of the study
included 1,016 students (431 intervention and 585 comparison) at the beginning of the study
and 590 students (272 intervention and 318 comparison) at the time of the TerraNova assessment,
which was administered in fall of the third follow-up year.
For the quasi-experimental portion of the study, students in 41 KIPP schools that were established
before or during the 2009–10 school year were matched to comparison students who had
never attended a KIPP middle school. KIPP students enrolled in a KIPP school in the fifth or sixth
grade, and they were matched to non-KIPP students enrolled in the same district who had similar
demographic characteristics and prior achievement scores using nearest neighbor propensity
score matching without replacement. Between two and 10 cohorts of students per school were
included in the study, depending on data availability, and outcome data were drawn from the
2001–02 through 2010–11 school years. The study included 31,832 students in the investigation
of reading and math outcomes in year one (half were KIPP students and half were non-KIPP).
Students were matched once and then followed over time and across subjects. Therefore, study
sample sizes in later analysis years and for science and social studies outcomes vary depending
on the availability of outcome data for the originally matched sample.
Intervention Group
The intervention groups for both the experimental and quasi-experimental portions of the study attended KIPP schools, which are designed to engage students and parents in the educational
process, expand the amount of time dedicated to learning, reinforce students’ social competencies and positive behaviors, and improve academic achievement. The KIPP model rests on the “Five Pillars”: (a) high expectations for academic achievement; (b) choice and commitment of students and families to college preparatory education; (c) more time spent learning, both in academic and extracurricular activities; (d) power to lead for school principals, who are given freedom in budgeting, personnel, and other decisions; and (e) focus on results by regularly assessing student learning and driving accountability.
Comparison Group
In the experimental design, 62% of students in the comparison group attended traditional
public schools, 20% attended non-KIPP charter schools, 14% attended KIPP schools, and
4% attended private schools. Students in the quasi-experimental comparison group attended
non-KIPP middle schools in the feeder school district.
Outcome descriptions
Both the experimental and quasi-experimental studies measured state assessments in math
and reading, which are typically administered in spring of the school year. These outcomes
were measured for 4 follow-up years in the quasi-experimental study and for 2 follow-up years
in the experimental study. In addition, science and social studies state exams were included in
the quasi-experimental study. These outcomes were measured by the latest available middle
school score in each jurisdiction, which was typically eighth grade (i.e., 3 to 4 years postenrollment).
The TerraNova reading and mathematics exams were administered in the experimental
design only, in fall of the third follow-up year. For a more detailed description of these
outcome measures, see Appendix B.
Support for implementation
The study did not provide information about implementation support; however, authors noted
that staff at KIPP schools had considerable autonomy in the implementation process to set the
direction of the school.