NCEE 2011-4024May 2011

Study Sample

The process used to recruit 12 districts for the first year of the study was designed to produce a sample that was relevant to federal education programs—which tend to target low-income students—and large enough to provide power to detect impacts of the anticipated magnitude in teacher and student outcomes.

For the second year sample of 6 districts, we wanted to maintain the balance between PD providers. After excluding districts in which we expected the composition of the study schools to change as a result of restructuring initiatives, we selected the 3 districts for each provider with the largest number of schools in the sample, thus maximizing the statistical precision. Districts were selected before the first-year results were known, so findings about the impact of the first year of the PD on teachers and students—overall or in specific districts—did not inform the choice of districts to participate in the second year of the study.

Thirty-nine schools participated in the second year of the study. The second-year impact analysis sample included 92 teachers and 2,132 students, distributed across treatment and control groups as shown in Table ES-2. Among the 92 teachers, 51 (23 in the treatment group and 28 in the control group) had participated in the study since baseline (fall 2007).

All eligible teachers teaching at least one regular seventh-grade mathematics class in spring 2009 were members of the second-year teacher impact sample, and a random sample of all seventh-grade students who were in the teachers' regular seventh-grade mathematics classes in spring 2009 were members of the second-year student impact sample.6, 7 This definition of the teacher and student samples implies that the study is a test of the impact of mandatory PD, as opposed to PD selected by individual teachers.

Table ES-3 provides descriptive information about the characteristics of the sample of 39 schools in the two-year districts compared with the characteristics of schools serving seventh-grade students in the national sample of similar districts from which the original 12-district sample was recruited for the study. On some key characteristics, the study sample schools were statistically different from the larger pool of eligible schools. The study sample schools were less likely to be in the South and more likely to be in the Northeast region and to be in cities rather than in urban fringe communities, towns, or rural areas. On average, they had smaller enrollments than schools in the national sample (753 students vs. 920 students) and smaller teaching staffs (48.5 FTEs vs. 54.9 FTEs). The schools in the two-year districts also were less likely than schools in the national sample to be middle schools (67 percent vs. 95 percent) and more likely to serve a combination of elementary and middle school grades (33 percent vs. 3 percent).

Despite these differences between study schools and all eligible schools, the teachers in study schools were not statistically distinguishable from teachers in a national sample of seventh-grade mathematics teachers in large urban school districts on any of the teacher characteristics presented in Table ES-4.

6 "Eligible teachers" are defined as regular teachers, not short-term substitutes. (Long-term substitutes were included.)
7 At each school, the study focused on seventh-grade teachers who taught regular, middle-track seventh-grade mathematics classes. This focus excluded advanced classes, such as gifted and talented programs and algebra, as well as remedial classes and self-contained special education classes.