Summary of Findings After Two Years: Treatment-Control Differences in One-Year Districts

Induction Services Received

Within one-year districts, during Year 1—the year in which comprehensive teacher induction was implemented—we found statistically significant differences between the treatment and control group; the treatment group reported receiving more induction support than the control group across a broad range of measures of the amount, types, and content of supports.

In Year 2—the year in which treatment teachers no longer received comprehensive teacher induction supports—the percentage of teachers with an assigned mentor and the weekly minutes spent with that mentor declined from Year 1 to Year 2 (differences with a pvalue of 0.000) for both the treatment and control groups. During this second year, we found statistically significant negative impacts on these and other measures of support, as described below.

Because teachers in one-year districts were not surveyed in the spring of Year 2, we focus the discussion on findings for the fall of each year.⁶ Estimates were computed using an ordinary least squares model with district and grade assignment fixed effects that accounted for clustering of teachers within schools; weights were applied to adjust for survey nonresponse and the study design.⁷

Amount of Mentoring. In Year 1, we found statistically significant differences in the likelihood of teachers reporting having a mentor assigned to them and having a full-time mentor. As part of the intervention, every treatment teacher was assigned a mentor by ETS or NTC, but that did not guarantee that all teachers would work with their mentor or acknowledge having had one assigned to them. Still, treatment teachers were more likely than control teachers to report having a mentor assigned to them (90 versus 70 percent) and to report having a full-time mentor (74 versus 8 percent). We found statistically significant differences in teachers' likelihood of having a mentor who was another teacher and in the amount of time teachers reported spending with a mentor during the most recent full week of teaching. Treatment teachers were less likely than control teachers to report having a mentor who was another teacher (25 versus 64 percent). In addition, treatment teachers reported spending an average of 87 minutes per week in mentor meetings compared to 67 minutes for control teachers, with the 20-minute difference attributable entirely to differences in the duration of scheduled meetings, as opposed to informal meetings.

In Year 2, we found statistically significant differences in the prevalence of and time spent in mentoring. Treatment teachers were less likely than control teachers to report having a mentor assigned to them (20 versus 29 percent). Treatment teachers were also less likely than control teachers to report having a mentor who was another teacher (21 versus 31 percent). Treatment teachers spent less time in mentor meetings than control teachers (19 versus 39 minutes per week). Figure ES.1 shows treatment-control differences for having an assigned mentor and time in mentor meetings in Year 1 and Year 2.

Mentor Activities and Assistance. In Year 1, treatment and control teachers' reports showed statistically significant differences in the amounts of time in various mentor activities and the kinds of assistance received from their mentors. Treatment teachers reported spending more time during the most recent full week of teaching being observed by mentors (34 versus 10 minutes), meeting one-on-one with mentors (34 versus 23 minutes), meeting with mentors together with other first-year teachers (29 versus 9 minutes), and having mentors model lessons (9 versus 6 minutes). During the most recent full week of teaching, treatment teachers were 14 to 27 percentage points more likely than control teachers to report having received mentors' assistance in a variety of topic areas, such as receiving suggestions to improve practice (77 versus 53 percent) and discussing instructional goals (73 versus 48 percent).

By Year 2, we found statistically significant differences in the amount of time teachers reported being observed by mentors during the most recent full week of teaching in fall 2006. Treatment teachers reported less time in a list of six common mentoring activities (22 versus 36 minutes per week) including less time being observed by mentors than control teachers (2 versus 6 minutes). No statistically significant differences were found between treatment and control group teachers on their reported time spent in any of the other five activities covered by the survey. During the most recent full week of teaching in fall 2006, treatment and control teachers' reports showed statistically significant differences in the likelihood of receiving mentors' assistance in each of the topic areas covered by the survey. Treatment teachers were less likely than control teachers to report receiving mentors' assistance in each topic area, with effects ranging from 8 to 14 percentage points, including, for example, impacts on receiving suggestions to improve practice (15 versus 27 percent) and discussing instructional goals (14 versus 24 percent).

Professional Development. We did not find statistically significant differences between treatment and control teachers in their reported attendance in professional development, except in certain areas. Of the 12 professional development topics covered by the survey, treatment teachers were less likely than control teachers to report having attended professional development sessions in two areas in fall 2005 (Year 1): content area knowledge (61 versus 72 percent) and preparing students for standardized testing (30 versus 41 percent). We did not find statistically significant differences between treatment and control teachers in their reported attendance in any of the 12 professional development activities in fall 2006 (Year 2).

Student Achievement

In Year 2 (school year 2006-2007), we found no statistically significant impacts on reading or math scores in the one-year districts. We compared the test scores for students of treatment teachers to those of control teachers using post-test scores measured in 2007 adjusted for pre-test scores measured in 2006. The test score analysis was based on standardized achievement tests that the district normally conducts.⁸ Though districtadministered test scores do not cover every domain of student achievement that induction might affect, they do capture the content that school districts or states deem most important and worthy of assessing. We aggregated test scores across districts and grades by standardizing each test to a common metric called a z-score, which has a mean of zero and a standard deviation of one. We kept two broad subject areas, math and reading, distinct. The benchmark model accounts for the nesting of students within schools, using the normalized student pretest score and district-by-grade fixed effects as covariates.

The benchmark impacts on math and reading scores in Year 2 were not significantly different from zero (see Table ES.1). We confirmed that the impact on math and reading in the second year was not statistically significant when the impacts were re-estimated using different samples, sets of covariates, or estimation techniques.

Teacher Retention

We found that comprehensive teacher induction had no statistically significant impact on teacher retention after two years. We measured teacher retention in terms of the percentage of teachers who remained in their originally assigned school, their district, and the teaching profession. Table ES.2 shows the result of the three hypothesis tests specifically focused on retention in the school, in the district, and in the profession as binary outcomes. For each of the outcomes, there was no statistically significant impact. The same result was obtained when we expanded the number of outcomes to differentiate between moving to a school in another public school district and moving to a private, parochial, or other school, and expanded the outcomes for leaving to include leaving to stay at home, leaving to attend school or take a new job, and other reasons for leaving.

We also examined the reasons that teachers who left their districts (movers) or left the teaching profession (leavers) gave for leaving and found no statistically significant impacts of treatment. When we asked leavers whether they expected to return and if so, when they would do so, we did not find evidence of a treatment-control difference. In addition, we found that treatment teachers did not report feeling more satisfied with their jobs than control teachers.

Composition of District Teaching Force

The last major research question concerned the impact of comprehensive teacher induction on the composition of the teaching workforce in the district. As shown below, we found no statistically significant impacts on the composition of the district teaching force in one-year districts after two years.

For comprehensive teacher induction to affect the composition of the district's teaching workforce, it has to produce a difference in the types of teachers who decide to remain in the district. As teachers leave the district, the average qualifications of the teachers who remain in the district begin to change, perhaps differentially between the treatment and control groups. We tested this hypothesis by comparing the characteristics of district stayers between the treatment and control groups along two dimensions: (1) their impact on student achievement; and (2) their professional characteristics such as SAT/ACT scores and advanced degrees. The student achievement outcome is regression-adjusted using the same model used in the main analysis.

We found that the treatment had no statistically significant impacts on the student achievement or professional background characteristics of district stayers. Table ES.3 presents the impacts on student achievement outcomes for district stayers. Table ES.4 shows the background characteristics of teachers by mobility status.

Top