Skip Navigation
Impacts of Comprehensive Teacher Induction:

NCEE 2009-4072
August 2009

Methods and Data

We used a model-based approach to estimate program impacts. The statistical model explicitly acknowledges the hierarchical structure of the data—for example, the nesting of teachers within schools—an approach that is sometimes referred to as a hierarchical linear model (HLM). Accordingly, we can properly specify the units of analysis (teachers and schools) and devise unbiased estimates of the standard errors that we used to conduct hypothesis tests. The model also allows us to control for the effects of a range of teacher and school characteristics on the outcomes of interest to increase the precision of the estimates of treatment effects.

For each outcome, we use a different set of control variables (covariates), described in the discussion of key study findings. The control variables used in the body of the report are called the benchmark control variables; in sensitivity analyses presented in appendices to the report, we alter the control variables to test the robustness of the results. These sensitivity tests included re-estimation of the study's main impacts with different sets of covariates, using different samples or sample weights, and different statistical model assumptions.

Data for the study were collected from a variety of sources. In fall 2005 we surveyed mentors participating in the comprehensive induction programs on their background characteristics and reviewed program documents from ETS and NTC. We administered a baseline survey of beginning teachers in fall 2005, at which time we also requested teachers' permission to obtain their college entrance examination scores (SAT or ACT). The baseline survey asked teachers about their formal education, professional training, current teaching assignment, and personal background. We surveyed teachers twice during the 2005-2006 school year on the induction activities in which they participated, including questions about duration and intensity of mentoring and professional development as well as questions about satisfaction with different aspects of their current teaching position. During the 2006-2007 school year, we surveyed teachers in the two-year districts twice and teachers in the one-year districts once on the induction activities in which they participated and on their job satisfaction.

For the report's core outcomes measuring the impacts of comprehensive teacher induction, we collected districts' student records data at the end of the 2006-2007 school year and conducted the second of three mobility surveys in fall 2007 to learn about teacher retention. We measured student achievement outcomes using district-administered test score data from the spring 2007 (posttest) for students taught by study teachers in the 2006-2007 school year and students' linked scores from the prior grade in spring 2006 (pretest).4 We conducted all treatment-control comparisons within grade and within district to ensure that treatment status was not confounded with properties of the test. Response rates on teacher surveys ranged from 88 percent to 97 percent for the treatment group and 78 percent to 92 percent for the control group. We used nonresponse adjustment weights and sensitivity analyses to address the differential response rates in the analysis of teacher mobility.

Top

4 For three districts that tested at least some students in the fall, we used a fall 2006 test as a pretest and/or a fall 2007 test as a posttest.