Do Teachers Matter?
The answer may seem so obvious that the question isn't worth asking. One reason is that all of us can generate anecdotes about teachers who have made a difference in our lives. I remember my 11th grade English teacher whose interest in my writing and the books I was reading inspired me to think about careers involving words. But however powerful such personal narratives may seem, we need to remember that in science the plural of anecdote is not evidence. Most undergraduates believe in extrasensory perception and will tell stories about experiencing it. That doesn't mean that extrasensory perception is a fact.
The Coleman study
Contrary to our intuitions and anecdotes about the importance of teachers, the landmark 1966 study, Equality of Educational Opportunity, by sociologist James Coleman, suggested that differences in teachers did not matter much. This was a huge study employing 60,000 teachers in grade 6 and beyond in over 3,000 schools. The principal finding was that nearly all of the variability in how students achieved was attributable to their socioeconomic background rather than to the schools they attended. On the subject of teacher attributes, Coleman wrote, "A list of variables concerning such matters as teachers' scores on a vocabulary test, their own level of education, their years of experience, showed little relation to achievement of white students, but some for Negroes.... Even so, none of these effects was large."
Coleman's methodology is now understood to have been seriously flawed. All of his analyses were conducted on data that had been aggregated to the school level. For example, the average vocabulary score for all teachers in a school was related to the average test score for all children in a school. Researchers now understand that aggregating data in this way can distort findings. I am reminded of the man who had his head in the oven and his feet in the freezer but whose temperature, on average, was just right. If you average together the effective teachers with the ineffective teachers, and the high performing students with the low performing students, you don't get to see the cold and hot spots where teacher characteristics might make a difference.
Recent multi-level studies
More recent studies in the tradition of Coleman's work have analyzed multilevel data that goes down to individual classrooms and students. Statistical techniques are used to apportion differences in children's academic achievement among the different environments that are assumed to affect their learning and development. Such studies typically parse out the influence of the individual abilities and knowledge the child brings to the classroom, the classroom itself, and the characteristics of the school in which that classroom is housed. With enough children and teachers and schools, and with some fancy statistics, it is possible to estimate the relative contribution of each of these factors to the differences that are observed among children in academic achievement. These studies generate much higher estimates of the relative influence of teachers and schooling on academic achievement than reported by Coleman.
The pie chart that follows reflects findings from a recent scholarly review of this literature (Scheerens & Bosker, 1997). Roughly 20% of the differences in student achievement is associated with the schools children attend, another 20% is associated with individual classrooms and teachers, and the remaining 60% is associated with differences among the children in each classroom, including the effects of their prior achievement and their socioeconomic background.
Note two things about these multilevel studies. First, they are only able to indicate the relative contribution of teachers to academic achievement, not the mechanisms by which teachers affect student learning. Thus, we find that teachers are important, by not why. Second, because the data are collected at a single point in time, the influence of teachers may be substantially underestimated. This is because the 60% effect attributable to students in the pie chart includes the effects of instruction in previous grades. Some children in a given class will have had an effective teacher the previous year and some will have had an ineffective teacher. But we can't see these influences if the children are measured only at one point in time. These unmeasured effects of previous teachers get folded into the unexplained differences among children in the same classroom. This increases the estimated influence of children compared to teachers and schools.
Value-added studies
Value-added methods are a new and more powerful way of addressing the question of whether teachers matter. Value-added methods examine students' gains from year to year rather than their scores at a single point in time. Teachers who are adding value to student achievement will be those whose students gain most over the school year. Thus if a math teacher has children who start the year at the 95th percentile and end the year at the 90th percentile, she would not be considered an exemplary teacher even if the performance of her students was the highest in the district. In contrast, a teacher who raised her students' performance from the 45th to the 60th percentile over the course of a year would be deemed very effective even if her children performed below the average in the district. Value-added methods require that children be followed longitudinally, i.e., the same children must be tested each year and identified uniquely in the resulting database.
Sanders and Rivers (1996) used value-added methods to examine the cumulative effects of teacher quality on academic achievement. The effectiveness of all math teachers in grades 3, 4, & 5 in two large metropolitan school districts in Tennessee was estimated by determining the average amount of annual growth of the students in their classrooms. These data were used to identify the most effective (top 20%) and the least effective (bottom 20%) teachers. The progress of children assigned to these low and high performing teachers was tracked over a three-year period. The next figure illustrates the results.
Children assigned to three effective teachers in a row scored at the 83rd percentile in math at the end of 5th grade, while children assigned to three ineffective teachers in a row scored at the 29th percentile.
The next figure illustrates results from an equivalent study on math performance in Dallas (Jordan, Mendro, & Weerasinghe, 1997). The results are very similar.
Understand that these studies overestimate the actual effect of teachers on academic achievement because the assignment of students to teachers from year to year is essentially random, at least in elementary school (Rowan, 2002). The typical child is not lucky enough to get 3 highly effective teachers or unlucky enough to get 3 highly ineffective teachers in a row. However, these studies demonstrate persuasively that the potential effect of teacher quality on academic achievement is quite high.
In summary, we now know that Coleman was wrong: Teachers do matter, as our anecdotal experiences suggest and as Congress assumed when it reauthorized ESEA and authorized $3 billion annually for teacher training and professional development. Whew!