CALDER, the National Center for Analysis of Longitudinal Data in Education Research, is a federally funded national research and development center supported by a $10 million grant from the Institute for Education Sciences. The Center has brought together a consortium of researchers who are using state and district administrative data on individual teachers and students for insights into how state and local policies—especially teacher policies, governance policies, and accountability policies—affect teachers and students. Comprehensive databases in Florida, Missouri, New York, North Carolina, Texas, and Washington State represented the initial core of the research focus.
Jane Hannaway, director of CALDER, recently took some time to answer some questions posed by NCER Acting Commissioner Elizabeth Albro about the research and accomplishments of the CALDER project.
What have the biggest accomplishments or findings been that have resulted from this award?
Making our research broadly accessible to policy leaders, researchers and the media and ensuring that it contributes to national education policy discussion and debates, rather than solely an academic exercise, is of great import to CALDER.
CALDER's most significant findings relate to teacher effectiveness, as measured by student test performance. Our research was among the first to show that teacher effects on student performance are educationally significant and that differences among teachers in effectiveness are significant. Using sophisticated analytic methods, CALDER research shows that teacher effectiveness is not strongly related to teacher characteristics such as licensure status, degree and experience—the very factors that typically determine employment eligibility and salaries.
CALDER's work provides the research underpinnings for major federal initiatives—such as the Race to the Top and the Teacher Incentive Fund—that emphasize the effectiveness of individual teachers. Variation in effectiveness within school, regardless of student characteristics, is at least as large as variation across schools. Highly effective teachers are not neatly sorted across schools serving advantaged and disadvantaged students. Regardless of their effectiveness, however, teachers are more likely to leave schools serving disadvantaged populations, resulting in higher rates of teacher turnover in high poverty schools.
But what makes our findings particularly significant from a national policy perspective is that they use data from multiple states to explore research questions. CALDER is structured around multi-state teams composed of seasoned, mainly university-based, researchers who work in a coordinated, but semi-independent, way with data from different states. This structure provides both depth and breadth advantages. It allows us to become expert in the policy and data details of individual states and also gives us the ability to replicate studies across states to assess the external validity of the findings. Our basic findings have been exceedingly robust, even though state policy environments, tests, and student populations differ. When replicated, the findings highlight issues for national policy attention.
What are some of the challenges that you and the CALDER team have experienced?
CALDER faces three ongoing challenges. The first is producing and releasing reliable research in a timely way so that it can influence policy. Because our researchers are up and running with the data, they can move in a fast, informed and skilled way. And because CALDER research partners deal with similar research questions and use similar analytic strategies, they are able to provide each other quick-turnaround expert reviews before research is released.
A third challenge is research independence. To insure the usefulness of our work, we work closely with states in developing our research plans; but the credibility of our work rests on its independence. In large part due to the integrity and professionalism of the state authorities with whom we work, we have been able to exercise our best judgment in determining research strategies and interpreting results.
What technical and analytic contributions to the field has CALDER made, as their researchers have mined growing sources of state administrative longitudinal data with state-of-the-art econometric techniques?
Our research has made technical contributions in two areas. First, while value-added measures of teacher effectiveness are central to much of our research, they are also relatively new indicators of teacher quality and, consequently, controversial. For both of these reasons, in addition to conducting research on factors related to these teacher quality indicators, and the distribution of teachers of different quality across schools serving different populations, we have also conducted research to help better understand the strengths and weaknesses of these measures, including how well value-added measures predict future teacher performance. We are also beginning to examine how high value-added teachers may have effects beyond test scores, such as on-time student progress, high school graduation and later earnings.
Second, our research has capitalized on the analytic advantages provided by the longitudinal and census nature of state administrative data in ways that increase the internal validity of our findings. The data allow us to produce highly credible results—for example, by using observational units as their own controls, such as with fixed-effects models, and by using different quasi-experimental techniques, such as regression discontinuity—that permit reasonable causal interpretation of findings without random assignment.
Are there other things that CALDER has learned about how teacher policies, governance policies, and social and economic community conditions affect outcomes for teachers and their students that you would like to share with our readers?
Our work on teacher layoffs is an apt example that highlights both the virtues of CALDER's multi-state team structure, and our approach to policy research. A year ago, when teacher layoffs emerged as a very real policy issue for many school districts, we mounted an effort to examine the effect on student achievement of different layoff strategies—one based on seniority versus one based on measures of teacher effectiveness. In New York, we conducted a simulation that compared two different imposed conditions: (1) laying off teachers identified with a LIFO-based policy (last in-first out) and (2) laying off teachers identified as weak by value-added measures.
The research showed the two approaches resulted in radically different sets of teachers laid off and that the differences had educationally meaningful estimated effects on subsequent student achievement. We followed up with analyses in Washington State where we used data on teachers who had actually received layoff notices. The research confirmed that layoffs were being decided primarily by seniority. Both studies reached similar conclusions: students in affected classrooms learned more and fewer teaching posts were lost under effectiveness-based layoff policies. Indeed, the Washington analysis showed that where less effective teachers were retained, students lost about 3 months of learning. And the New York study showed that 25 percent fewer teachers would have to be laid off under the effectiveness policy than under the seniority policy to meet a 5 percent budget reduction. Because we were able to conduct the research quickly, and because we could replicate findings across states, the results received heavy weight in policy determinations across the country.