Improving Education Policy Analysis and Practice by Addressing Test Score Measurement Error

Test scores are instrumental in education research and decision making. They are being used for a growing number of high-stakes decisions about students, teachers, and schools, and to measure the effectiveness of education policies and interventions. In observational studies where treatment and control groups of students typically differ on attributes related to achievement, test scores are often the key control variables used to adjust for pre-existing differences among groups. When test scores are used as covariates, test score measurement error can bias parameter estimates from statistical models. Observational analyses in education applications that use test scores as control variables, whether for the estimation of treatment effects, teacher value-added effects, student growth distributions, or peer effects, risk potentially large biases if measurement error is not addressed.

Tools and guidance for adjusting properly for measurement error are lacking. In this project, researchers will fill this gap in a number of ways. The research team will conduct case studies of teacher value-added estimation that apply different measurement error correction approaches to understand their behavior in real data and to answer outstanding questions about the properties and feasibility of these approaches The case studies will lead to new knowledge and methodological recommendations that will allow analysts to make more informed decisions during analyses of observational data. For the popular statistical software package R, researchers will construct a library that provides functions to support implementation of all common approaches to measurement error correction using test scores and conditional standard error of measure (CSEM) information. Education researchers will therefore have relatively user-friendly access to using the models and incorporating the recommendations derived from this project.