Value-Added Models and Accountability: Next Steps

Value-added models and indicators are being considered for use and are being used to measure the performance of schools, teachers, programs, and policies on an ongoing basis. Most current value-added models fall into two somewhat distinct groups, those that treat individual effects as random and those that treat individual effects as fixed. Random-effects models have been found to yield relatively precise estimates of school and classroom effects but can suffer from an inability to control for student selectivity (e.g., schools or classrooms serve different student populations) that produces estimates subject to selection bias. Fixed effects models control for time-invariant selectivity but typically yield relatively less precise school and classroom effects estimates.

This project sought to develop and refine a new value-added model that combines the best features of models with random and fixed individual effects—that is, high precision and low selection bias. Known as the generalized value-added model with conditional random effects and multivariate shrinkage (CRE-MS) this model is to yield reductions in expected mean squared error relative to other value-added models and provide estimates of precision that take into account the error due to uncontrolled selection bias as well as traditional estimation error. The development of the CRE-MS model builds off of two research strands. First is the literature on testing the assumptions of the fixed effects model and imposing them in a more flexible manner that can lead to the development of conditional random effects (CRE) model which captures the nonrandom selection/assignment of students to schools and classrooms. Second is the work on shrinkage estimation that can be applied to the CRE model and used to optimally balance precision and selection bias. Different approaches for computing multivariate shrinkage estimates will be investigated for the CRE-MS model, in part to address the computational burden involved and the potential need to use information across multiple student cohorts.

In addition to developing CRE-MS model, the project extended it to accommodate: (a) midyear testing; (b) multidimensionality in measured achievement, possible violations of the equal interval property, and irregularities in horizontal and vertical equating; and (c) multiple assessment systems in the same district, including short-cycle assessments.