Value-Added Models and Accountability: Next Steps

This project sought to develop and refine a new value-added model that combines the best features of models with random and fixed individual effects—that is, high precision and low selection bias. Known as the generalized value-added model with conditional random effects and multivariate shrinkage (CRE-MS) this model is to yield reductions in expected mean squared error relative to other value-added models and provide estimates of precision that take into account the error due to uncontrolled selection bias as well as traditional estimation error. The development of the CRE-MS model builds off of two research strands. First is the literature on testing the assumptions of the fixed effects model and imposing them in a more flexible manner that can lead to the development of conditional random effects (CRE) model which captures the nonrandom selection/assignment of students to schools and classrooms. Second is the work on shrinkage estimation that can be applied to the CRE model and used to optimally balance precision and selection bias. Different approaches for computing multivariate shrinkage estimates will be investigated for the CRE-MS model, in part to address the computational burden involved and the potential need to use information across multiple student cohorts.

In addition to developing CRE-MS model, the project extended it to accommodate: (a) midyear testing; (b) multidimensionality in measured achievement, possible violations of the equal interval property, and irregularities in horizontal and vertical equating; and (c) multiple assessment systems in the same district, including short-cycle assessments.

Four urban school districts worked with the project and their data was used to test the viability of the CRE-MS models when using real data and determine 1) if the models provide clear and useful information to the schools and districts, 2) if the information is more accurate and useful than what is now provided using existing value-added models.