Measurement error issues adversely affect results obtained from typical modeling approaches used to analyze data from assessment and evaluation studies. In particular, measurement error can weaken the validity of inferences from student assessment data, such as the inferences made from the results of using value-added models. Measurement error also can reduce the statistical power of impact studies and can diminish the ability of researchers to identify the causal mechanisms that lead to an intervention improving the desired outcome.

The purpose of this project is to develop models and statistical software to account properly for the impacts of measurement error. Researchers plan to use multilevel latent variable modeling to develop multi-stage and single-stage estimation methods that will address some problems created by measurement error. By the end of the study, researchers will develop statistical models, computer software, and accompanying manuals that will be useful for addressing these difficulties related to measurement error. In addition, the researchers plan to offer workshops and web-based materials that will help applied researchers make use of the models and software they have developed.

Products and Publications

Book chapter

Cai, L., Choi, K., and Kuhfeld, M. (2016). On the Role of Multilevel Item Response Models in Multisite Evaluation Studies for Serious Games. In H.F. O'Neil, E.L. Baker, and R.S. Perez (Eds.), Using Games and Simulations for Teaching and Assessment: Key Issues (pp. 280–301). New York: Routledge.

Cai, L., and Thissen, D. (2014). Modern Approaches to Parameter Estimation in Item Response Theory. In S.P. Reise, and D. A. Revicki (Eds.), Handbook of Item Response Theory Modeling: Applications to Typical Performance Assessment (pp. 41–59). Routledge.

Thissen, D., And Cai, L. (2014). Modern Approaches to Parameter Estimation in Item Response Theory. In S.P. Reise, and D. A. Revicki (Eds.), Handbook of Item Response Theory Modeling (pp. 59–77). Routledge.

Yang, J. S., and Seltzer, M. (2015). Handling Measurement Error in Predictors Using a Multilevel Latent Variable Plausible Values Approach. In Advances in Multilevel Modeling for Educational Research: Addressing Practical Issues Found in Real-World Applications (pp. 295–333). Information Age Publishing.

Journal article, monograph, or newsletter

Bonifay, W., and Cai, L. (2017). On the Complexity of Item Response Theory Models. Multivariate Behavioral Research, 52(4), 465–484.

Cai, L., Choi, K., Hansen, M., and Harrell, L. (2016). Item Response Theory. Annual Review of Statistics and Its Application, 3: 297–321.

Falk, C.F., and Cai, L. (2016). Semiparametric Item Response Functions in the Context of Guessing. Journal of Educational Measurement, 53(2), 229–247.

Falk, C.F., and Cai, L. (2016). Maximum Marginal Likelihood Estimation of a Monotonic Polynomial Generalized Partial Credit Model With Applications to Multiple Group Analysis. Psychometrika, 81(2), 434–460.

Falk, C.F., and Cai, L. (2016). A Flexible Full-Information Approach to the Modeling of Response Styles. Psychological Methods, 21(3), 328.

Hansen, M., Cai, L., Monroe, S., and Li, Z. (2016). Limited-Information Goodness-of-Fit Testing of Diagnostic Classification Item Response Models. British Journal of Mathematical and Statistical Psychology, 69(3), 225–252.

Lee, T., Cai, L., and Kuhfeld, M. (2016). A Poor Person's Posterior Predictive Checking of Structural Equation Models. Structural Equation Modeling: A Multidisciplinary Journal, 23(2), 206–220.

Li, Z., and Cai, L. (2012). Summed Score Likelihood–Based Indices for Testing Latent Variable Distribution Fit in Item Response Theory. Educational and Psychological Measurement, 0013164417717024.

Monroe, S., and Cai, L. (2015). Examining the Reliability of Student Growth Percentiles Using Multidimensional IRT. Educational Measurement: Issues and Practice, 34(4), 21–30.

Monroe, S., and Cai, L. (2015). Evaluating Structural Equation Models for Categorical Outcomes: A New Test Statistic and a Practical Challenge of Interpretation. Multivariate Behavioral Research, 50(6), 569–583.

Yang, J., and Cai, L. (2014). Estimation of Contextual Effects Through Nonlinear Multilevel Latent Variable Modeling With a Metropolis-Hastings Robbins-Monro Algorithm. Journal of Educational and Behavioral Statistics, 39(6): 550–582.