Skip to main content

Breadcrumb

Home arrow_forward_ios Information on IES-Funded Research arrow_forward_ios Novel Models and Methods to Address ...
Home arrow_forward_ios ... arrow_forward_ios Novel Models and Methods to Address ...
Information on IES-Funded Research
Grant Closed

Novel Models and Methods to Address Measurement Error Issues in Educational Assessment and Evaluation Studies

NCER
Program: Statistical and Research Methodology in Education
Program topic(s): Core
Award amount: $895,108
Principal investigator: Li Cai
Awardee:
University of California, Los Angeles
Year: 2014
Project type:
Methodological Innovation
Award number: R305D140046

Purpose

Measurement error issues adversely affect results obtained from typical modeling approaches used to analyze data from assessment and evaluation studies. In particular, measurement error can weaken the validity of inferences from student assessment data, such as the inferences made from the results of using value-added models. Measurement error also can reduce the statistical power of impact studies and can diminish the ability of researchers to identify the causal mechanisms that lead to an intervention improving the desired outcome. The purpose of this project is to develop models and statistical software to account properly for the impacts of measurement error.

Project Activities

Researchers plan to use multilevel latent variable modeling to develop multi-stage and single-stage estimation methods that will address some problems created by measurement error. By the end of the study, researchers will develop statistical models, computer software, and accompanying manuals that will be useful for addressing these difficulties related to measurement error. In addition, the researchers plan to offer workshops and web-based materials that will help applied researchers make use of the models and software they have developed.

People and institutions involved

IES program contact(s)

Allen Ruby

Associate Commissioner for Policy and Systems
NCER

Products and publications

ERIC Citations: Find available citations in ERIC for this award here.

Book chapter

Cai, L., Choi, K., and Kuhfeld, M. (2016). On the Role of Multilevel Item Response Models in Multisite Evaluation Studies for Serious Games. In H.F. O'Neil, E.L. Baker, and R.S. Perez (Eds.), Using Games and Simulations for Teaching and Assessment: Key Issues (pp. 280-301). New York: Routledge.

Cai, L., and Thissen, D. (2014). Modern Approaches to Parameter Estimation in Item Response Theory. In S.P. Reise, and D. A. Revicki (Eds.), Handbook of Item Response Theory Modeling: Applications to Typical Performance Assessment (pp. 41-59). Routledge.

Thissen, D., And Cai, L. (2014). Modern Approaches to Parameter Estimation in Item Response Theory. In S.P. Reise, and D. A. Revicki (Eds.), Handbook of Item Response Theory Modeling (pp. 59-77). Routledge.

Yang, J. S., and Seltzer, M. (2015). Handling Measurement Error in Predictors Using a Multilevel Latent Variable Plausible Values Approach. In Advances in Multilevel Modeling for Educational Research: Addressing Practical Issues Found in Real-World Applications (pp. 295-333). Information Age Publishing.

Journal article, monograph, or newsletter

Bonifay, W., and Cai, L. (2017). On the Complexity of Item Response Theory Models. Multivariate Behavioral Research, 52(4), 465-484.

Cai, L., Choi, K., Hansen, M., and Harrell, L. (2016). Item Response Theory. Annual Review of Statistics and Its Application, 3: 297-321.

Falk, C.F., and Cai, L. (2016). Semiparametric Item Response Functions in the Context of Guessing. Journal of Educational Measurement, 53(2), 229-247.

Falk, C.F., and Cai, L. (2016). Maximum Marginal Likelihood Estimation of a Monotonic Polynomial Generalized Partial Credit Model With Applications to Multiple Group Analysis. Psychometrika, 81(2), 434-460.

Falk, C.F., and Cai, L. (2016). A Flexible Full-Information Approach to the Modeling of Response Styles. Psychological Methods, 21(3), 328.

Hansen, M., Cai, L., Monroe, S., and Li, Z. (2016). Limited-Information Goodness-of-Fit Testing of Diagnostic Classification Item Response Models. British Journal of Mathematical and Statistical Psychology, 69(3), 225-252.

Lee, T., Cai, L., and Kuhfeld, M. (2016). A Poor Person's Posterior Predictive Checking of Structural Equation Models. Structural Equation Modeling: A Multidisciplinary Journal, 23(2), 206-220.

Li, Z., and Cai, L. (2012). Summed Score Likelihood-Based Indices for Testing Latent Variable Distribution Fit in Item Response Theory. Educational and Psychological Measurement, 0013164417717024.

Monroe, S., and Cai, L. (2015). Examining the Reliability of Student Growth Percentiles Using Multidimensional IRT. Educational Measurement: Issues and Practice, 34(4), 21-30.

Monroe, S., and Cai, L. (2015). Evaluating Structural Equation Models for Categorical Outcomes: A New Test Statistic and a Practical Challenge of Interpretation. Multivariate Behavioral Research, 50(6), 569-583.

Yang, J., and Cai, L. (2014). Estimation of Contextual Effects Through Nonlinear Multilevel Latent Variable Modeling With a Metropolis-Hastings Robbins-Monro Algorithm. Journal of Educational and Behavioral Statistics, 39(6): 550-582.

Questions about this project?

To answer additional questions about this project or provide feedback, please contact the program officer.

 

Tags

MathematicsData and Assessments

Share

Icon to link to Facebook social media siteIcon to link to X social media siteIcon to link to LinkedIn social media siteIcon to copy link value

Questions about this project?

To answer additional questions about this project or provide feedback, please contact the program officer.

 

You may also like

Zoomed in IES logo
Workshop/Training

Data Science Methods for Digital Learning Platform...

August 18, 2025
Read More
Zoomed in IES logo
Workshop/Training

Meta-Analysis Training Institute (MATI)

July 28, 2025
Read More
Zoomed in Yellow IES Logo
Workshop/Training

Bayesian Longitudinal Data Modeling in Education S...

July 21, 2025
Read More
icon-dot-govicon-https icon-quote