Skip to main content

Breadcrumb

Home arrow_forward_ios Information on IES-Funded Research arrow_forward_ios Innovative Statistical Learning Met ...
Home arrow_forward_ios ... arrow_forward_ios Innovative Statistical Learning Met ...
Information on IES-Funded Research
Grant Open

Innovative Statistical Learning Methods and Software for Large-Scale Assessment

NCER
Program: Statistical and Research Methodology in Education
Program topic(s): Core
Award amount: $764,021
Principal investigator: Chun Wang
Awardee:
University of Washington
Year: 2020
Project type:
Methodological Innovation
Award number: R305D200015

Purpose

When the constructs that educational assessments try to measure are multifaceted, whether by design or not, multidimensional item response theory (MIRT), also known as item factor analysis, provides a unified framework and convenient statistical tool for item analysis, calibration, and scoring. The advancement of computational and statistical techniques has allowed for increased usage of MIRT models, but even with state-of-the-art algorithms, the computation can still be time-consuming, especially when the number of factors is at least five. The purpose of this grant is to develop more efficient statistical models, software, and guidelines for high-dimensional data, large sample sizes, large item banks, and complex designs.

Project Activities

After theoretical work to develop the statistical models, the research team will conduct simulation studies based on real data in order to test the properties of the new models and to compare them to currently used models. The team will use data from NELS 88 and NAEP to provide an applied demonstration of the new models. They will also develop software for estimating the model parameters and for detecting differential item functioning (DIF) within the MIRT framework, with a plan to post the source code on GitHub. The software will have a more user-friendly form via a Shiny app, which will have a user's manual that documents detailed examples and recommended guidelines for students, researchers, and assessment practitioners. The team will use online modules and in-person workshops to test and the user-friendliness of the software and make refinements as needed. The software and the theoretical work will be disseminated through peer-reviewed journal articles and conference presentations.

People and institutions involved

IES program contact(s)

Charles Laurin

Education Research Analyst
NCER

Products and publications

Chen, Y., Li, C., Ouyang, J., & Xu, G. (2023). A note on statistical inference for noisy incomplete 1-bit matrix. Journal of Machine Learning Research.

Chiou, S. H., Xu, G., Yan, J., & Huang, C. Y. (2023). Regression modeling for recurrent events possibly with an informative terminal event using R package reReg. Journal of Statistical Software, 105, 1-34.

Chiou, S. H., Xu, G., Yan, J., & Huang, C. Y. (2021). Regression Modeling for Recurrent Events Using R Package reReg. arXiv preprint arXiv:2104.11708.

Cho, A. E., Xiao, J., Wang, C., & Xu, G. (2022). Regularized variational estimation for exploratory item factor analysis. Psychometrika, 1-29.

Cho, A. E., Wang, C., Zhang, X., & Xu, G. (2021). Gaussian variational estimation for multidimensional item response theory. British Journal of Mathematical and Statistical Psychology, 74, 52-85.

Gu, Y., Erosheva, E. A., Xu, G., & Dunson, D. B. (2023). Dimension-grouped mixed membership models for multivariate categorical data. Journal of Machine Learning Research, 24(88), 1-49.

Jiang, S., Xiao, J., & Wang, C. (2022). On-the-fly parameter estimation based on item response theory in item-based adaptive learning systems. Behavior Research Methods, 1-21.

Liu, T., Wang, C., & Xu, G. (2022). Estimating three-and four-parameter MIRT models with importance-weighted sampling enhanced variational auto-encoder. Frontiers in Psychology, 13, 935419.

Lu, J., Wang, C., & Shi, N. (2023). A mixture response time process model for aberrant behaviors and item nonresponses. Multivariate Behavioral Research, 58(1), 71-89.

Lu, J., Wang, C., Zhang, J., & Wang, X. (2023). A sequential Bayesian changepoint detection procedure for aberrant behaviours in computerized testing. British Journal of Mathematical and Statistical Psychology.

Ma, C., Ouyang, J., & Xu, G. (2023). Learning latent and hierarchical structures in cognitive diagnosis models. Psychometrika, 88(1), 175-207.

Ma, W., Wang, C., & Xiao, J. (2023). A testlet diagnostic classification model with attribute hierarchies. Applied Psychological Measurement, 47(3), 183-199.

Ma, C., & Xu, G. (2021). Hypothesis testing for hierarchical structures in cognitive diagnosis models. arXiv preprint arXiv:2106.03218.

Meng, X., & Xu, G. (2022). A mixed stochastic approximation EM (MSAEM) algorithm for the estimation of the four-parameter normal ogive model. Psychometrika, 1-36.

Ouyang, J., & Xu, G. (2022). Identifiability of latent class models with covariates. psychometrika, 87(4), 1343-1360.

Wang, C., Zhu, R., & Xu, G. (2023). Using lasso and adaptive lasso to identify DIF in multidimensional 2PL models. Multivariate Behavioral Research, 58(2), 387-407.

Wang, Z., Wang, C., & Weiss, D. J. (2022). Termination criteria for grid multiclassification adaptive testing with multidimensional polytomous items. Applied Psychological Measurement, 46(7), 551-570.

Zhang, X., & Wang, C. (2022). Modified Item-Fit Indices for Dichotomous IRT Models with Missing Data. Applied Psychological Measurement, 46(8), 705-719.

Zhang, J., Wang, C., & Lu, J. (2023). Modeling item revisiting behavior in computer-based testing: Exploring the effect of item revisitations as collateral information. Behavior Research Methods, 1-21.

Supplemental information

Co-Principal Investigator: Xu, Gongjun

Questions about this project?

To answer additional questions about this project or provide feedback, please contact the program officer.

 

Tags

MathematicsData and Assessments

Share

Icon to link to Facebook social media siteIcon to link to X social media siteIcon to link to LinkedIn social media siteIcon to copy link value

Questions about this project?

To answer additional questions about this project or provide feedback, please contact the program officer.

 

You may also like

Zoomed in IES logo
Workshop/Training

Data Science Methods for Digital Learning Platform...

August 18, 2025
Read More
Zoomed in IES logo
Workshop/Training

Meta-Analysis Training Institute (MATI)

July 28, 2025
Read More
Zoomed in Yellow IES Logo
Workshop/Training

Bayesian Longitudinal Data Modeling in Education S...

July 21, 2025
Read More
icon-dot-govicon-https icon-quote