Skip to main content

Breadcrumb

Home arrow_forward_ios Information on IES-Funded Research arrow_forward_ios Generalized, Multilevel, and Longit ...
Home arrow_forward_ios ... arrow_forward_ios Generalized, Multilevel, and Longit ...
Information on IES-Funded Research
Grant Open

Generalized, Multilevel, and Longitudinal Psychometric Models for Evaluating Educational Interventions

NCER
Program: Statistical and Research Methodology in Education
Program topic(s): Core
Award amount: $899,995
Principal investigator: Matthew Madison
Awardee:
University of Georgia
Year: 2022
Award period: 4 years (07/01/2022 - 06/30/2026)
Project type:
Methodological Innovation
Award number: R305D220020

Purpose

Diagnostic classification models (DCMs) can provide useful information about students' strengths and weaknesses in an academic domain beyond the norm-referenced or percentile results typically achieved through item response theory. DCMs, however, need to be able to accommodate the multilevel framework in which most education takes place, or they run the risk of providing substantially inaccurate results. The purpose of this project is to develop and make accessible to applied researchers a multilevel extension to the longitudinal diagnostic classification model (DCM) to help researchers take into account contextual effects that can impact the fidelity and effectiveness of an educational intervention.

Project Activities

The research team will first develop the mathematical models and then program them for testing via Monte Carlo simulation studies to evaluate the validity and reliability of the proposed modeling framework in a variety of research contexts. They will also conduct secondary data analyses to demonstrate the utility and added value of the proposed methods, relative to other common approaches. The results of the simulations and the secondary data comparisons will render a synthesis of practical recommendations for researchers interested in applying the proposed methods in intervention studies.

People and institutions involved

IES program contact(s)

Charles Laurin

Education Research Analyst
NCER

Project contributors

Minjeong Jeon

Co-principal investigator

Products and publications

The research team will develop user-friendly software for conducting multilevel longitudinal DCM analyses and will provide workshops to applied audiences, in addition to supporting the software with online training materials.

Publications:

ERIC Citations: Find available citations in ERIC for this award here.

Publications:

Jurich, D., & Madison, M. J. (2023) Measuring item influence for diagnostic classification models. Educational Assessment. https://doi.org/10.1080/10627197.2023.2244411. 

Maas, L., Madison, M. J., Brinkhuis, M. J. S. (2024). Properties and performance of the one parameter log-linear cognitive diagnosis model. Frontiers in Education. https://doi.org/10.3389/feduc.2024.1287279.

Madison, M.J., Chung, S., Kim, J., and Bradshaw, L.P. (2023). Approaches to Estimating Longitudinal Diagnostic Classification Models. Online first version of article to be published in Behaviormetrika. https://doi.org/10.1007/s41237-023-00202-5.

Madison, M. J., Jeon, M. Cotterell, M. E., Haab, S., & Zor, S. (2025). TDCM: An R package for estimating longitudinal diagnostic classification models. Multivariate Behavioral Research. https://doi.org/10.1080/00273171.2025.2453454.

Madison, M. J., Wind, S. A., Maas, L., Yamaguchi, K., & Haab, S. (2024). A one-parameter diagnostic classification model with familiar properties. Journal of Educational Measurement. https://doi.org/10.1111/jedm.12390.

Ravand, H., Effatpanah, F., Kunina-Habenicht, O., & Madison, M. J. (2025). A didactic illustration of writing skill growth through a longitudinal diagnostic classification model. Frontiers in Psychology, 15. https://doi.org/10.3389/fpsyg.2024.1521808. 

Schellman, M., & Madison, M. J. Estimating the reliability of skill transition in longitudinal diagnostic classification models. (2024). Journal of Educational and Behavioral Statistics. https://doi.org/10.3102/10769986241256032.

Additional project information

The team has produced two software packages to date:

Madison, M. J., Haab., S., Jeon, M., & Cotterell, M. E. (2023-2025). TDCM: An R package for estimating longitudinal diagnostic classification models. Available at https://cran.r-project.org/web/packages/TDCM/index.html.  

Mardones-Segovia, C., Nadar, Y., Madison, M.J., & Cotterell, M. E. (2025). TDCMApp: A Shiny App for TDCM Estimation and Reporting.

Related projects

Psychometric Models for 21st Century Educational Survey Assessments

R305D110027

Developing Enhanced Assessment Tools for Capturing Students' Procedural Skills and Conceptual Understanding in Math

R324A150035

Supplemental information

Co-Principal Investigators: Jeon, Minjeong; Cotterell, Michael

Questions about this project?

To answer additional questions about this project or provide feedback, please contact the program officer.

 

Tags

Academic AchievementData and Assessments

Share

Icon to link to Facebook social media siteIcon to link to X social media siteIcon to link to LinkedIn social media siteIcon to copy link value

Questions about this project?

To answer additional questions about this project or provide feedback, please contact the program officer.

 

You may also like

Zoomed in IES logo
Workshop/Training

Data Science Methods for Digital Learning Platform...

August 18, 2025
Read More
Zoomed in IES logo
Workshop/Training

Meta-Analysis Training Institute (MATI)

July 28, 2025
Read More
Zoomed in Yellow IES Logo
Workshop/Training

Bayesian Longitudinal Data Modeling in Education S...

July 21, 2025
Read More
icon-dot-govicon-https icon-quote