|Title:||Improving Software and Methods for Estimating Diagnostic Classification Models and Evaluating Model Fit|
|Principal Investigator:||Thompson, William||Awardee:||University of Kansas|
|Program:||Statistical and Research Methodology in Education–Early Career [Program Details]|
|Award Period:||3 years (08/01/2021 - 07/31/2023)||Award Amount:||$224,996|
|Type:||Methodological Innovation||Award Number:||R305D210045|
The goals of this project are to develop new software for the estimation and evaluation of the log-linear cognitive diagnosis model (the R package measr), and to use the software to conduct a simulation study evaluating the efficacy and efficiency of different model fit measures under a variety of conditions. Diagnostic classification models (DCMs) have shown the ability to provide more fine-grained and actionable scores to guide instruction and detect intervention effects. Despite these benefits, DCMs have yet to gain widespread use in applied research or operational settings. Most existing methods are either limited in their ability to fully assess absolute and relative model fit or are under-researched in the context of DCMs.
In the first stage of the project, the research team will develop measr. The software will estimate the linear cognitive diagnosis model by interfacing with the Stan programming language. This will allow for the implementation of a powerful estimator in a user-friendly interface that is free and available in an open-source environment. The software will also include functions for evaluating the absolute and relative fit of estimated models. Once developed, the research team will use the software in the second stage of the project during which they will conduct Monte Carlo simulations to determine the efficiency and efficacy of various measures of absolute and relative model fit that have been proposed for evaluating DCMs. They will refine the software per the simulation results and based on user-testing that will occur at multiple points during software development. The grant team will publish their findings and software developments in peer-reviewed journals and provide training workshops at a national conference and at Stats Camp to facilitate dissemination and use of measr.