Multiple Imputation Procedures for Multilevel Data

The primary goal of this project was to develop a user-friendly stand-alone software package that accommodates missing data in two- and three-level data structures. Missing data are a frequent occurrence in education research, and they pose a number of challenges for multilevel analyses that do not arise in single-level analyses. Although maximum likelihood estimation (or full information maximum likelihood, FIML) is the default estimation approach in most multilevel software packages, it is not optimal for handling missing data. Most multilevel modeling programs necessarily exclude cases with missing predictors, and cases with missing outcome scores are also excluded from cross-sectional analyses. This removal of cases is problematic when data are missing at level 2 (or higher) because entire clusters are excluded from the analysis.

In the first part of this study, the research team developed and evaluated a multiple imputation algorithm for two-level data structures, with the algorithm being able to accommodate nominal, ordinal, and interval/ratio variables at every level of the data hierarchy. The imputation model also allowed for random slopes between any pair of level-1 variables. For the second part, the research team extended the model and software to three-level data structures. For the third part of the study, the research team examined the behavior of test statistics on data that have undergone multiple imputation. Statistical significance tests are often used in complete-data multilevel analyses, and analogous procedures are available for multiply imputed data, but these tests were originally developed for single-level analyses and have not been systematically studied for use in multilevel analyses. All three parts of the study included Monte Carlo simulations for testing the performance of the imputation procedures and test statistics under various conditions. The research team disseminated the results of this project through peer-reviewed conference presentations and journal manuscripts. The research team held workshops on the techniques and software at major conferences and made the user-friendly software freely available online.