Skip Navigation
Technical Methods Report: What to Do When Data Are Missing in Group Randomized Controlled Trials

NCEE 2009-0049
October 2009

4. Testing the Performance of Selected Missing Data Methods

To provide further guidance to education researchers regarding what to do about missing data in an RCT, we conducted simulations to test the performance of selected missing data analysis methods described in Chapter 3. The simulations were run under conditions that varied on three dimensions: (1) the amount of missing data assumed, relatively low (5% missing) vs. relatively high (40% missing); (2) the level at which data are missing—at the level of schools (the assumed unit of randomization) and for students missing within schools; and, (3) the previously discussed underlying missing data mechanisms—i.e., MCAR, MAR, and NMAR. The benefit of conducting such simulations is that we know the true impact of our hypothetical intervention, and this allows us to compare the magnitude and precision of the estimated impacts produced by the different missing data methods under these varying conditions. This chapter begins with a description of the simulation methodology (greater details can be found in Appendices C and E), and then summarizes the simulation results (complete results are provided in Appendix D).