Exhibit II.a.1. Scenario II, Missing Data Dependent on Pretest: Data Missing for 5% of Students

Technical Methods Report: What to Do When Data Are Missing in Group Randomized Controlled Trials

NCEE 2009-0049
October 2009

List of Tables

Data	Pre-test Data Available?	Estimate	True Imact	Bias	Bias Level	Standard Estimate	Unbiased Estimate	Bias	Bias Level	% of Samples in which 90% Cl Contains .20
		Imact Estimate				Standard Error of Imact Est.				90% Cl
No Missing Data	No	0.203	0.200	0.003	Low Bias	0.085	0.088	-0.002	Low Bias 0.892 0.892
	Yes	0.203	0.200	0.003	Low Bias	0.062	0.062	0.000	Low Bias	0.886
A. Pre-Test (X) Data Missing
Case Deletion	No	0.201	0.200	0.001	Low Bias	0.062	0.062	0.000	Low Bias	0.889
	Yes
Dummy Variable Method	No	0.208	0.200	0.008	Low Bias	0.062	0.063	-0.001	Low Bias	0.884
	Yes
Mean Value Imputation	No	0.213	0.200	0.013	Low Bias	0.062	0.063	-0.001	Low Bias	0.878
	Yes
Single, Non-stochastic	No	0.206	0.200	0.006	Low Bias	0.062	0.062	-0.001	Low Bias	0.883
Regression Imputation	Yes
Single, Stochastic	No	0.206	0.200	0.006	Low Bias	0.062	0.062	0.000	Low Bias	0.881
Regression Imputation	Yes
Multiple Stochastic	No	0.207	0.200	0.007	Low Bias	0.062	0.062	-0.001	Low Bias	0.881
Regrssion Imputation (n=5)	Yes
EM Algorithm with Multiple Imputation (n=5)	No	0.206	0.200	0.006	Low Bias	0.062	0.062	0.000	Low Bias	0.884
	Yes
B. Post-Test (Y) Data Missing
Case Deletion	No	0.191	0.200	-0.009	Low Bias	0.085	0.089	-0.003	Low Bias	0.883
	Yes	0.201	0.200	0.001	Low Bias	0.062	0.062	0.000	Low Bias	0.889
Mean Value Imputation	No	0.189	0.200	-0.011	Low Bias	0.081	0.087	-0.008	Low Bias	0.864
	Yes	0.189	0.200	-0.011	Low Bias	0.059	0.063	-0.004	Low Bias	0.869
Single, Non-stocastic	No	0.191	0.200	-0.009	Low Bias	0.086	0.089	-0.003	Low Bias	0.882
Regression Imputation	Yes	0.202	0.200	0.002	Low Bias	0.062	0.062	0.000	Low Bias	0.890
Single, Stochastic	No	0.191	0.200	-0.009	Low Bias	0.086	0.089	-0.003	Low Bias	0.883
Regression Imputation	Yes	0.201	0.200	0.002	Low Bias	0.062	0.062	0.000	Low Bias	0.889
Multiple, Stochastic	No	0.191	0.200	-0.009	Low Bias	0.086	0.089	-0.003	Low Bias	0.884
Regression Imputation (n=5)	Yes	0.201	0.200	0.001	Low Bias	0.062	0.062	0.000	Low Bias	0.893
EM Algorithm with Multiple Imputation (n=5)	No	0.191	0.200	-0.009	Low Bias	0.086	0.089	-0.003	Low Bias	0.891
	Yes	0.201	0.200	0.001	Low Bias	0.062	0.062	0.000	Low Bias	0.890
Weighting - Simple	No	0.191	0.200	-0.009	Low Bias	0.086	0.089	-0.003	Low Bias	0.881
	Yes	0.201	0.200	0.001	Low Bias	0.062	0.062	0.000	Low Bias	0.891
Weighting - Sophisticated	No	0.191	0.200	-0.009	Low Bias	0.086	0.089	-0.003	Low Bias	0.884
	Yes	0.202	0.200	0.002	Low Bias	0.062	0.062	0.000	Low Bias	0.891
Fully Specified Regression Models	No	0.202	0.200	0.002	Low Bias	0.062	0.062	-0.001	Low Bias	0.891
w/ Treatment-Covariate Interactions	Yes
Note: When pre-test scores are available, they are used as a covariate in the analysis model. In addition, we used pre-test scores to impute values and create weights. Bias estimates were computed as described in Chapter 4 and repeated at the beginning of this appendix. The level of the bias is characterized as "High Bias" or "Low Bias" based on the criteria established in Chapter 4. 90% CI refers to the 90-percent confidence interval around the impact estimate. For more details on the simulations, see Chapter 4 and Appendix C.