Exhibit I.a.1. Scenario I, Missing Data Not Dependent on Pretest or Post-test: Data Missing for 5% of Students

Technical Methods Report: What to Do When Data Are Missing in Group Randomized Controlled Trials

NCEE 2009-0049
October 2009

List of Tables

Data	Pre-test Data Available?	Estimate	True Imact	Bias	Bias Level	Standard Estimate	Unbiased Estimate	Bias	Bias Level	% of Samples in which 90% Cl Contains .20
		Imact Estimate				Standard Error of Imact Est.				90% Cl
No Missing Data	No	0.203	0.200	0.003	Low Bias	0.085	0.088	0.002	Low Bias	0.892
	Yes	0.203	0.200	0.003	Low Bias	0.062	0.062	0.000	Low Bias	0.886
A. Pre-Test (X) Data Missing
Case Deletion	No	0.201	0.200	0.001	Low Bias	0.062	0.063	-0.001	Low Bias	0.886
	Yes
Dummy Variable Method	No	0.201	0.200	0.001	Low Bias	0.062	0.063	-0.001	Low Bias	0.886
	Yes
Mean Value Imputation	No	0.201	0.200	0.001	Low Bias	0.062	0.063	-0.001	Low Bias	0.887
	Yes
Single, Non-stochastic	No	0.201	0.200	0.001	Low Bias	0.062	0.063	-0.001	Low Bias	0.882
Regression Imputation	Yes
Single, Stochastic	No	0.201	0.200	0.001	Low Bias	0.062	0.063	-0.001	Low Bias	0.883
Regression Imputation	Yes
Multiple Stochastic	No	0.201	0.200	0.001	Low Bias	0.062	0.063	-0.001	Low Bias	0.883
Regrssion Imputation (n=5)	Yes
EM Algorithm with Multiple Imputation (n=5)	No	0.201	0.200	0.001	Low Bias	0.062	0.063	-0.001	Low Bias	0.884
	Yes
B. Post-Test (Y) Data Missing
Case Deletion	No	0.200	0.200	0.000	Low Bias	0.860	0.860	0.000	Low Bias	0.895
	Yes	0.201	0.200	0.001	Low Bias	0.062	0.063	-0.001	Low Bias	0.886
Mean Value Imputation	No	0.200	0.200	0.000	Low Bias	0.081	0.086	-0.004	Low Bias	0.876
	Yes	0.201	0.200	0.001	Low Bias	0.059	0.063	-0.004	Low Bias	0.868
Single, Non-stocastic	No	0.200	0.200	0.000	Low Bias	0.086	0.086	0.000	Low Bias	0.891
Regression Imputation	Yes	0.201	0.200	0.001	Low Bias	0.059	0.063	-0.001	Low Bias	0.883
Single, Stochastic	No	0.200	0.200	0.000	Low Bias	0.086	0.086	0.000	Low Bias	0.891
Regression Imputation	Yes	0.201	0.200	0.001	Low Bias	0.620	0.063	-0.001	Low Bias	0.890
Multiple, Stochastic	No	0.199	0.200	-0.001	Low Bias	0.086	0.086	0.000	Low Bias	0.898
Regression Imputation (n=5)	Yes	0.200	0.200	0.000	Low Bias	0.063	0.063	-0.001	Low Bias	0.890
EM Algorithm with Multiple Imputation (n=5)	No	0.200	0.200	0.000	Low Bias	0.086	0.086	0.001	Low Bias	0.899
	Yes
Weighting - Simple	No	0.200	0.200	0.000	Low Bias	0.086	0.086	0.000	Low Bias	0.895
	Yes	0.201	0.200	0.001	Low Bias	0.062	0.063	-0.001	Low Bias	0.886
Weighting - Sophisticated	No	0.200	0.200	0.000	Low Bias	0.086	0.086	0.000	Low Bias	0.896
	Yes	0.201	0.200	0.001	Low Bias	0.062	0.063	-0.001	Low Bias	0.886
Fully Specified Regression Models	No	0.201	0.200	0.001	Low Bias	0.062	0.063	-0.001	Low Bias	0.888
w/ Treatment-Covariate Interactions	Yes
Notes: When pre-test scores are available, they are used as a covariate in the analysis model. In addition, we used pre-test scores to impute values and create weights. Bias estimates were computed as described in Chapter 4 and repeated at the beginning of this appendix. The level of the bias is characterized as "High Bias" or "Low Bias" based on the criteria established in Chapter 4. 90% CI refers to the 90-percent confidence interval around the impact estimate. For more details on the simulations, see Chapter 4 and Appendix C.