Exhibit I.b.2. Scenario I, Missing Data Not Dependent on Pretest or Post-test: Data Missing for 40% of Students

Technical Methods Report: What to Do When Data Are Missing in Group Randomized Controlled Trials

NCEE 2009-0049
October 2009

List of Tables

Data	Pre-test Data Available?	Estimate	True Imact	Bias	Bias Level	Standard Estimate	Unbiased Estimate	Bias	Bias Level	% of Samples in which 90% Cl Contains .20
		Imact Estimate				Standard Error of Imact Est.				90% Cl
No Missing Data	No	0.203	0.200	0.003	Low Bias	0.085	0.088	-0.002	Low Bias	0.892
	Yes	0.203	0.200	0.003	Low Bias	0.062	0.062	0.000	Low Bias	0.886
A. Pre-Test (X) Data Missing
Case Deletion	No	0.202	0.200	0.002	Low Bias	0.081	0.080	0.001	Low Bias	0.901
	Yes
Dummy Variable Method	No	0.203	0.200	0.003	Low Bias	0.073	0.075	-0.001	Low Bias	0.893
	Yes
Mean Value Imputation	No	0.203	0.200	0.003	Low Bias	0.073	0.079	-0.006	Low Bias	0.859
	Yes
Single, Non-stochastic	No	0.201	0.200	0.001	Low Bias	0.055	0.081	-0.026	Low Bias	0.736
Regression Imputation	Yes
Single, Stochastic	No	0.201	0.200	0.001	Low Bias	0.062	0.079	-0.017	Low Bias	0.780
Regression Imputation	Yes
Multiple Stochastic	No	0.205	0.200	0.005	Low Bias	0.069	0.074	-0.005	Low Bias	0.871
Regrssion Imputation (n=5)	Yes
EM Algorithm with Multiple Imputation (n=5)	No	0.200	0.200	0.000	Low Bias	0.072	0.076	-0.004	Low Bias	0.875
	Yes
B. Post-Test (Y) Data Missing
Case Deletion	No	0.202	0.200	0.002	Low Bias	0.112	0.112	0.000	Low Bias	0.886
	Yes	0.202	0.200	0.002	Low Bias	0.081	0.080	0.001	Low Bias	0.901
Mean Value Imputation	No	0.202	0.200	0.002	Low Bias	0.065	0.112	-0.047	Low Bias	0.679
	Yes	0.202	0.200	0.002	Low Bias	0.055	0.097	-0.042	Low Bias	0.677
Single, Non-stocastic	No	0.202	0.200	0.002	Low Bias	0.065	0.112	-0.047	Low Bias	0.679
Regression Imputation	Yes	0.201	0.200	0.001	Low Bias	0.048	0.082	-0.034	Low Bias	0.674
Single, Stochastic	No	0.205	0.200	0.005	Low Bias	0.083	0.129	-0.046	Low Bias	0.694
Regression Imputation	Yes	0.203	0.200	0.003	Low Bias	0.060	0.095	-0.034	Low Bias	0.702
Multiple, Stochastic	No	0.195	0.200	-0.005	Low Bias	0.107	0.114	-0.007	Low Bias	0.863
Regression Imputation (n=5)	Yes	0.196	0.200	-0.004	Low Bias	0.077	0.084	-0.007	Low Bias	0.853
EM Algorithm with Multiple Imputation (n=5)	No	0.204	0.200	0.004	Low Bias	0.085	0.083	0.002	Low Bias	0.878
	Yes
Weighting - Simple	No
	Yes
Weighting - Sophisticated	No
	Yes	0.202	0.200	0.002	Low Bias	0.081	0.080	0.001	Low Bias	0.899
Fully Specified Regression Models	No	0.202	0.200	0.002	Low Bias	0.081	0.080	0.001	Low Bias	0.900
w/ Treatment-Covariate Interactions	Yes
Note: When pre-test scores are available, they are used as a covariate in the analysis model. In addition, we used pre-test scores to impute values and create weights. Bias estimates were computed as described in Chapter 4 and repeated at the beginning of this appendix. The level of the bias is characterized as "High Bias" or "Low Bias" based on the criteria established in Chapter 4. 90% CI refers to the 90-percent confidence interval around the impact estimate. For more details on the simulations, see Chapter 4 and Appendix C.