Exhibit II.b.2. Scenario II, Missing Data Dependent on Pretest: Data Missing for 40% of Schools

Technical Methods Report: What to Do When Data Are Missing in Group Randomized Controlled Trials

NCEE 2009-0049
October 2009

List of Tables

Data	Pre-test Data Available?	Estimate	True Imact	Bias	Bias Level	Standard Estimate	Unbiased Estimate	Bias	Bias Level	% of Samples in which 90% Cl Contains .20
		Imact Estimate				Standard Error of Imact Est.				90% Cl
No Missing Data	No	0.203	0.200	0.003	Low Bias	0.085	0.088	-0.002	Low Bias 0.892 0.892
	Yes	0.203	0.200	0.003	Low Bias	0.062	0.062	0.000	Low Bias	0.886
A. Pre-Test (X) Data Missing
Case Deletion	No	0.199	0.200	-0.001	Low Bias	0.081	0.082	-0.002	Low Bias	0.888
	Yes
Dummy Variable Method	No	0.213	0.200	0.013	Low Bias	0.073	0.074	-0.002	Low Bias	0.890
	Yes
Mean Value Imputation	No	0.233	0.200	0.033	Low Bias	0.073	0.078	-0.006	Low Bias	0.837
	Yes
Single, Non-stochastic	No	0.224	0.200	0.024	Low Bias	0.055	0.083	-0.029	Low Bias	0.698
Regression Imputation	Yes
Single, Stochastic	No	0.219	0.200	0.019	Low Bias	0.062	0.081	-0.019	Low Bias	0.783
Regression Imputation	Yes
Multiple Stochastic	No	0.215	0.200	0.015	Low Bias	0.078	0.074	0.004	Low Bias	0.908
Regrssion Imputation (n=5)	Yes
EM Algorithm with Multiple Imputation (n=5)	No	0.222	0.200	0.022	Low Bias	0.074	0.076	-0.002	Low Bias	0.874
	Yes
B. Post-Test (Y) Data Missing
Case Deletion	No	0.165	0.200	-0.035	Low Bias	0.111	0.114	-0.003	Low Bias	0.873
	Yes	0.199	0.200	-0.001	Low Bias	0.081	0.082	-0.002	Low Bias	0.888
Mean Value Imputation	No	0.165	0.200	-0.035	Low Bias	0.064	0.114	-0.050	Low Bias	0.622
	Yes	0.165	0.200	-0.035	Low Bias	0.055	0.099	-0.044	Low Bias	0.615
Single, Non-stocastic	No	0.165	0.200	-0.035	Low Bias	0.064	0.114	-0.050	Low Bias	0.622
Regression Imputation	Yes	0.202	0.200	0.002	Low Bias	0.048	0.086	-0.038	Low Bias	0.651
Single, Stochastic	No	0.165	0.200	-0.035	Low Bias	0.083	0.120	-0.037	Low Bias	0.710
Regression Imputation	Yes	0.200	0.200	0.000	Low Bias	0.061	0.090	-0.029	Low Bias	0.729
Multiple, Stochastic	No	0.170	0.200	-0.030	Low Bias	0.152	0.121	0.031	Low Bias	0.929
Regression Imputation (n=5)	Yes	0.050	0.200	0.005	Low Bias	0.112	0.092	0.202	Low Bias	0.932
EM Algorithm with Multiple Imputation (n=5)	No	0.194	0.200	-0.006	Low Bias	0.087	0.089	-0.002	Low Bias	0.879
	Yes
Weighting - Simple	No
	Yes
Weighting - Sophisticated	No
	Yes	0.201	0.200	0.001	Low Bias	0.081	0.082	-0.002	Low Bias	0.891
Fully Specified Regression Models	No	0.202	0.200	0.002	Low Bias	0.081	0.082	-0.002	Low Bias	0.889
w/ Treatment-Covariate Interactions	Yes
Note: When pre-test scores are available, they are used as a covariate in the analysis model. In addition, we used pre-test scores to impute values and create weights. Bias estimates were computed as described in Chapter 4 and repeated at the beginning of this appendix. The level of the bias is characterized as "High Bias" or "Low Bias" based on the criteria established in Chapter 4. 90% CI refers to the 90-percent confidence interval around the impact estimate. For more details on the simulations, see Chapter 4 and Appendix C.