Purpose: The National Assessment of Educational Progress (NAEP) conducts surveys on the educational achievement of students in the United States. Multistage cluster sampling schemes are widely employed in those surveys, and multilevel non-responses at both the school and student levels are common in those surveys. The non-response adjusted Horvitz-Thompson (HT) estimator, which weights cases by the inverse of their selection probabilities adjusted by the non-response rate, has been regularly used to report the findings for the NAEP data. The non-response adjusted HT estimator provides a consistent estimate when the multilevel non-responses in the NAEP data are ignorable. However, in many of the NAEP assessments, non-responses may be related to the outcome variable. For example, schools and students with lower performances may be reluctant to participate in the assessment. In such cases, the non-responses are informative and thus are not ignorable. Using the non-response adjusted HT estimator in such situations would lead to biased estimates.

Project Activities: To address the problem of bias in the NAEP data, the researchers will develop a flexible modeling procedure to take into account multilevel, non-ignorable, non-responses in order to provide an unbiased assessment of American student performance. More specifically, the objectives of this proposal are: (1) to develop hierarchical linear models to incorporate multilevel non-ignorable, non-response mechanisms of the NAEP data into the modeling process; (2) to develop methodology for estimating parameters of interest in the proposed hierarchical linear models; (3) to develop statistical tools for model selection and model adequacy assessment; (4) to provide estimators for population mean and other parameters of interest in the NAEP data based on the selected hierarchical linear model; and (5) to compare the proposed methodology with existing methods.

Products and Publications

Journal article, monograph, or newsletter

Yu, Y., and Li, J. (2015). A Nonparametric Test of Missing Completely at Random for Incomplete Multivariate Data. Psychometrika, 80(3): 707–726.