Hierarchical Linear Modeling Under Multilevel Non-Ignorable Non-Responses with Applications to NAEP Data

The National Assessment of Educational Progress (NAEP) conducts surveys on the educational achievement of students in the United States. Multistage cluster sampling schemes are widely employed in those surveys, and multilevel non-responses at both the school and student levels are common in those surveys. The non-response adjusted Horvitz-Thompson (HT) estimator, which weights cases by the inverse of their selection probabilities adjusted by the non-response rate, has been regularly used to report the findings for the NAEP data. The non-response adjusted HT estimator provides a consistent estimate when the multilevel non-responses in the NAEP data are ignorable. However, in many of the NAEP assessments, non-responses may be related to the outcome variable. For example, schools and students with lower performances may be reluctant to participate in the assessment. In such cases, the non-responses are informative and thus are not ignorable. Using the non-response adjusted HT estimator in such situations would lead to biased estimates.