Grant Closed

Hierarchical Linear Modeling Under Multilevel Non-Ignorable Non-Responses with Applications to NAEP Data

NCER

Program: Statistical and Research Methodology in Education

Program topic(s): Core

Award amount: $171,742

Principal investigator: Jun Li

Awardee:

University of California, Riverside

Year: 2009

Award period: 3 years (07/01/2009 - 06/30/2012)

Project type:

Methodological Innovation

Award number: R305D090019

Purpose

The National Assessment of Educational Progress (NAEP) conducts surveys on the educational achievement of students in the United States. Multistage cluster sampling schemes are widely employed in those surveys, and multilevel non-responses at both the school and student levels are common in those surveys. The non-response adjusted Horvitz-Thompson (HT) estimator, which weights cases by the inverse of their selection probabilities adjusted by the non-response rate, has been regularly used to report the findings for the NAEP data. The non-response adjusted HT estimator provides a consistent estimate when the multilevel non-responses in the NAEP data are ignorable. However, in many of the NAEP assessments, non-responses may be related to the outcome variable. For example, schools and students with lower performances may be reluctant to participate in the assessment. In such cases, the non-responses are informative and thus are not ignorable. Using the non-response adjusted HT estimator in such situations would lead to biased estimates.

Project Activities

To address the problem of bias in the NAEP data, the researchers will develop a flexible modeling procedure to take into account multilevel, non-ignorable, non-responses in order to provide an unbiased assessment of American student performance. More specifically, the objectives of this proposal are: (1) to develop hierarchical linear models to incorporate multilevel non-ignorable, non-response mechanisms of the NAEP data into the modeling process; (2) to develop methodology for estimating parameters of interest in the proposed hierarchical linear models; (3) to develop statistical tools for model selection and model adequacy assessment; (4) to provide estimators for population mean and other parameters of interest in the NAEP data based on the selected hierarchical linear model; and (5) to compare the proposed methodology with existing methods.