A Study of the Missing Data Assumptions of the Non-equivalent Groups with Anchor Test Design and Their Implications for Test Equating

With the introduction of the No Child Left Behind Act of 2001, the number of tests taken by U.S. grade school students grew. Many of these tests administer different test forms on different dates so that students who take the test on different dates will not answer exactly the same test questions. To ensure fair scores on high-stakes exams, individual differences in scores taken on exams on different test dates must reflect differences in achievement levels of the students and not differences in test difficulty. To achieve this goal, researchers employed a technique called equating, a statistical procedure to ensure that scores from different test forms can be used interchangeably. Among the different equating designs, the Non-Equivalent groups with Anchor Test (NEAT) design is widely used for equating tests used to help make high-stakes decisions, such as the SAT, Advanced Placement tests, and standardized state assessments. Three popular equating methods that can be used with a NEAT design are: the poststratification equipercentile equating method (also called the frequency estimation method), the chain equipercentile equating method, and the item response theory observed score equating method. Each of these methods makes distinct assumptions about the missing data in the NEAT design. These assumptions are usually neither explicit nor testable under the usual equating situations. This project team examined the missing data assumptions and their implications on equating, and obtained a deeper understanding of the comparative performance of the three abovementioned equating methods.