Co-Principal Investigators: Umansky, Ilana; Tanner-Smith, Emily

Purpose: The purpose of this project was to conduct a systematic review and meta-analysis of English learner (EL) reclassification studies. Reclassification refers to the determination that an EL student is English proficient and is ready to exit EL classification. The goal of this project was to advance the field's understanding of two critical topics around EL reclassification: (1) timing to reclassification and (2) effects of reclassification on student learning outcomes. By synthesizing and summarizing findings about these outcomes—including moderators, patterns, similarities, and differences across settings—the researchers sought to provide a summary of what we, as a field, know so far about how to study, evaluate, design, and revise reclassification standards for EL students.

Project Activities: The researchers carried out a literature search, a systematic review, and a meta-analysis to address the two issues of timing to reclassification and effects of reclassification on student learning outcomes.

Key Outcomes: Key outcomes will be included once they are published and publicly available in ERIC.

Pre-registration Site: https://osf.io/pfvxe

Structured Abstract

Setting: This systematic review and meta-analysis included studies of EL reclassification in the United States that include data from 2002 or later.

Sample: The meta-analytic sample included studies that reported on students identified as kindergarten through grade 12 ELs in the United States and reported 1 of the following outcomes: (1) time to reclassification, (2) probability of reclassification, (3) academic achievement (as measured by standardized achievement tests), or (4) academic attainment (as measured by graduation).

Factors: The central malleable factor in this review was reclassification, which is the determination that an EL student is English proficient and is ready to exit EL classification. The two research questions focused on reclassification as an outcome of receiving EL services and as an input (intervention) that structures reclassified students' access to certain kinds of instructional environments and opportunities. Reclassification is, in turn, governed by a set of malleable factors that researchers studied as moderators using meta-regression. These factors include (a) the instruments used to identify ELs and annually assess their language proficiency; (b) the instructional programs and practices used to teach ELs language and content; (c) the standards, processes, and decision rules that determine when a student is ready to be reclassified; and (d) the policies that govern which courses ELs and non-ELs may or must take.

Research Design and Methods: The researchers carried out a literature search, a systematic review, and a meta-analysis. For the literature review, they systematically and exhaustively searched both academic and grey literature, such as dissertations and working papers, using a pre-specified set of search terms and procedures. They then screened titles and abstracts to identify potentially eligible studies and applied explicit inclusion criteria to the full text of potentially eligible studies to come up with a final list of studies eligible for inclusion in their systematic review. For the second research question on the effects of reclassification, the researchers restricted their sample to studies that used regression discontinuity or rigorous quasi-experimental designs to directly estimate or infer the effects of reclassification. For the systematic review, they extracted information about student, setting/policy, and design characteristics, as well as outcomes and effect sizes, to create an analytic dataset for the meta-analysis. For the meta-analysis, they used the data analytic strategy described below to estimate main effects and treatment effect heterogeneity and moderators for each outcome. At all stages, at least two independent reviewers viewed all studies and recorded their judgments using researcher-developed protocols and tools.

Control Condition: In the research question that treated reclassification as an outcome, there was no control condition. For the research question that treated reclassification as an input, the control condition was remaining classified as an EL.

Key Measures: For the first research question, focal outcomes were time to reclassification, measured using a median or mean effect size that represented the typical time to reclassification in the study sample, or probability of reclassifying, which the researchers measured using a binary proportion effect size metric. For the second research question, focal outcomes were academic achievement and attainment. The researchers used the Hedges' g small sample-adjusted standardized mean difference effect size to measure group differences on achievement, and the log odds ratio effect size metric to measure group differences in graduation rates (attainment). They built separate models for each research question and outcome.

Data Analytic Strategy: The researchers used both sample-weighted and inverse-variance weighted meta-regression modelling approaches (depending on the effect size) to estimate average effects, heterogeneity in effects, and the associations between various setting/study characteristics and effect size magnitude. They used multilevel linear/logistic meta-regression models with up to three levels to account for clustering, with participants (level 1) nested within effect sizes (level 2) nested within studies (level 3). They estimated these meta-regression models using robust variance estimation and used likelihood ratio tests to assess the significance of model coefficients. They reported confidence intervals and prediction intervals around all parameter estimates of interest.

Related IES Projects: Study of the Impact of English Learner Classification and Reclassification Policies

Products and Publications

ERIC Citations: Find available citations in ERIC for this award here.

Publicly Available Data: https://osf.io/pfvxe