Co-Principal Investigators: Umansky, Ilana; Tanner-Smith, Emily

Purpose: The purpose of this project is to conduct a systematic review and meta-analysis of English learner (EL) reclassification studies. Reclassification refers to the determination that an EL student is English proficient and is ready to exit EL classification. The goal of this project is to advance the field's understanding of two critical topics around EL reclassification: (1) timing to reclassification and (2) effects of reclassification on student learning outcomes. By synthesizing and summarizing findings about these outcomes—including moderators, patterns, similarities, and differences across settings—the researchers will provide a summary of what we, as a field, know so far about how to study, evaluate, design, and revise reclassification standards for EL students.

Project Activities: Researchers will carry out a literature search, a systematic review, and a meta-analysis to address the two issues of timing to reclassification and effects of reclassification on student learning outcomes.

Products: The researchers will report the findings from this study in conference findings and peer-reviewed publications.

Structured Abstract

Setting: This systematic review and meta-analysis will include studies of EL reclassification in the United States that include data from 2002 or later.

Sample: The meta-analytic sample will include studies that report on students identified as kindergarten through grade 12 ELs in the United States and report one of the following outcomes: (1) time to reclassification, (2) probability of reclassification, (3) academic achievement (as measured by standardized achievement tests) or (4) academic attainment (as measured by graduation).

Malleable Factors: The central malleable factor in this review is reclassification. The two research questions focus on reclassification as an outcome of receiving EL services and as an input (intervention) that structures reclassified students' access to certain kinds of instructional environments and opportunities. Reclassification is, in turn, governed by a set of malleable factors that researchers will study as moderators using meta-regression: the instruments used to identify ELs and annually assess their language proficiency; the instructional programs and practices used to teach ELs language and content; the standards, processes, and decision rules that determine when a student is ready to be reclassified; and the policies that govern which courses ELs and non-ELs may or must take.

Research Design and Methods: Researchers will carry out a literature search, a systematic review, and a meta-analysis. For the literature review, they will systematically and exhaustively search both academic and grey literature using a pre-specified set of search terms and procedures. They will then screen titles and abstracts to identify potentially eligible studies and apply explicit inclusion criteria to the full text of potentially eligible studies to come up with a final list of studies eligible for inclusion in our systematic review. For the second research question on the effects of reclassification, researchers will restrict their sample to regression discontinuity or rigorous quasi-experimental designs that allow them to directly estimate or infer the effects of reclassification. For the systematic review, they will extract information about student, setting/policy, and design characteristics, as well as outcomes and effect sizes, to create an analytic dataset for the meta-analysis. For the meta-analysis, they will use the data analytic strategy described below to estimate main effects and treatment effect heterogeneity and moderators for each outcome. At all stages, at least two independent reviewers will view all studies and record their judgments using researcher-developed protocols and tools.

Control Condition: In the research question that treats reclassification as an outcome, there is no control condition. For the research question that treats reclassification as an input, the control condition is remaining classified as an EL.

Key Measures: For the first research question, focal outcomes are time to reclassification, measured using a median or mean effect size that represents the typical time to reclassification in the study sample, or probability of reclassifying, which the researchers will measure using a binary proportion effect size metric. For the second research question, focal outcomes are academic achievement and attainment. The researchers will use the Hedges' g small sample-adjusted standardized mean difference effect size to measure group differences on achievement, and the log odds ratio effect size metric to measure group differences in graduation rates (attainment). They will build separate models for each research question and outcome.

Data Analytic Strategy: Researchers will use an inverse-variance weighted meta-regression modelling approach to estimate average effects, heterogeneity in effects, and the associations between various setting/study characteristics and effect size magnitude. They plan to use multilevel linear/logistic meta-regression models with three levels to account for clustering, with participants (level 1) nested within effect sizes (level 2) nested within studies (level 3). They will estimate these multilevel meta-regression models using restricted maximum likelihood estimation and will use likelihood ratio tests to assess the significance of model coefficients. They will control familywise error using the Benjamini-Hochberg method and will report confidence intervals and prediction intervals around all parameter estimates of interest.