|Title:||Examining Heterogeneity in English Learner Program Effects With Meta-Analysis|
|Principal Investigator:||Williams, Ryan||Awardee:||American Institutes for Research (AIR)|
|Program:||English Learners [Program Details]|
|Award Period:||2 years (07/01/2020 - 06/30/2022)||Award Amount:||$599,829|
Co-Principal Investigators: Garrett, Rachel; Polanin, Joshua
Purpose: Researchers in this study will conduct a systematic literature review to identify rigorous studies of interventions aimed to improve English learner (EL) student learning outcomes and meta-analyze the studies. Researchers will use the meta-analysis (a) to estimate not only overall impact by substrate of interest (e.g., different grade levels, different target outcomes) but also the variability within those strata and (b) to explore malleable factors that contribute to heterogeneity of intervention effects on EL student learning outcomes.
Project Activities: Researchers will focus on identifying malleable factors that explain variation among effects and studies as they carry out the meta-analysis. Specifically, they will test differences in (a) student, teacher, or school characteristics; (b) program type and program features; (c) outcomes; (d) settings or context; and (e) design and methodological features.
Products: The team will present their results in practitioner- and policymaker-friendly metrics, such as improvement in test scores or grades. They will also create various types of dissemination materials for multiple audiences, including conference presentations and peer-reviewed publications.
Setting: The included studies must evaluate interventions set in school settings (prekindergarten through 12th grade). Programs implemented outside school settings will be excluded.
Study Sample: The included studies must estimate effects specifically for EL students.
Interventions: Included studies will evaluate the efficacy or effectiveness of an EL intervention. The research team defines an EL intervention as a program or practice with the explicit goal of improving EL student learning and achievement outcomes, which may include interventions targeted at a broader student population but that examine effects specifically for EL students as a subgroup.
Research Design and Methods: Researchers will include studies that used a randomized controlled trial, a quasi-experimental design, a regression discontinuity, or a single-case design, as long as two groups have been evaluated.
Control Condition: Studies must include a treatment-as-usual or no-treatment control condition. Studies that use a one-group, pretest-posttest design will be excluded.
Measures: Studies must measure EL students on English language development, literacy, mathematics, science, and social studies learning outcomes.
Data Analytic Strategy: Researchers will use a state-of-the-art robust variance estimation and random-effects meta-analysis model to synthesize the effect sizes.