IES Blog

April is National Bilingual/Multilingual Learner Advocacy Month! In this guest blog, Dr. Ryan Williams, principal researcher at the American Institutes for Research, describes his IES-funded project focused on identifying factors that help explain variation in the effects programs have on English learner student outcomes using a broad systematic review and meta-analysis.  

Over the past two decades, empirical research on programs that support English language and multilingual learners has surged. Many of the programs that researchers have studied are designed to support English literacy development and are tailored to the unique needs of English learners. Other programs are more general, but researchers often study program impacts on English learners in addition to impacts on a broader population of students. Relatively few attempts have been made to identify common findings across this literature. Even fewer attempts have been made to identify meaningful sources of variation that drive program impacts for English learner students—that is, understanding what works, for whom, and under what conditions. To help provide educators and policymakers answers to those important questions, we conducted a systematic review and meta-analysis of the effectiveness of programs and strategies that may support English language learner students.

Our Systematic Review Process

We conducted a broad search that combed through electronic databases, unpublished ‘grey’ literature (for example, working papers, conference presentations, or research briefs), and sources that required hand-searching such as organizational websites. After documenting our primary decision-making factors within a review protocol, we applied a set of rigorous criteria to select studies for inclusion in the meta-analysis. We ultimately identified 83 studies that met our inclusion criteria. Each of these were randomized field studies that included English learner students in grades PK-12 and student academic learning outcomes such as English literacy, mathematics, science, and social studies. Each of the included studies was systematically coded to capture characteristics about the research methods, students and schools, settings, programs, outcome measures, and importantly, the program impacts that the studies reported. We then conducted a meta-analysis to understand the relationships between the characteristics we coded and the program impacts.

Preliminary Findings

We are still working on finalizing our analyses; however, our initial analyses revealed several interesting findings.

• Programs that included support for students to develop their first language skills tended to have larger improvements in student learning. This is consistent with prior research that suggests that supporting first language development can lead to improved learning in core content areas. However, the initial findings from this meta-analysis build on the prior research by providing empirical evidence across a large number of rigorous studies.
• There are some particularly promising practices for educators serving English learner students. These promising practices include the use of content differentiation, the use of translation in a student’s first language, and a focus on writing. Content differentiation aligns with best practices for teaching English learners, which emphasize the importance of providing instruction that is tailored to language proficiency levels and academic needs. The use of first language translation can be helpful for English learner students, as it can support their ability to access and comprehend academic content while they are still building their English proficiency. Focusing on writing can also be particularly important for English learners, as writing is often the last domain of language proficiency for students to develop. Our preliminary findings that English learner writing skills are responsive when targeted by instructional programs may hold implications for how to focus support for students who are nearing but not yet reaching English proficiency.
• The type of test used to measure program impact was related to the size of the program impact on student learning that studies found. Specifically, we found that it is reasonable to expect smaller program impacts when examining state standardized tests and larger impacts for other types of tests. This is consistent with findings from prior meta-analyses based on more general student populations, and it demonstrates the same applies when studying program impacts for English learner students. Statewide standardized tests are typically designed to cover a broad range of state content standards and thus may not reflect improvements in more specific areas of student learning targeted by a given program. On the other hand, researcher-developed tests may align too closely with a program and may not reflect broader, policy-relevant, changes in learning. Our initial evidence suggests that to understand program impacts for English learner students—or any group of students—we may want to use established, validated assessments but not only consider statewide standardized tests.

Next Steps

In terms of next steps, we will complete the meta-analysis work this summer and focus on disseminating the findings through multiple avenues, including a journal publication, review summaries on the AIR website, and future conference proceedings. In addition, we are working to deepen our understanding of the relationships identified in this study and explore promising avenues for practice and future research.

If you’d like to continue learning and see the results of this study, please continue to check back at AIR’s Methods of Synthesis and Integration Center project page, located here.

This blog was produced by Helyn Kim (Helyn.Kim@ed.gov), program officer for the English Learners portfolio, NCER.

Young women and men with disabilities face unique barriers in the transition from school to adulthood. In recognition of the IES 20^th anniversary, we are spotlighting Paths 2 the Future, a career development intervention for students with disabilities with gender-specific versions for boys and girls. For this blog post, virtual intern Audrey Im checks in with IES grantees Dr. Lauren Lindstrom (University of California, Davis) and Dr. John Lind (University of Oregon) about their experiences iteratively developing Paths 2 the Future. What started as an intervention to provide career guidance to high school girls with disabilities has now expanded to a package of interventions that also address the needs of high school-aged men with disabilities and underserved students of all genders.

In 2007, Lauren Lindstrom (then a senior research associate at the University of Oregon) received a grant from the National Center for Special Education Research (NCSER) to develop PATHS, a curriculum to improve education and career outcomes of high school girls with learning disabilities, ADD/ADHD, and emotional or behavioral disabilities. Lindstrom and her team created a curriculum advancing gender equity, disability awareness, and career readiness, which was then implemented in six high schools as an 18-week program.

According to Dr. Lindstrom, her team created PATHS just for girls after examining the disparate post-school outcomes for high school girls with disabilities enrolled in existing transition programs. “I consistently noticed that the girls were less likely to go to work, and if they went to work, they were working in really low-wage jobs,” she said. “And this was with the benefit of an intervention, right? Same kind of disabilities, same schools, but very different outcomes. So that really sparked my interest.”

In 2015, Dr. Lindstrom received a second grant to conduct a randomized controlled trial to test whether the intervention, now called Paths 2 the Future (P2F), improved career knowledge and skills among participants.

“We realized that this was probably one of the very first randomized controlled trials of a gender-specific career intervention,” Lindstrom said.

Lindstrom and her team sampled 366 girls with high-incidence disabilities in 26 Oregon high schools. The girls randomly assigned to the P2F intervention received the curriculum’s four core modules on self-awareness, disability knowledge, gender identity, and career and college readiness. They also received extensive information on career-related activities. The girls in the control group received the existing transition services of their respective schools. This study period lasted one 18-week semester and included a 6-month follow up with the students.

The P2F study found that the girls in the treatment group not only had more awareness of their identity and career possibilities after completing the curriculum, but they also had more confidence to talk about those topics. “The nature of being in a girl-only class really mattered,” Lindstrom said. “The students told us they felt safe there. They said things like ‘I’m a different person now. I feel empowered to talk, to think differently about my future.’” Lindstrom’s study also found that students in the treatment group were more likely to seek and have work experience in high school, an important observation as early work experience has proven to be a predictor for their future employment.

Lindstrom’s co-PI and research collaborator, Dr. John Lind, wondered if the P2F model would also work for boys. Lind, a research associate at the University of Oregon, received a 2019 IES grant Paths to the Future for Young Men (P2F-Young Men) to modify the P2F curriculum to take into account the specific needs of high school boys with high-incidence disabilities. These needs included (but were not limited to) building healthy relationships, breaking down gender stereotypes, and managing anger and stress.

“I think these needs are applicable to a range of genders but doing it in a classroom with just young men opens up the opportunity for potentially deeper discussions,” Lind said. “And that’s feedback that we’ve gotten anecdotally from the teachers we work with.”

After fully developing the P2F-Young Men curriculum, the researchers are currently conducting a small randomized controlled trial with eight teachers and their students at Oregon high schools. Although they are still in the process of collecting data for this study, Lind noted that teachers report that having a gender-specific curriculum helped the boys feel more comfortable in having discussions. “This is anecdotal at this point,” Lind acknowledged, “but if that stands true by the end of our study, I think that’s a really important finding.”

To Lindstrom and Lind, having separate curriculum interventions for different genders was necessary to address gender-specific issues and foster a safe learning environment. At the same time, they felt that it was important for all students across the gender spectrum to have access to these curricula to promote social-emotional development and build knowledge of career pathways.

“Teachers and schools have come to us and said, well is it just for cisgender students or people who are born as a certain gender? And our answer to that is no,” Lind affirmed. “What we’ve done with P2F-Young Men is create a transition curriculum for people who identify as young men. We start early in the curriculum of getting to know yourself, exploring yourself, your strength.”

In 2017, through funding from the National Center for Education Research (NCER), Lindstrom and Lind also developed a non-gender-specific version of the curriculum called P2F for All. This curriculum was targeted to underserved youth who face barriers to educational attainment and, due to a variety of reasons, may not be receiving transition services or college and career readiness support. Their study developed and tested the new P2F for All curriculum and found that it increased participating students’ career readiness, emotional coping skills, and interpersonal skills.

P2F for All aimed to take the findings from their gender-specific studies focused on the needs of students with disabilities and create a new, comprehensive career readiness curriculum—one that succeeded at addressing the needs of underserved students, not just those identified for special education services, regardless of gender. “What we strive to do in special education is provide services that are individualized and meet the needs of the person,” Lind said. “I think we’ve got a range of lessons to address that, and, ultimately, I think that lessons could be pulled out of a menu to meet specific needs for all students.”

Lauren Lindstrom is a professor and dean of the School of Education at the University of California, Davis. Prior to UC-Davis, Dr. Lindstrom served more than 25 years as an academic and administrator at the University of Oregon’s College of Education. Dean Lindstrom is an active researcher whose areas of interest include inclusive education, gender equity, career and college readiness and transition services for youth with disabilities. 

John Lind is a research associate at the University of Oregon’s College of Education. As a former special education teacher, Dr. Lind has extensive experience developing and implementing strength-based interventions for youth with disabilities, including adolescents with emotional and behavior disabilities. He has also worked as an educational consultant, providing training and technical assistance to international, national, and state departments of education on issues related to IDEA, effective classroom management, multi-tiered levels of support, and inclusion. Currently, he is the director of the SIGnetwork, a clearinghouse of resources for the OSEP-funded State Personnel Development Grantees.

This blog was written by Virtual Student Federal Service Intern Audrey Im and produced by Katina Stapleton (Katina.Stapleton@ed.gov). Akilah Nelson (Akilah.Nelson@ed.gov) is the program officer for the IES Transition to Postsecondary Education, Career, and/or Independent Living portfolio. The blog is part of a larger series on DEIA in Education Research.

International and national assessment data show that many U.S. students struggle with mathematics, and there continues to be a gap between students with and without disabilities. The recent 2022 NAEP mathematics results continue to showcase these disparities, which have been further exacerbated as a result of the COVID-19 pandemic, particularly for lower-performing students and students of color.

In honor of Mathematics and Statistics Awareness Month, we want to highlight the research IES is supporting to improve mathematics achievement and access to educational opportunities for all learners, especially learners who have been historically underserved and underrepresented in STEM education.

IES is supporting research through its discretionary grant competitions to measure, explore, develop, and evaluate effective mathematics programs, practices, and policies for all students, including those with or at risk for disabilities. Here are a few highlights of some new research supported by IES:

• Interleaved Mathematics Practice – Bryan Matlen (WestEd) and colleagues are conducting a systematic replication of a highly promising mathematics learning intervention, interleaved practice, in 7th grade classrooms. With the interleaved practice intervention, some of the assigned math practice problems are rearranged so that problems of different kinds are mixed together, which improves learning, and problems of the same kind are distributed across multiple assignments, which improves retention. Numerous studies in the laboratory and classroom have demonstrated that merely rearranging practice problems so that the students receive a higher dose of interleaved practice can dramatically boost scores on measures of learning. This replication study will determine whether this promising intervention can improve math learning and achievement and whether the intervention can scale to a widely-used online intervention that currently reaches tens of thousands of students in diverse settings.
• Educational Technology Approaches to K-12 Mathematics – Jennifer Morrison (Johns Hopkins University) and colleagues are conducting a meta-analysis of rigorous evaluations of approaches that use technology to improve student mathematics achievement in grades K to 12. Using meta-analytic techniques, the team will be identifying conditions under which various types of technology applications are most effective in teaching mathematics. The results will provide researchers and education leaders with up-to-date information on effective uses of technology, including computer assisted instruction, cooperative learning, intelligent tutoring systems, games, simulations, virtual reality, inquiry/discovery, project-based learning, and media-infused instruction.
• Specialized Intervention to Reach All Learners - Sarah Powell (University of Texas at Austin) and colleagues are conducting an initial efficacy evaluation of Math SPIRAL, an educator-provided mathematics intervention for students identified as needing intervention services through state achievement testing in grades four and five. Educators are provided with an evidence-based word problem intervention (Pirate Math Equation Quest), associated professional development, and coaching to support implementation and address the needs of their learners who are struggling in math. The research team will evaluate the impact of Math SPIRAL on mathematics outcomes for upper elementary students identified as being with or at risk for a disability. The results will provide information on the efficacy of Math SPIRAL as a tool to accelerate the learning of students in need of math intervention.
• Math and Reading Acquisition Co-Adaptive System – Jess Gropen (Center for Applied Special Technology), Steve Ritter (Carnegie Learning), and their research team are iteratively developing and studying a set of individualized reading supports for students embedded within an adaptive mathematics learning system (MATHia) and an associated teacher application (LiveLab). Heuristics will determine when reading supports or scaffolds should be provided or recommended to students. In addition, adaptive supports for teachers will alert them when students are likely exhibiting reading challenges and provide recommendations for intervention. The findings will determine whether these reading supports that can be embedded into a variety of digital and/or adaptive math tools to decrease reading challenges and increase students' ability to engage effectively with math. The findings and generated technical resources (such as design assets and heuristics) will be Creative Commons licensed and made available through GitHub for use by other developers.

In August 2022, IES also launched the Learning Acceleration Challenge (LAC) Math Prize to identify and award school-based, digital interventions that significantly improve math outcomes for upper elementary school students with or at risk for a disability that affects math performance. Interventions for the Math Prize needed to specifically focus on fractions and could also include prerequisite skills such as whole numbers and operations. Two interventions are currently competing for the math prize and the winner will be announced Fall 2023.

In addition, IES has developed Practice Guides with evidence-based recommendations for educators to address challenges in their classrooms and schools. A list of the mathematics focused Practice Guides can be found here.

This blog was written by Christina Chhin (christina.chhin@ed.gov), NCER; Sarah Brasiel (sarah.brasiel@ed.gov), NCSER; and Britta Bresina (britta.bresina@ed.gov), NCSER.

Whenever we get the chance to share information about the Regional Educational Laboratory (REL) program with the public, we’re often asked, “How are the 2022-2027 RELs different from past REL cycles?” In this blog, we focus on one major new effort that each REL is undertaking: the creation of a toolkit for educators based on one of the What Works Clearinghouse (WWC) Practice Guides. Each REL toolkit will include a set of resources for educators to implement and institutionalize evidence-based recommendations from a WWC Practice Guide. Importantly, each REL is co-developing their resources with educators, school, and district leaders or with postsecondary faculty and staff to ensure the toolkits’ relevance and actionability. Following the toolkit development phase, RELs will partner with educators not involved in developing the toolkits to test the usability of each toolkit and its efficacy in improving student and teacher outcomes. The RELs have current partners for toolkit development and usability testing but are looking for partner schools, districts, and postsecondary institutions in which to test the efficacy of the toolkits. These efficacy-testing partners will be among the first to benefit from the evidence-based toolkits.

Why this investment of REL and partner time and resources? WWC Practice Guides are among IES’ premier resources for translating evidence on effective practice into accessible and usable strategies for educators. Each Guide is based on a synthesis of the most rigorous research on teaching a particular subject or achieving a particular education goal. Each Guide is also based on the input of a panel of expert practitioners and researchers and includes—

• Key recommendations for educational practice based on a synthesis of rigorous research
• Supporting evidence for each recommendation
• Steps to carry out each recommendation
• Examples of the practices
• Discussions of common implementation challenges and strategies for overcoming those challenges

RELs Emphasize Active Learning to Support Implementation of Evidence-Based Practices

Although WWC Practice Guides are some of IES’ most popular products, we also know that teachers and leaders cannot simply read about a new practice to master it. Instead, they need to engage in active learning by observing the new practice, discussing it, implementing it, receiving feedback on the practice, and continuing to improve. The REL toolkits are designed to support educators in the creation and implementation of a professional learning community (PLC) focused on the evidence-based practices outlined in a WWC Practice Guide. In these PLCs, educators will learn about the Practice Guide recommendations by reading about the practices, discussing them with colleagues, and by developing plans for implementing the practices in their classrooms. Educators will also put those plans into action and then debrief on those implementation experiences. To support this work, the toolkits will include PLC guides, workbooks, self-study guides, and rubrics. Some toolkits will also include videos of teachers effectively implementing the practices.

Each toolkit will also include the following:

• An initial diagnostic and ongoing monitoring instrument for assessing instructional practices against the practices recommended in the WWC Practice Guide
• A tool that enables teachers, teacher leaders, and administrators to assess the extent to which their school, district or postsecondary institution supports the implementation and ongoing monitoring of the evidence-based practice recommendations
• A discussion of implementation steps for institutionalizing supports that help educators, building leaders, and other administrators adopt the evidence-based practices and sustain them over time

Some RELs have already started usability testing of their toolkits. Across 2025 and 2026, nine of our 10 RELs will publish final versions of their toolkits and efficacy studies on their toolkit. Both will be freely available on the REL website.[1] Visit our Newsflash page and sign up to receive newsflashes from the RELs and the IES center that houses the program—the National Center for Education Evaluation and Regional Assistance (NCEE).[2]

Partner with RELs: Help IES Study REL Toolkits

RELs will soon recruit partner schools, districts, and postsecondary institutions in their regions to conduct the toolkit efficacy studies. If you are interested in having your school, district, or institution participate in an efficacy study and benefit from being one of the first users of these toolkits, please email us at Elizabeth.Eisner@ed.gov or Chris.Boccanfuso@ed.gov. The efficacy study for each REL’s toolkit must take place within each REL’s region. Not sure which REL region is yours? Check out the “About the RELs page” on the IES website or the map visualization on our program homepage.

If you have other questions, concerns, or ideas about this work, please reach out to us. We welcome your input so that you can help IES and the RELs make the toolkits as useful and effective as possible.

Liz Eisner, Associate Commissioner for Knowledge Use

Chris Boccanfuso, REL Branch Chief

In honor of Autism Awareness Month, we took a deep look into NCSER-funded research on augmentative and alternative communication (AAC) for students with autism spectrum disorder (ASD) and/or moderate-to-severe intellectual disabilities (ID) who have complex communication needs. Principal investigators Drs. Ganz, Reichle, and Pustejovsky discussed their research on AAC (such as communication board or speech output device), which provides an alternative means of communication for persons who are nonverbal or minimally verbal and ensures they have the opportunity to communicate their wants and needs. This research team’s current IES project examines treatment intensity factors (how often or how long an intervention takes place) related to teaching AAC use. In the interview below, they discuss their current project and how it builds upon their previous research on AAC interventions.

What is the purpose of your current project?

Individuals with ASD and/or ID who have complex communication needs typically require intensive, costly, and individualized educational interventions to develop communication. However, there is little information to guide parents and instructional personnel in selecting the most effective dose and duration of treatment. Similarly, there is a lack of guidance about when and how treatment integrity and strategies for generalization (use in various contexts) and maintenance (sustaining treatment over time) affect treatment outcomes. The purpose of this current project is to examine the effects of various treatment intensity parameters on expressive communication outcomes for students with ASD and/or ID through a meta-analysis. This investigation aims to guide the development of protocols for instructional personnel and parents so they can implement efficient, acceptable, and effective treatment for improving communication for these students.

What motivated you to conduct this research?

Educational interventions to treat ASD can be costly. This can lead to disparities wherein wealthier families can access high-quality services while most Americans cannot. Social services—including educational and healthcare services—are typically underfunded, impeding the provision of quality services for this population. For example, behavioral experts have recommended 25-40 hours per week of intensive, one-on-one educational and behavioral services for young children with ASD. However, there has been limited research aimed at comparing the relative efficacy of interventions based on various factors associated with treatment intensity. Not all individuals will need the same level of treatment intensity, but more research is needed to understand how treatment intensity needs can be differentiated by student characteristics, intervention types, and service context. Interventions that are efficient and tailored to individual student need may allow them to be more accessible to a wider range of families. In addition to studying these factors, this project aims to develop a treatment integrity template that can be used by others in determining appropriate treatment intensity levels for a range of interventions and populations. Such investigations hold promise for significant improvements in intervention efficiency, potentially giving schools ways to effectively serve more individuals.

Our goal is to provide information to family members and practitioners to enable them to better individualize AAC interventions, allowing them to match treatment intensity needs to individual characteristics, precursor skills, background, and consumer preferences and needs. By doing so, we can provide guidance for the allocation of services where needs are greatest.

How does this project build upon your previous research in AAC?

In 2021, we completed a research project that examined AAC interventions using similar meta-analytic methods with the same population as those studied in our current research. In that project, we focused on the ways in which instructional features and contexts are associated with learner performance. We found that AAC interventions are commonly implemented in school, home, and community settings with no significant differences in learner outcomes based on the setting. This tells us that AAC use does not need to be limited to one setting and can include caregivers and family members in this process. Across studies, a wide range of instructional strategies were used to teach AAC use, with behavioral and naturalistic strategies the most common.

Similarly, there was a range of teaching formats used during instruction. We looked at instructor- versus child-led, contrived versus naturalistic, and one-on-one versus group contexts, with structured approaches (one-on-one instruction, instructor-led, and contrived learning opportunities) the most common. However, just as with settings, no significant differences in outcomes were observed across instructional strategies or formats, indicating that there is no “one-size-fits-all” approach to AAC use and it can be individualized to the needs of each learner. The current grant offers a close examination of treatment intensity factors—such as how many sessions of intervention per week, how many minutes per session, and how many communicative opportunities the learner has during each session—and their potential effect on learner performance. Overall, the study asks, “what is the association between AAC dosage and successful learner outcomes?”

What can your research tell us about the relationship between education outcomes and AAC use for students with ASD and ID

We are hopeful that it will provide clarity for successful intervention protocols by specifying aspects of treatment intensity. Factors of treatment intensity and related intervention characteristics we are looking at include dosage rate, duration, form, and frequency; total intervention duration; degree that the treatment is implemented with integrity; and implementation of generalization and maintenance strategies. Additionally, we will explore possible associations between key skills that are important for students with ASD and ID to develop (such as imitation and matching) and choice of treatment intensity parameters.

Communication is the basis for most other learning, including social skills, literacy, and other functional life skills; thus, improving and increasing communication production and comprehension for individuals with ASD and ID who are minimally verbal or nonverbal will build a foundation for further academic and functional progress.

What do families and caregivers need to know about AAC use?

We believe that families should encourage communication in a range of modalities, including aided AAC, but also natural gestures, speech, and speech approximations. Although there is a myth that AAC use discourages speech, research has shown that individuals often learn speech simultaneously with AAC learning. Further, by increasing fluent communication, frustration and challenging behavior are often decreased. Communication in all forms must be targeted across people, settings, and vocabulary to provide minimally verbal and nonverbal individuals with opportunities to learn and use new language.

We hope to provide information to family members and practitioners that better enables them to individualize AAC interventions, allowing them to match treatment intensity needs to individual characteristics, precursor skills, background, and consumer preferences and needs. By doing so, we will be able to target services where needs are greatest and preserve resources for those in most need.

Many thanks to Drs. Ganz, Reichle and Pustejovsky for sharing their work with our readers!

Joe Reichle serves as the PI for this project and is a former Department Chair and current Professor Emeritus in the Department of Communication Disorders at the University of Minnesota.

J. Birdie Ganz is a professor of Special Education at Texas A&M University and serves as current Project Director and co-PI for this project.

James Pustejovsky is an associate professor in the School of Education at the University of Wisconsin-Madison and serves as co-PI for this project.

This blog was written by Shanna Bodenhamer, virtual student federal service intern at IES and graduate student at Texas A&M University. She also serves as a research assistant on this project.