IES Blog

April is National Bilingual/Multilingual Learner Advocacy Month! As part of the IES 20^th Anniversary celebration, we are highlighting NCER’s investments in field-initiated research. In this guest blog, Drs. Nikki Edgecombe (Community College Research Center at Teachers College, Columbia University) and George Bunch (University of California, Santa Cruz) discuss their IES-funded study focused on identifying the policies and practices that support multilingual learners (MLs) in community colleges, the important role of adult education English as a Second Language (adult ed ESL), and some of the lessons learned. MLs in community colleges, a significantly understudied population, include students who were classified as English learners in K-12 but also more recent arrivals to the US with a wide variety of education backgrounds and adult immigrants and refugees who have lived in the US for a number of years.  

Vital Role of Adult Education ESL

The demands on and opportunities for adult education programs, specifically adult ed ESL, are growing. The programs are affordable and accessible to immigrant communities and people with lower levels of academic preparation—many of whom were hit hard by the pandemic and disproportionately experienced negative education, health, and economic consequences. Simultaneously, the pandemic highlighted the increasing, critical need for multilingual workers across fields, including education, healthcare, and government. Adult ed ESL programs can help bring those workers into the labor market.

According to the National Reporting System for Adult Education, in program year 2021-22, nationally, free or low-cost adult education programs enrolled about 900,000 students, with nearly 50 percent in English language acquisition or integrated English literacy and civics education programs. Sixteen states house adult education programs in community and technical colleges, often in addition to other community-based and educational settings (see this site for information about state grants). These programs provide a unique opportunity for students to improve “everyday” English skills and serve as a ready-made pathway to a postsecondary credential. Yet, the programs and their students face a number of challenges. 

Asking the Questions Practitioners Want Answers To

For the last 4 years, we examined policies and practices that affect the experiences and outcomes of MLs in a large midwestern community college district. The research has focused on adult ed ESL, in part at the recommendation of our district partners. The district serves thousands of ESL students annually, and institutional leaders have actively pursued improvements to program access, instruction, and progression. As we learned more about the improvement efforts underway, we were able to gain a clearer picture of the stringent federal and state policies adult education operates under, which have at times challenged district leaders' ability to make the kinds of changes necessary to enhance student outcomes. Our institutional partners are not deterred, however, and continue to seek ways to strengthen their adult education program, make it more student-centered, and make policy more effective. They consider research an important resource in this improvement process. As such, we both documented their efforts and examined how the policy context affected what they were doing.

What We Are Finding

We wanted to better understand who the MLs are, their life circumstances, their college experiences, and their goals. We reported the following trends to our community college partners.  

• Adult ed ESL students are older and less likely to be working than their peers in credit programs. As a group, they are more likely to either have not earned a high school diploma or GED or have previously earned a baccalaureate degree or higher. Adult ed ESL students in our survey sample report enrolling in adult ed ESL to improve their everyday English literacy skills, to strengthen their employment prospects and prepare themselves for the language and literacy demands of further postsecondary education.
• The multiple ESL levels required by policy may generate obstacles to progression, particularly for students who initially place in the lower levels of the sequence. MLs in our partner district place into 1 of 6 adult ed ESL courses. As research has previously established, community college students rarely persist through long sequences of courses, and our preliminary administrative data analysis shows students in our sample generally persist for less than 2 semesters. In response, our district partner developed a full-time position to help students in the transitions into, through, and out of adult education; has offered short (4- and 8-week) courses; built out dedicated academic and nonacademic supports; and created an intentional on-ramp to credit programs. Nonetheless, the length of the sequence appears to undermine retention and progression.
• The prescribed assessment and placement procedures make it difficult for community college-based adult ed ESL programs to meet the varied English language learning needs of enrollees. Language learning is a complex phenomenon that requires students to develop a range of productive (speaking and writing) and receptive (listening and reading) literacy skills for a wide range of academic, professional, and community participation goals. That learning can look quite different for different students in different education environments and proficiency measurement is equally complex. Our district partner used one of the federally mandated assessments to both place students and measure their proficiency gains. The test is relatively inexpensive and easy to administer, but it only measures reading, just one aspect of language proficiency, leaving no consistent record of proficiency levels in speaking, listening, and writing.

New Directions in Adult Ed ESL Policy Research

Early findings from the study have the potential to inform changes in policy and practice at our district partner. Our findings also raise issues that federal and state policymakers and practitioners working on the ground may need to work together to answer, including how policy systems can balance the perceived need for standards and accountability with community colleges’ need to structure and administer the programs in ways that best meet the needs of MLs.

To pursue this line of inquiry, we will explore the origin and rationale for adult ed ESL policy and how that policy translates from federal to state to institutional providers. We will also learn more about the goals and experiences of adult ed ESL students coming out of the pandemic, explore the perceptions and experiences of adult educators and program staff, and provide formative feedback on the reform efforts underway at our district partner with a particular focus on whether and how policy is helping or hindering their ability to meet their goals.

This blog was produced by Dr. Meredith Larson (Meredith.Larson@ed.gov), research analyst and program officer at NCER.

April is National Bilingual/Multilingual Learner Advocacy Month! In this guest blog, Dr. Ryan Williams, principal researcher at the American Institutes for Research, describes his IES-funded project focused on identifying factors that help explain variation in the effects programs have on English learner student outcomes using a broad systematic review and meta-analysis.  

Over the past two decades, empirical research on programs that support English language and multilingual learners has surged. Many of the programs that researchers have studied are designed to support English literacy development and are tailored to the unique needs of English learners. Other programs are more general, but researchers often study program impacts on English learners in addition to impacts on a broader population of students. Relatively few attempts have been made to identify common findings across this literature. Even fewer attempts have been made to identify meaningful sources of variation that drive program impacts for English learner students—that is, understanding what works, for whom, and under what conditions. To help provide educators and policymakers answers to those important questions, we conducted a systematic review and meta-analysis of the effectiveness of programs and strategies that may support English language learner students.

Our Systematic Review Process

We conducted a broad search that combed through electronic databases, unpublished ‘grey’ literature (for example, working papers, conference presentations, or research briefs), and sources that required hand-searching such as organizational websites. After documenting our primary decision-making factors within a review protocol, we applied a set of rigorous criteria to select studies for inclusion in the meta-analysis. We ultimately identified 83 studies that met our inclusion criteria. Each of these were randomized field studies that included English learner students in grades PK-12 and student academic learning outcomes such as English literacy, mathematics, science, and social studies. Each of the included studies was systematically coded to capture characteristics about the research methods, students and schools, settings, programs, outcome measures, and importantly, the program impacts that the studies reported. We then conducted a meta-analysis to understand the relationships between the characteristics we coded and the program impacts.

Preliminary Findings

We are still working on finalizing our analyses; however, our initial analyses revealed several interesting findings.

• Programs that included support for students to develop their first language skills tended to have larger improvements in student learning. This is consistent with prior research that suggests that supporting first language development can lead to improved learning in core content areas. However, the initial findings from this meta-analysis build on the prior research by providing empirical evidence across a large number of rigorous studies.
• There are some particularly promising practices for educators serving English learner students. These promising practices include the use of content differentiation, the use of translation in a student’s first language, and a focus on writing. Content differentiation aligns with best practices for teaching English learners, which emphasize the importance of providing instruction that is tailored to language proficiency levels and academic needs. The use of first language translation can be helpful for English learner students, as it can support their ability to access and comprehend academic content while they are still building their English proficiency. Focusing on writing can also be particularly important for English learners, as writing is often the last domain of language proficiency for students to develop. Our preliminary findings that English learner writing skills are responsive when targeted by instructional programs may hold implications for how to focus support for students who are nearing but not yet reaching English proficiency.
• The type of test used to measure program impact was related to the size of the program impact on student learning that studies found. Specifically, we found that it is reasonable to expect smaller program impacts when examining state standardized tests and larger impacts for other types of tests. This is consistent with findings from prior meta-analyses based on more general student populations, and it demonstrates the same applies when studying program impacts for English learner students. Statewide standardized tests are typically designed to cover a broad range of state content standards and thus may not reflect improvements in more specific areas of student learning targeted by a given program. On the other hand, researcher-developed tests may align too closely with a program and may not reflect broader, policy-relevant, changes in learning. Our initial evidence suggests that to understand program impacts for English learner students—or any group of students—we may want to use established, validated assessments but not only consider statewide standardized tests.

Next Steps

In terms of next steps, we will complete the meta-analysis work this summer and focus on disseminating the findings through multiple avenues, including a journal publication, review summaries on the AIR website, and future conference proceedings. In addition, we are working to deepen our understanding of the relationships identified in this study and explore promising avenues for practice and future research.

If you’d like to continue learning and see the results of this study, please continue to check back at AIR’s Methods of Synthesis and Integration Center project page, located here.

This blog was produced by Helyn Kim (Helyn.Kim@ed.gov), program officer for the English Learners portfolio, NCER.

Young women and men with disabilities face unique barriers in the transition from school to adulthood. In recognition of the IES 20^th anniversary, we are spotlighting Paths 2 the Future, a career development intervention for students with disabilities with gender-specific versions for boys and girls. For this blog post, virtual intern Audrey Im checks in with IES grantees Dr. Lauren Lindstrom (University of California, Davis) and Dr. John Lind (University of Oregon) about their experiences iteratively developing Paths 2 the Future. What started as an intervention to provide career guidance to high school girls with disabilities has now expanded to a package of interventions that also address the needs of high school-aged men with disabilities and underserved students of all genders.

In 2007, Lauren Lindstrom (then a senior research associate at the University of Oregon) received a grant from the National Center for Special Education Research (NCSER) to develop PATHS, a curriculum to improve education and career outcomes of high school girls with learning disabilities, ADD/ADHD, and emotional or behavioral disabilities. Lindstrom and her team created a curriculum advancing gender equity, disability awareness, and career readiness, which was then implemented in six high schools as an 18-week program.

According to Dr. Lindstrom, her team created PATHS just for girls after examining the disparate post-school outcomes for high school girls with disabilities enrolled in existing transition programs. “I consistently noticed that the girls were less likely to go to work, and if they went to work, they were working in really low-wage jobs,” she said. “And this was with the benefit of an intervention, right? Same kind of disabilities, same schools, but very different outcomes. So that really sparked my interest.”

In 2015, Dr. Lindstrom received a second grant to conduct a randomized controlled trial to test whether the intervention, now called Paths 2 the Future (P2F), improved career knowledge and skills among participants.

“We realized that this was probably one of the very first randomized controlled trials of a gender-specific career intervention,” Lindstrom said.

Lindstrom and her team sampled 366 girls with high-incidence disabilities in 26 Oregon high schools. The girls randomly assigned to the P2F intervention received the curriculum’s four core modules on self-awareness, disability knowledge, gender identity, and career and college readiness. They also received extensive information on career-related activities. The girls in the control group received the existing transition services of their respective schools. This study period lasted one 18-week semester and included a 6-month follow up with the students.

The P2F study found that the girls in the treatment group not only had more awareness of their identity and career possibilities after completing the curriculum, but they also had more confidence to talk about those topics. “The nature of being in a girl-only class really mattered,” Lindstrom said. “The students told us they felt safe there. They said things like ‘I’m a different person now. I feel empowered to talk, to think differently about my future.’” Lindstrom’s study also found that students in the treatment group were more likely to seek and have work experience in high school, an important observation as early work experience has proven to be a predictor for their future employment.

Lindstrom’s co-PI and research collaborator, Dr. John Lind, wondered if the P2F model would also work for boys. Lind, a research associate at the University of Oregon, received a 2019 IES grant Paths to the Future for Young Men (P2F-Young Men) to modify the P2F curriculum to take into account the specific needs of high school boys with high-incidence disabilities. These needs included (but were not limited to) building healthy relationships, breaking down gender stereotypes, and managing anger and stress.

“I think these needs are applicable to a range of genders but doing it in a classroom with just young men opens up the opportunity for potentially deeper discussions,” Lind said. “And that’s feedback that we’ve gotten anecdotally from the teachers we work with.”

After fully developing the P2F-Young Men curriculum, the researchers are currently conducting a small randomized controlled trial with eight teachers and their students at Oregon high schools. Although they are still in the process of collecting data for this study, Lind noted that teachers report that having a gender-specific curriculum helped the boys feel more comfortable in having discussions. “This is anecdotal at this point,” Lind acknowledged, “but if that stands true by the end of our study, I think that’s a really important finding.”

To Lindstrom and Lind, having separate curriculum interventions for different genders was necessary to address gender-specific issues and foster a safe learning environment. At the same time, they felt that it was important for all students across the gender spectrum to have access to these curricula to promote social-emotional development and build knowledge of career pathways.

“Teachers and schools have come to us and said, well is it just for cisgender students or people who are born as a certain gender? And our answer to that is no,” Lind affirmed. “What we’ve done with P2F-Young Men is create a transition curriculum for people who identify as young men. We start early in the curriculum of getting to know yourself, exploring yourself, your strength.”

In 2017, through funding from the National Center for Education Research (NCER), Lindstrom and Lind also developed a non-gender-specific version of the curriculum called P2F for All. This curriculum was targeted to underserved youth who face barriers to educational attainment and, due to a variety of reasons, may not be receiving transition services or college and career readiness support. Their study developed and tested the new P2F for All curriculum and found that it increased participating students’ career readiness, emotional coping skills, and interpersonal skills.

P2F for All aimed to take the findings from their gender-specific studies focused on the needs of students with disabilities and create a new, comprehensive career readiness curriculum—one that succeeded at addressing the needs of underserved students, not just those identified for special education services, regardless of gender. “What we strive to do in special education is provide services that are individualized and meet the needs of the person,” Lind said. “I think we’ve got a range of lessons to address that, and, ultimately, I think that lessons could be pulled out of a menu to meet specific needs for all students.”

Lauren Lindstrom is a professor and dean of the School of Education at the University of California, Davis. Prior to UC-Davis, Dr. Lindstrom served more than 25 years as an academic and administrator at the University of Oregon’s College of Education. Dean Lindstrom is an active researcher whose areas of interest include inclusive education, gender equity, career and college readiness and transition services for youth with disabilities. 

John Lind is a research associate at the University of Oregon’s College of Education. As a former special education teacher, Dr. Lind has extensive experience developing and implementing strength-based interventions for youth with disabilities, including adolescents with emotional and behavior disabilities. He has also worked as an educational consultant, providing training and technical assistance to international, national, and state departments of education on issues related to IDEA, effective classroom management, multi-tiered levels of support, and inclusion. Currently, he is the director of the SIGnetwork, a clearinghouse of resources for the OSEP-funded State Personnel Development Grantees.

This blog was written by Virtual Student Federal Service Intern Audrey Im and produced by Katina Stapleton (Katina.Stapleton@ed.gov). Akilah Nelson (Akilah.Nelson@ed.gov) is the program officer for the IES Transition to Postsecondary Education, Career, and/or Independent Living portfolio. The blog is part of a larger series on DEIA in Education Research.

International and national assessment data show that many U.S. students struggle with mathematics, and there continues to be a gap between students with and without disabilities. The recent 2022 NAEP mathematics results continue to showcase these disparities, which have been further exacerbated as a result of the COVID-19 pandemic, particularly for lower-performing students and students of color.

In honor of Mathematics and Statistics Awareness Month, we want to highlight the research IES is supporting to improve mathematics achievement and access to educational opportunities for all learners, especially learners who have been historically underserved and underrepresented in STEM education.

IES is supporting research through its discretionary grant competitions to measure, explore, develop, and evaluate effective mathematics programs, practices, and policies for all students, including those with or at risk for disabilities. Here are a few highlights of some new research supported by IES:

• Interleaved Mathematics Practice – Bryan Matlen (WestEd) and colleagues are conducting a systematic replication of a highly promising mathematics learning intervention, interleaved practice, in 7th grade classrooms. With the interleaved practice intervention, some of the assigned math practice problems are rearranged so that problems of different kinds are mixed together, which improves learning, and problems of the same kind are distributed across multiple assignments, which improves retention. Numerous studies in the laboratory and classroom have demonstrated that merely rearranging practice problems so that the students receive a higher dose of interleaved practice can dramatically boost scores on measures of learning. This replication study will determine whether this promising intervention can improve math learning and achievement and whether the intervention can scale to a widely-used online intervention that currently reaches tens of thousands of students in diverse settings.
• Educational Technology Approaches to K-12 Mathematics – Jennifer Morrison (Johns Hopkins University) and colleagues are conducting a meta-analysis of rigorous evaluations of approaches that use technology to improve student mathematics achievement in grades K to 12. Using meta-analytic techniques, the team will be identifying conditions under which various types of technology applications are most effective in teaching mathematics. The results will provide researchers and education leaders with up-to-date information on effective uses of technology, including computer assisted instruction, cooperative learning, intelligent tutoring systems, games, simulations, virtual reality, inquiry/discovery, project-based learning, and media-infused instruction.
• Specialized Intervention to Reach All Learners - Sarah Powell (University of Texas at Austin) and colleagues are conducting an initial efficacy evaluation of Math SPIRAL, an educator-provided mathematics intervention for students identified as needing intervention services through state achievement testing in grades four and five. Educators are provided with an evidence-based word problem intervention (Pirate Math Equation Quest), associated professional development, and coaching to support implementation and address the needs of their learners who are struggling in math. The research team will evaluate the impact of Math SPIRAL on mathematics outcomes for upper elementary students identified as being with or at risk for a disability. The results will provide information on the efficacy of Math SPIRAL as a tool to accelerate the learning of students in need of math intervention.
• Math and Reading Acquisition Co-Adaptive System – Jess Gropen (Center for Applied Special Technology), Steve Ritter (Carnegie Learning), and their research team are iteratively developing and studying a set of individualized reading supports for students embedded within an adaptive mathematics learning system (MATHia) and an associated teacher application (LiveLab). Heuristics will determine when reading supports or scaffolds should be provided or recommended to students. In addition, adaptive supports for teachers will alert them when students are likely exhibiting reading challenges and provide recommendations for intervention. The findings will determine whether these reading supports that can be embedded into a variety of digital and/or adaptive math tools to decrease reading challenges and increase students' ability to engage effectively with math. The findings and generated technical resources (such as design assets and heuristics) will be Creative Commons licensed and made available through GitHub for use by other developers.

In August 2022, IES also launched the Learning Acceleration Challenge (LAC) Math Prize to identify and award school-based, digital interventions that significantly improve math outcomes for upper elementary school students with or at risk for a disability that affects math performance. Interventions for the Math Prize needed to specifically focus on fractions and could also include prerequisite skills such as whole numbers and operations. Two interventions are currently competing for the math prize and the winner will be announced Fall 2023.

In addition, IES has developed Practice Guides with evidence-based recommendations for educators to address challenges in their classrooms and schools. A list of the mathematics focused Practice Guides can be found here.

This blog was written by Christina Chhin (christina.chhin@ed.gov), NCER; Sarah Brasiel (sarah.brasiel@ed.gov), NCSER; and Britta Bresina (britta.bresina@ed.gov), NCSER.

When it comes to using research to inform practice, teachers tend to want succinct tips and strategies that can work in their own classrooms. Researchers can use social media channels to tailor their messages from their research findings and disseminate where teachers are already active. In this guest blog, Dr. Sam Van Horne, University of Delaware, describes the work that researchers conducted as part of the originally IES-funded Center for Research Use in Education. The goal of the center was to understand the gaps between researcher and practitioner perspectives on the usefulness of research in practice so that the center can address issues around how researcher communicate about their research, how practitioners can use research more effectively in their classrooms, and how to build stronger connections between the two communities.

Using a large cross-sectional survey of school-based practitioners, we found that practitioners report consuming research through multiple channels, and more than half of reported using social media in the last year with the goal of improving their practice. Social media channels, therefore, provide education researchers with an opportunity to connect with practitioners, but where are researchers likely to find teachers on social media? And how can researchers distill their findings for sharing in mediums that are vastly different than traditional academic forms? Here are some recommendations based on our research.

• Finding and Connecting with Educators on Social Media: One review of research about social media use among teachers found that Facebook and Twitter are some of the main sites that teachers use. But teachers also use Pinterest and Instagram as methods for learning from other teachers about teaching strategies. Posting in multiple channels may make it more likely that a message can reach educators. To find educators, researchers can search for public lists on education-focused topics or see who is using hashtags like #edtwitter, #TeachersofInstagram, or #EduTooters. By following lists, researchers can efficiently find educators to follow and tag (i.e., add the educator’s username to a social media message) with their messages about research-informed practice. This can aid with directly engaging practitioners and beginning conversations about applying research to practice.
• Using Hashtags or Tagging Specific People on Social Media: Social media networks like Twitter can overwhelm users with the volume of content being shared, so it’s critical to use tools like hashtags to find a practice-focused community who may be interested in applying research findings. Users search for content with hashtags that are discipline specific or likely to reach educators, such as #edutooters on Mastodon, #edutwitter on Twitter, or #teachersofinstagram or #teachersfollowteachers on Instagram. The key is identifying teachers or knowledge brokers (i.e., people or organizations who support practitioners in applying research evidence to teaching practice) that may be interested in the message and who may retweet or boost the message to their own followers.
• Tailoring Messages to Focus on What Practitioners Can Do: When the audience is identified, researchers can ask themselves, “What do I want this group to consider doing based on these research findings?” Then, social media messages can incorporate those ideas rather than just summarizing research findings. Social media messages describing how research can inform education practice should be economical and capture interest. Links to the original paper can be appended to a post for those who want to read more.
•  When possible, include links to publications or resources in publicly available repositories and not to versions in subscription-based journals. IES grantees can increase the visibility of their research by submitting their publications as grantee submissions in ERIC. This not only fulfills public-access requirements but also gives practitioners access to important information for improving teaching practice.  
• Incorporating Visual Elements to Attract Attention to Education Research Findings: Messages that incorporate visual elements or video are better suited for sharing on social media. The visual abstract is a succinct summary of research findings that is well-suited for sharing in social media platforms, and researchers have found that visual abstracts are more often shared on social media platforms than plain text about research. You can find guidance on creating visual abstracts here, though the authors suggest collaborating with a designer. These visual abstracts are suited for visual platforms like Pinterest or Instagram. Some journals make a regular practice of posting brief video messages from authors who explain their research study and the significance of the findings. Animations can also attract more attention to messages about research.

Disseminating education research on social media is not a “one-and-done” practice but should be part of a professional social media presence. Many guides exist for developing a professional social media presence, such as these for Twitter and LinkedIn. In addition to posting about research and its implications for practice, researchers can post about research or general issues in the field. This helps with building a following that will be more likely to see posts about research. There are other benefits to disseminating research on social media channels, including providing researchers with metrics about how many times their messages are shared or retweeted (or boosted, on Mastodon), as well as enabling research about optimal ways to share research to reach the broadest audience. In fact, Dr. Farley-Ripple, a leader of CRUE, and colleagues have received funding from the National Science Foundation for a research study to investigate the effectiveness of different dissemination strategies on social media, including the effectiveness of the translational visual abstract.

Connecting with educators on social media is a process. Researchers can begin by creating a presence on social media networks where educators are found and then post often about education and use hashtags to help make messages visible to educators. Messages can be succinct posts that include recommendations or strategies that are feasible for educators to adopt or include multimedia messages like the translational visual abstract to attract attention in a medium that is suited to visuals. Over time, it’s possible to learn what works and what doesn’t and adapt strategies for reaching educators, while keeping in mind that the tools and networks available now will undoubtedly adapt and change themselves.

Sam Van Horne is a Data Scientist at the Center for Research Use in Education and the Center for Research in Education and Social Policy at the University of Delaware.

This blog was produced by Corinne Alfeld (Corinne.Alfeld@ed.gov), program officer, NCER.