IES Blog

Institute of Education Sciences

Regional Educational Laboratories Develop New Tools for Educators Based on WWC Practice Guides

Whenever we get the chance to share information about the Regional Educational Laboratory (REL) program with the public, we’re often asked, “How are the 2022-2027 RELs different from past REL cycles?” In this blog, we focus on one major new effort that each REL is undertaking: the creation of a toolkit for educators based on one of the What Works Clearinghouse (WWC) Practice Guides. Each REL toolkit will include a set of resources for educators to implement and institutionalize evidence-based recommendations from a WWC Practice Guide. Importantly, each REL is co-developing their resources with educators, school, and district leaders or with postsecondary faculty and staff to ensure the toolkits’ relevance and actionability. Following the toolkit development phase, RELs will partner with educators not involved in developing the toolkits to test the usability of each toolkit and its efficacy in improving student and teacher outcomes. The RELs have current partners for toolkit development and usability testing but are looking for partner schools, districts, and postsecondary institutions in which to test the efficacy of the toolkits. These efficacy-testing partners will be among the first to benefit from the evidence-based toolkits.

Why this investment of REL and partner time and resources? WWC Practice Guides are among IES’ premier resources for translating evidence on effective practice into accessible and usable strategies for educators. Each Guide is based on a synthesis of the most rigorous research on teaching a particular subject or achieving a particular education goal. Each Guide is also based on the input of a panel of expert practitioners and researchers and includes—

  • Key recommendations for educational practice based on a synthesis of rigorous research
  • Supporting evidence for each recommendation
  • Steps to carry out each recommendation
  • Examples of the practices
  • Discussions of common implementation challenges and strategies for overcoming those challenges

WWC Practice Guide Associated with Each REL Toolkit:

REL

Practice Guide

Appalachia

Teaching Math to Young Children

Central

Teaching Strategies for Improving Algebra Knowledge in Middle and School Students

Mid-Atlantic

Teaching Elementary School Students to Be Effective Writers

Midwest

Developing Effective Fractions Instruction for Kindergarten through 8th Grade

Northeast & Islands

Assisting Students Struggling with Math: Intervention in the Elementary Grades

Northwest

Using Technology to Support Postsecondary Student Learning 

Pacific

Teaching Secondary Students to Write Effectively

Southwest

Providing Reading Interventions for Students in Grades 4 – 9

Southeast

Assisting Students Struggling with Reading: Response to Intervention (RtI) and Multi-Tier Intervention in the Primary Grades

West

Improving Reading Comprehension in Kindergarten Through 3rd Grade

RELs Emphasize Active Learning to Support Implementation of Evidence-Based Practices

Although WWC Practice Guides are some of IES’ most popular products, we also know that teachers and leaders cannot simply read about a new practice to master it. Instead, they need to engage in active learning by observing the new practice, discussing it, implementing it, receiving feedback on the practice, and continuing to improve. The REL toolkits are designed to support educators in the creation and implementation of a professional learning community (PLC) focused on the evidence-based practices outlined in a WWC Practice Guide. In these PLCs, educators will learn about the Practice Guide recommendations by reading about the practices, discussing them with colleagues, and by developing plans for implementing the practices in their classrooms. Educators will also put those plans into action and then debrief on those implementation experiences. To support this work, the toolkits will include PLC guides, workbooks, self-study guides, and rubrics. Some toolkits will also include videos of teachers effectively implementing the practices.

Each toolkit will also include the following:

  • An initial diagnostic and ongoing monitoring instrument for assessing instructional practices against the practices recommended in the WWC Practice Guide
  • A tool that enables teachers, teacher leaders, and administrators to assess the extent to which their school, district or postsecondary institution supports the implementation and ongoing monitoring of the evidence-based practice recommendations
  • A discussion of implementation steps for institutionalizing supports that help educators, building leaders, and other administrators adopt the evidence-based practices and sustain them over time

Some RELs have already started usability testing of their toolkits. Across 2025 and 2026, nine of our 10 RELs will publish final versions of their toolkits and efficacy studies on their toolkit. Both will be freely available on the REL website.[1] Visit our Newsflash page and sign up to receive newsflashes from the RELs and the IES center that houses the program—the National Center for Education Evaluation and Regional Assistance (NCEE).[2]

Partner with RELs: Help IES Study REL Toolkits

RELs will soon recruit partner schools, districts, and postsecondary institutions in their regions to conduct the toolkit efficacy studies. If you are interested in having your school, district, or institution participate in an efficacy study and benefit from being one of the first users of these toolkits, please email us at Elizabeth.Eisner@ed.gov or Chris.Boccanfuso@ed.gov. The efficacy study for each REL’s toolkit must take place within each REL’s region. Not sure which REL region is yours? Check out the “About the RELs page” on the IES website or the map visualization on our program homepage.

If you have other questions, concerns, or ideas about this work, please reach out to us. We welcome your input so that you can help IES and the RELs make the toolkits as useful and effective as possible.

Past REL Professional Development Resources based on WWC Practice Guides:

The RELs have a successful track record of creating professional development resources that complement WWC Practice Guides. For example, see:

Professional Learning Community: Improving Mathematical Problem Solving for Students in Grades 4 Through 8 Facilitator’s Guide (REL Southeast).

Professional learning communities facilitator’s guide for the What Works Clearinghouse practice guide: Foundational skills to support reading for understanding in kindergarten through 3rd grade (REL Southeast).

Professional Learning Communities Facilitator's Guide for the What Works Clearinghouse Practice Guide Teaching Academic Content and Literacy to English Learners in Elementary and Middle School (REL Southwest).

The new toolkits will expand the number of WWC Practice Guides for which the RELs develop professional development resources and will also provide instruments for assessing instructional practice and implementing institutional supports. 

Liz Eisner, Associate Commissioner for Knowledge Use

Chris Boccanfuso, REL Branch Chief


[1] REL Southwest’s contract started 11 months after the contracts of the other 9 RELs, so the REL Southwest toolkit will be released in 2027.

[2] You can also sign up for Newsflashes from IES and its other three centers—NCES, NCER, & NCSER.

How State Education Agencies Can Leverage Their Regional Educational Laboratory to Support Students’ Academic, Social, and Mental Health Needs

(A Dear Colleague Letter sent to Chief State School Officers on February 23, 2023.)

Dear Colleague:

As state and local education agencies leaders reflect upon the successes and challenges of the 2022-2023 school year—and the opportunity that summer 2023 presents to further support students’ academic, social, and mental health needs—I am writing today to encourage you to take full advantage of the services offered by your Regional Educational Laboratory (REL).

The REL Program, sponsored by the U.S. Department of Education’s Institute of Education Sciences, supports educators and policymakers at the state and local levels in the use of data and evidence-based practices to improve student outcomes. All REL services are provided free of charge and are designed in partnership with state and local partners to meet their specific needs. Each REL is led by a Director with deep expertise in education policy, practice, and research who can help you navigate how best to leverage REL supports to address your state’s most pressing needs. A list of REL Directors, including their contact information, is attached.

Your REL can support a wide range of state and local initiatives. They include:

  • Analyzing student progress and outcome data (e.g., achievement, chronic absenteeism, graduation rate, English language proficiency) to understand the ongoing effects of the COVID-19 pandemic. Your REL can analyze longitudinal student data provided by the state or district partners to better understand the trajectory of student performance prior to the pandemic, during the pandemic, and today. When disaggregated by student group, school characteristics, or other relevant features, these analyses can support decision-makers in focusing resources, monitoring improvement, and adjusting implementation efforts. RELs Midwest and Mid-Atlantic recently provided similar services for their state and district-level partners.
  • Supporting the identification of existing, or the design of new, evidence-based practices to meet students’ academic, social, and mental health needs. Your REL can support state and local efforts to identify practices that prior evidence suggests can promote learning and development. REL Southeast recently published a review on the effectiveness of early literacy interventions across several domains in response to a request from partners regionwide. When high-quality evidence does not exist, or existing practices are not well-aligned to state or local needs, RELs can support efforts to design and pilot research-based innovations.
  • Coaching state and local education agency staff on the use of data to improve the ongoing implementation of education policies, programs, and practices. Your REL offers coaching and training services for state and local leaders on data-driven approaches to continuous quality improvement. These services are particularly beneficial when a program is relatively new to a state or district and leaders are focused on timely feedback to ensure an evidence-based practice is well-implemented at scale. REL Southwest recently supported the Oklahoma State Department of Education’s (OSDE) rollout of Oklahoma Excel, a data-driven and job-embedded professional development program for educators in participating districts. A 2-part video series provides background on the program, and the supports REL Southwest provided to OSDE staff who administer the program.
  • Evaluating the impact of state or local interventions on important student outcomes. Your REL can support the rigorous evaluation of well-implemented policies, programs, or practices to document those efforts’ impacts on important student outcomes. For example, a 2021 REL Northwest study examined the implementation and impact of full-day kindergarten in Oregon in light of a funding structure shift that incentivized districts to offer the programming. When a rigorous evaluation is not feasible, your REL can advise you on credible, alternative approaches to understanding the outcomes associated with a policy or program.
  • Coaching state or local education agency staff on the use of existing REL tools and resources. Through their work with state and local partners, RELs have developed a wide range of actionable resources designed to support the implementation of evidence-based practices. Your REL can coach state and local education agency staff on how to  customize and use tools developed elsewhere to meet your needs. Examples include REL Appalachia’s Community Math Night Facilitators’ Toolkit and REL Southeast’s Professional Learning Community on Emergent Literacy.
  • Providing expert guidance to senior state or local education agency leaders. Finally, your REL can leverage its network of internal and external experts to offer guidance on data- and evidence-driven approaches to addressing problems of policy and practice. This “Ask-an-Expert” service is available to senior leaders of both state and local education agencies. A recent REL Appalachia “Ask an Expert” response to a Tennessee-based partner shared best practices for administering and using data from Kindergarten readiness screeners.

REL Directors are routinely in contact with senior education agency leadership as part of their on-going work to better understand the kinds of supports that might benefit states in their region. However, if you or senior members of your leadership team have not yet had the opportunity to meet with your REL Director (or have not done so recently), please consider contacting them at your convenience. I am also glad to facilitate that connection at your request.

Finally, I would be remiss if I did not mention the critical relationship between your REL and the Regional Comprehensive Center (RCC) that serves your state. Sponsored by the Department’s Office of Elementary and Secondary Education, RCCs support state education agencies in their efforts to implement evidence-based policy and practice and realize the goals set in their Consolidated State Plans.

If you have any questions about the REL Program, please do not hesitate to contact me or a senior member of my team.

Sincerely,

Matthew Soldner
Commissioner, National Center for Education Evaluation and Regional Assistance
Matthew.Soldner@ed.gov 

Note: This blog reflects slight edits to the letter sent to Chief State School Officers. References to an attached brochure and a contact list for REL Program staff have been removed.

How the 2017-2022 Cohort of RELs Supported the Use of Evidence in Education

Three adults discuss a chart that is displayed on a laptop.

This winter is a special season that comes along once every five years for the Regional Educational Laboratory (REL) program at IES. It’s a winter when the REL team manages the end of five-year REL contracts and oversees the launch of nine new REL contracts.[i]  During this exciting time, we actively reflect on the successes and lessons of the previous contracts—the 2017-2022 REL cohort—and channel those reflections into our work with the 2022-2027 REL cohort. 

As I collaborate with the REL team on the new RELs, I want to share some of the successes of the RELs that finished their work early this year. We expect the new RELs to build on these successes and to engage in new, innovative work that I will discuss in a future blog.

As we look back at the large body of work that the last cohort of RELs produced, I want to share some exciting results. Over three-quarters of participants in REL researcher-practitioner partnerships who responded to the REL Stakeholder Feedback Survey (SFS) reported that they used or were in the process of using the research or data that they learned about through the REL partnerships. On average across the last three years, an additional 17 percent reported that they were making plans to use research or data presented by the REL:

Image of a chart entitled “Responses to REL Partnership Stakeholder Feedback Survey (SFS).” The chart shows that in 2019, 77 percent of 695 respondents reported that they used or were in the process of using the research data they learned through REL partnerships, 19 percent said they were making plans to use the research, and 4 percent said they had no plans to use the research; in 2020, 81 percent of 397 respondents reported that they used or were in the process of using the research data they learned through REL partnerships, 17 percent said they were making plans to use the research, and 2 percent said they had no plans to use the research; and in 2021, 82 percent of 582 respondents reported that they used or were in the process of using the research data they learned through REL partnerships, 15 percent said they were making plans to use the research, and 3 percent said they had no plans to use the research.

While these survey results are promising, I want to provide a more vivid picture of how the RELs partnered with stakeholders to use evidence to improve teaching and learning. Read on to learn how REL work has been integral to education policy and practice across the country.

REL Mid-Atlantic and REL Southeast both engaged in projects that supported efforts to safely educate students during the pandemic:

  • In Pennsylvania, REL Mid-Atlantic helped the Pennsylvania Department of Education (PDE) provide evidence to inform the reopening of schools in the state during the first year of the COVID-19 pandemic. REL Mid-Atlantic worked with PDE to produce an extensive memo that included (1) a rapid review of existing evidence on public-health and educational issues relevant to the reopening of schools, (2) findings from interviews with a cross-section of stakeholders from across Pennsylvania to assess concerns and challenges related to reopening, and (3) agent-based modeling simulations of the potential spread of COVID-19 under alternative approaches to reopening schools.  The two largest school districts in the state—the School District of Philadelphia and the Pittsburgh Public School District—along with at least 25 other school districts and one Catholic archdiocese drew on the findings in the memo to make decisions about whether and how to reopen schools. 
  • Shortly after two of four of REL Southeast's teacher guides were released in early 2020, schools across the country shut down due to the COVID-19 pandemic. The REL realized that the content of the guides—originally created to support teachers in working with families to facilitate their children’s literacy development—would be immediately useful to parents across the county who were suddenly thrust into the role of teacher for their children at home. The content of the guides was based on the What Works Clearinghouse Practice Guide: Foundational Skills to Support Reading for Understanding in Kindergarten Through 3rd Grade.

REL Southeast made all the content, which included videos and activities, available on the REL website so that parents could easily access them and use them to support their children during that difficult time.The landing page for the content—Supporting Your Child's Reading at Home—has been visited nearly 130,000 times since April of 2020. And landing pages for the four guides for teachers—A Kindergarten Teacher's Guide, A First Grade Teacher's Guide, A Second Grade Teacher's Guide and A Third Grade Teacher's Guide—have each been accessed between 1,300 and 7,500 times since their release. 

REL West and REL Midwest both worked with states in their regions to support student health and the need to identify and recruit more teachers.  These topics proved to be particularly  important as a result of the pandemic:

  • Robla Elementary School District (RESD) and several other districts in California’s Central Valley began offering telemedicine services during the 2017/18 school year as part of a broader “whole-child” strategy for improving student health, well-being, and attendance. Telemedicine is the remote evaluation, diagnosis and treatment of patients using telecommunications technology. RESD contracted with and paid Hazel Health, a telemedicine provider that operates virtual health clinics in school settings.  The telemedicine visits were free to students and families and did not require scheduled appointments. To learn more about the implementation of the program and whether it was associated with students staying in school throughout the day, RESD enlisted REL West for assistance.

REL West's study of the telemedicine services found that districtwide, a little over one-quarter of students used the services at least once over two years, with nine percent of students using telemedicine multiple times. Non-communicable physical illnesses/conditions such as stomach aches, headaches, allergies, and asthma were consistently the most common reason for school-based telemedicine visits across the two years of implementation. Ninety-four percent of all telemedicine visits resulted in students returning to class and receiving, on average, three more hours of instruction in the school day. Approximately 39 percent of Black students used telemedicine services compared with 17 percent of Asian students. Due to these findings, the district decided to continue with the program. The telemedicine provider is working to identify possible reasons for the differences in use by different student groups to ensure that all students are comfortable accessing the services.

  • Even before the COVID-19 pandemic, Michigan was experiencing teacher shortages in several subjects and geographic areas. This led Michigan members of the REL Midwest Alliance for Teacher Preparation to partner with the REL to examine nonteaching-certified teachers’ reasons for not teaching and incentives that could motivate them to return to the classroom. The REL Midwest study found that salary and certification/recertification requirements were among the most frequent barriers to teachers entering or remaining in the teaching profession.

As a result, the Michigan Department of Education launched the “Welcome Back Proud Michigan Educator” campaign, which seeks to recruit nonteaching educators into the teacher workforce. The first wave of the campaign, which began in April 2021, recruited educators with expired teaching certificates by reducing—and in some cases eliminating—professional learning requirements for recertification. The second wave, which began in October 2021, recruited teachers who had a valid certificate but were not teaching in public schools. As of January 2022, 218 educators have been recertified or issued a teaching permit, and 27 educators are in the pipeline to reinstate their teaching credentials. Of those with valid certificates, 123 educators started in a teaching position in fall 2021 and an additional 244 educators took a non-teaching assignment, such as day-to-day substitute teaching.

Concerns about the lack of equity in educational opportunities and in disciplinary practices led stakeholders to partner with REL Appalachia and REL Northwest:

  • Throughout the country, students are often encouraged to study Algebra I in middle school so that they can take more advanced math courses in high school and can graduate with a college-ready diploma. Concerned that economically disadvantaged students and English learners might be taking Algebra I later than their peers and earning college preparatory diplomas at lower rates than other students, Virginia’s Department of Education asked REL Appalachia for assistance analyzing the state’s data. The REL researchers found that the Department of Education’s hypotheses were correct. They found that, among all 5th graders rated as “advanced proficient” on the state’s math assessment, economically disadvantaged and English learner students were less likely take Algebra before 9th grade and less likely to earn a college preparatory diploma. As a result of these findings, the Virginia Department of Education asked the REL to work with school districts across the state to analyze data to identify student course-taking patterns and to further examine district-level policies and practices that may be contributing to the inequitable course-taking patterns and outcomes. 
  • REL Northwest undertook several projects with the Equity in School Discipline (ESD) collaborative: a cross-state collaborative of districts, state education agencies, community-based organizations, and institutions of higher education in Oregon and Washington committed to increasing equity in school climate and discipline policies and practices. ESD sought to reduce the use of exclusionary discipline practices and to eliminate disproportionately high rates of exclusion for students who identify as American Indian, Black, and Hispanic. REL Northwest and ESD district leaders in four districts partnered to co-design and pilot training materials to help school and district teams increase equity in school discipline practices. REL Northwest also produced a tool so other districts and states can identify discipline disproportionality.

REL Pacific helped to make a language program more evidence-based:

  • Recognizing the role of the school in sustaining Kosrae’s cultural and linguistic heritage and preparing students for the global world, Kosrae Department of Education (KDOE) leaders reached out to REL Pacific for support in creating a new language immersion policy and program that better supports the goal of building student proficiency in both Kosraean and English. REL Pacific supported KDOE by providing coaching on the research behind effective bilingual education models, policy implementation frameworks, and language assessments. REL Pacific and Region 18 Comprehensive Center (RC18) subsequently collaborated to provide complementary supports to ensure KDOE had increased capacity to implement its bilingual language policy in schools across the island. As REL Pacific continued support in best practices in bilingual instruction, classroom observation, and teacher professional learning, RC18 provided supports such as bilingual materials development and financing options for the new policy. KDOE began piloting the new policy in two elementary schools in the fall of 2021.

REL Central supported Nebraska by providing evidence-based resources and training to support the implementation of new legislation:

  • In 2018, the Nebraska Reading Improvement Act was passed to decrease the number of struggling readers in grade 3 across the state. The Nebraska State Board of Education (NSBE) and the Nebraska Department of Education enlisted REL Central’s support in providing the state’s elementary school teachers with evidence-based practices for the teaching of reading. To meet this need, REL Central reviewed strategies in eight What Works Clearinghouse practice guides on reading, writing, and literacy instruction and distilled the information into summary documents that were aligned with the state’s initiative. Each document is featured on NDE’s NebraskaREADS website and each describes a practice guide recommendation, how it should be implemented, and discusses the appropriate grade level or target student population (for example, English learners). REL Central also provided trainings to support regional education service unit staff and school-based educators in reviewing, selecting, and testing evidence-based reading strategies.

Finally, through applied research studies, REL Northeast and Islands and REL Southwest helped education leaders answer important questions about whether students in certain localities had equitable access to important services. These studies informed leaders’ decisions about state programs or indicators:

  • In an effort to increase the percentage of children ready for kindergarten, Vermont passed Act 166 in 2014 that provided access to high-quality prekindergarten (pre-K) for all 3- and 4-year-olds and for 5-year-olds not yet in kindergarten. As universal pre-K began in the 2016/17 school year, officials were concerned about unequal distribution and availability of high-quality pre-K programs across the state. The Vermont Agency of Education, the Agency of Human Services’ Department for Children and Families, and Building Bright Futures (Vermont’s early childhood advisory council) participated in the Vermont Universal PreK Research Partnership with REL Northeast & Islands to answer these important questions. Through one study, the REL found that although the majority of pre-K children were enrolled in the highest quality programs, some children had less access to high quality programs in their home districts. These findings led the Vermont legislature to maintain a provision that allows families to enroll their children in programs outside their home district.
  • Texas House Bill 3 (HB3), a comprehensive reform of the state’s school finance system passed in 2019, established a college, career, and military readiness outcomes bonus, which provides extra funding to districts for each annual graduate demonstrating college, career, or military readiness under the state accountability system. Leaders at the Texas Education Agency (TEA) were concerned that it may be hard for small and rural districts to demonstrate career readiness through the required accountability measure. Through a partnership with TEA, REL Southwest conducted a study that found that there were no substantive differences by district size or locale with respect to the percentage of students meeting the career readiness standard. Further, the study found that students who fell into two of the alternative career readiness options—CTE completers and work-based learners—had higher rates of college enrollment than graduates who met the existing career readiness accountability standard. The study also indicated that CTE completers had higher rates of either college persistence or of credential attainment after high school than graduates who met the existing career readiness accountability standard. These findings led the Commissioner of Education to recommend, and the Texas legislature to create, a new measure of career readiness in the state accountability system that met the needs of the districts across the state.

From these examples, one takeaway is clear: REL work can make a difference. RELs supported educators’ and policymakers’ efforts to improve educational programs, policies, and outcomes through use of research and evidence-based practice between 2017 and 2022. The new RELs will continue this work and, as I will write about in a future blog, they will also undertake some new types of projects. Until then, please visit the new REL website or reach out to me at Elizabeth.Eisner@ed.gov  if you have questions about the REL program and how it can help your community.

Liz Eisner is the associate commissioner of the Knowledge Use Division at the National Center for Education Evaluation and Regional Assistance


[i] One REL contract—REL Southwest (REL SW)—is on a different schedule. The current REL SW contract ends in late November of 2022 and the next REL SW contract will begin the day after the current contract ends. The contracts that just ended were the 2017-2022 contracts and the contracts that just started are the 2022-2027 contracts.

When “More Research is Needed” Is the Key Finding: Improving the Evidence Base for Nonacademic Interventions for Postsecondary Success in Rural and High-Poverty Contexts

Stakeholders in rural and high-poverty districts in Regional Educational Laboratory (REL) Appalachia’s region have noticed a troubling trend: many students graduate from high school academically well prepared but fail to enroll in college or enroll in college only to struggle and drop out within the first year. Stakeholders believe these high-performing students may face nonacademic challenges to postsecondary success, such as completing financial aid paperwork, securing transportation and housing at colleges far from home, or adjusting to campus life. To address these challenges, education leaders are looking for interventions that address nonacademic competencies: the knowledge, skills, and behaviors that enable students to navigate the social, cultural, and other implicit demands of postsecondary study.

To fill this need, REL Appalachia researchers conducted a review of the existing evidence of the impact of nonacademic interventions – that is, those designed to address nonacademic competencies – on postsecondary enrollment, persistence, and completion. The review had a particular focus on identifying interventions that also have evidence of effectiveness in communities serving students similar to those in Appalachia—high-poverty, rural students. Only one intervention, Upward Bound, demonstrated impact in rural, high-poverty communities. The review showed that Upward Bound, as implemented in the early 1990s, benefited high-poverty rural students’ college enrollment, with no demonstrated impact on persistence or completion.

Schools and communities need access to nonacademic interventions that benefit students served in high-poverty rural communities. Researchers: read on to learn more about the methods used in the evidence review, its findings, and steps you can take to support rural and high-poverty communities in improving enrollment and success in postsecondary education!

Nonacademic challenges to postsecondary success for rural students

All students face nonacademic challenges to postsecondary success, but rural populations and high-poverty populations in particular may benefit from interventions addressing those challenges because they enroll in and complete college at significantly lower rates than their nonrural or low-poverty peers. Although academic challenges contribute to this gap, rural and high-poverty populations also face unique nonacademic challenges to postsecondary enrollment and success. For example, rural students are less likely to encounter college-educated role models and high-poverty students often face inadequate college counseling at their schools (see research here, here, and here). As a result, rural and high-poverty students may have inadequate access to knowledgeable adults who can help them understand the steps needed to enroll or prepare them for the challenges of persisting in postsecondary education.  Nonacademic interventions can support students in developing the knowledge, skills, and behaviors necessary to overcome these challenges and improve postsecondary enrollment and success for rural and high-poverty students.

The need for evidence-based interventions

To support decisionmakers at rural and high-poverty schools in identifying evidence-based nonacademic interventions, researchers at REL Appalachia conducted an extensive search of the published research. The search looked for rigorous studies of nonacademic interventions with evidence of positive impact on college enrollment, persistence, performance, and completion for students attending rural schools or who were identified as high poverty. The purpose of the project was to identify a suite of interventions to recommend to these education leaders.

The results of our review indicate there may be gaps in the evidence available to all decisionmakers who are trying to help their students succeed in postsecondary education. The search first identified any studies that focused on postsecondary outcomes of nonacademic interventions serving students ages 5–19. Of the 1,777 studies with the relevant keywords, only 65 focused on the postsecondary outcomes of nonacademic interventions. Next, we evaluated these 65 studies against the What Works Clearinghouse (WWC) design standards, which assess the quality of evaluation study designs. Only 17 studies met WWC’s rigorous study design standards with or without reservations. Finally, researchers from REL Appalachia identified studies that showed positive impacts on students overall, and studies that looked at rural students and students identified as high poverty in particular. Only eight studies showed positive, statistically significant impacts on students’ postsecondary enrollment or success overall. Of the eight studies that showed positive impacts of nonacademic interventions on postsecondary outcomes, only three focused on high-poverty populations, and only one reported specifically on rural populations.

This figure shows the number of studies remaining at each stage of screening. The original searches returned 1,777 unique studies. Of these, 65 focused on postsecondary outcomes of nonacademic interventions with students ages 5 to 19. At the next stage, 17 studies remained that met these criteria and also met WWC standards. At the final stage, 8 studies remained that met all criteria and had a positive effect on postsecondary outcomes.

 Without additional research that focuses on low-income and rural contexts, schools and districts are left to implement programs with limited or no evidence of effectiveness. For example, the Quantum Opportunity Program (QOP) provides mentors to students as part of a long-term after school program. However, WWC reviews of QOP studies (here and here) showed indeterminate effects of the program on postsecondary outcomes. The lack of evidence should not detract from the important role QOP has in serving students, but it leaves open the question of whether those efforts are having the intended effects. With few clear alternatives, schools and districts continue to implement programs with limited evidence of effectiveness.

Action steps

Nationwide, 19 percent of U.S. public school students are enrolled in a rural school, and 24 percent are enrolled in a high-poverty school. To help districts and schools provide effective supports to those students, researchers can provide high-quality evidence on the effectiveness of nonacademic interventions in these contexts.

Carry out more studies on specific interventions designed to improve nonacademic competencies. REL Appalachia’s review found that the research on nonacademic competencies often focuses on defining the competencies themselves, rather than on studying interventions designed to develop the competencies. Of the 1,777 unique studies identified in our review, only 65 (3 percent) studied outcomes of interventions designed to improve nonacademic competencies. From these, we identified only 17 studies, representing nine interventions, with sufficiently rigorous designs to examine evidence of effectiveness.

The limited availability of rigorous evaluations of interventions suggests that, as researchers, we need to increase our focus on evaluating new interventions as they are developed or tested. Decisionmakers rarely design their own programs or interventions from scratch; they need to be able to identify existing programs and policies that are within their power to implement and have been proven effective in similar communities. Researchers can help decisionmakers select and implement successful interventions by providing evidence on whether interventions that develop students’ nonacademic competencies have positive effects on students’ postsecondary outcomes.

Design studies to generalize to rural and high-poverty populations. As researchers, we can also increase our focus on rural and high-poverty populations. REL Appalachia’s review found only three studies that focused on a high-poverty population and one that focused on a rural population. As researchers, we can address this gap in two ways: (a) we can carry out more studies specifically focused on rural and high-poverty areas; and (b) when using large national datasets or multi-site studies, we can consider rural and high-poverty populations in our sampling and disaggregate our results for these populations.

Summary

Stakeholders in rural and high-poverty contexts are looking for nonacademic interventions that will be effective with their students. To that end, REL Appalachia carried out an extensive review of evidence-based interventions. The review found few rigorous studies of nonacademic interventions, and even fewer that examined findings for students identified as high poverty or in rural settings. Without additional research, schools and districts serving rural and high-poverty populations may implement interventions that are not designed for their circumstances and may not achieve intended outcomes. As a result, resources may be wasted while rural and high-poverty students receive inadequate support for postsecondary success.  In addition to investing in rigorous studies, which can take a long time to complete, researchers and practitioners can also collaborate to implement short-term research methods to identify early indicators of the success of these programs. For example, researchers may be able to support schools and districts in developing descriptive studies examining change over time or change in formative assessment outcomes.

 

 

 Researchers have a role in helping more high school graduates from rural communities enroll, persist, and succeed in postsecondary education.

 

Rural and high-poverty schools and districts have unique strengths and challenges, and the lack of information about how interventions perform in those contexts presents a dilemma for decisionmakers: do nothing, or else muddle through with existing evidence, investing in interventions that don’t address local needs. As researchers, we can help resolve this dilemma by providing rigorous evidence about effective interventions tailored to rural and high-poverty contexts, as well as supporting practitioners in using more accessible methods to investigate the short-term outcomes of the programs they are already implementing.

 

by Rebecca A. Schmidt and CJ Park, Regional Educational Laboratory Appalachia

 

Introducing REL 2022

As I write this, my colleagues and I at the Regional Educational Laboratory (REL) Program are thinking about a single number: 535. No, we’re not concerned about 535 because it represents the number of voting members of Congress, though that would be a good guess. We’re also not thinking about Interstate 535, the “2.78-mile-long Auxiliary Interstate Highway spur of I-35 in the U.S. states of Minnesota and Wisconsin,” though now I’m intensely interested in why it might be that, at least according to Wikipedia, this road is “known locally as the ‘Can of Worms’ interchange.” Instead, my colleagues and I are excited about 535 because it represents the number of days between now and the start of the next cycle of the REL program, affectionately known as REL 2022.

Over a year ago, we began a process that culminates in the awarding of contracts to run each of our regional labs. We are excited to share our preliminary thoughts about the contours of REL 2022 through a Request for Information, or RFI, which we have posted hereI hope you will take time to read the RFI. If you have questions or suggestions after doing so, I hope you are moved to comment. Details on how to offer your feedback can be found in the RFI.

Importantly, we aren’t proposing to radically restructure the REL program. Instead, we are retooling some existing expectations and adding a few new features. Below, I’ve highlighted a few proposed changes that merit special attention.

The purpose of RELs is to improve student outcomes. Not to put too fine a point on it, but everything that takes place in REL 2022 should be in service of improving student outcomes. This does not mean that every REL project will, by itself, have a directly observable impact on achievement. But the work of any given REL, in concert with the efforts of those with whom it works, should be trained on a singular focus: bettering the lives of the students through education. There is no other, better, or higher calling.

We accomplish our purpose by working in partnership with stakeholders to support their use of evidence-based practices. Evidence-based practice is “baked in” to the statute that authorizes the REL program, and the importance of building and using evidence in education—and government more generally—is reiterated throughout federal law. (See, for example, the Every Student Succeeds Act of 2015 and the Foundations for Evidence-based Policymaking Act of 2018.) However, our emphasis on evidence isn’t rooted in a statutory imperative. Instead, it’s based on a set of core beliefs about our work: that researchers and educators can strengthen education via the rigorous application of the scientific method; that resources, including money and time, are constrained and that efforts with demonstrated effectiveness should be prioritized; and that each and every student deserves the best of “what works” in education, no matter their circumstance.

Nothing changes if nothing changes. In the REL 2022 cycle, we are explicitly asking RELs to think of themselves as “change agents.” This expectation is, I believe, entirely new to the REL Program and is likely to be uncomfortable to some. For that reason, it is helpful to be clear about what we’re expecting and why. Here goes.

I daresay that, no matter how proud they might be of their students and their educators, there is not a state chief, a district superintendent, or building principal who would report they are serving each of their students as well as they wish they could. (If you’re the one who does, please stop reading this blog and call me. I want to share your successes!) Each of those leaders has something they want to do better on behalf of their students and are contemplating, if not actively pursuing, change. It is our hope that RELs can join them in making change, with evidence in hand and research tools at the ready. REL reports, resources, and trainings are not ends unto themselves. They are means to enable the change efforts of local, state, and regional education leaders, working on behalf of students to improve important outcomes.

RELs work in partnership. Education research and technical assistance must be done in partnership with those it is meant to inform. Absent that, it is likely to fail to achieve its goals. At best, potentially positive impacts will be blunted. At worst, harm will be done. There’s a simple solution: collaboration that authentically engages stakeholders in all phases of project design and execution. That isn’t, I realize, as simple to do as it is to write.

As vendors consider the REL 2022 cycle, we ask that they keep two things in mind about what we’ve traditionally called partnerships. First, there are no necessary restrictions on who RELs can partner with when working with stakeholders to achieve stakeholder goals. Does it make sense to partner across levels of education within a state? Do it. Is there a state or national advocacy association that would accelerate a partner’s progress? Engage it. Is there are role for business or industry? Leverage it. A second and closely related concept is that there are no restrictions on partnerships’ functional forms. In general, it does not matter one whit to IES whether you prefer NICs, DBIR, or any other particular form of research partnership. What does? That RELs build projects in partnership—however and with whomever—intentionally, with the goal of supporting partners’ change efforts to achieve the goals they have identified.

We encourage deeper, not broader, work. We believe RELs are more likely to achieve success when they focus partnerships on clearly defined problems of policy or practice in specific geographies. A “Six-State Research Alliance on High School Graduation” can do important and meaningful work—but the process of agreeing on the work to be done and the targets to be met, seeing that work through to completion, and then achieving pre-specified goals is likely to be exceptionally difficult. The “South-Central Kansas Partnership for Kindergarten Readiness” or the “Maricopa County Alliance for Reducing Chronic Absenteeism in High Schools” may be more likely to achieve impact. This is not to say that lessons learned locally should not be shared regionally or nationally, or that groups with common interests might not form “communities of practice” or other networks for the purpose of sharing information or building connection. Rather, we ask RELs be strategic in scoping their highest-intensity work.

We define success as achieving measurable stakeholder goals. Evaluating the impact of research and technical assistance projects is notoriously hard. Often, program managers and the evaluators with whom they work are forced to satisfice, relying upon end-user self-reports of the quality, relevance, and usefulness of a provider’s work. Counts of outputs, such as report downloads and attendees served, are particularly common metrics reported in evaluation studies. Satisfaction is the coin of the realm. Lest I be accused of throwing stones inside my own glass house, let me be clear that we currently use these very measures to characterize the effectiveness of the current REL program.

In REL 2022, it is our intention to shift focus beyond outputs to emphasize outcomes. We will ask RELs to demonstrate, on a regular basis, that they are making progress toward the goals stakeholders set for important student outcomes at the outset of their work, with the acknowledgment that outputs are often critical to achieving a long-term goal and that satisfaction can be an important leading indicator. In 2027, the mark of success won’t be a glowing narrative from a state superintendent or school superintendent about the REL cycle just passed. Instead, it’ll be seeing that the quantifiable goals those leaders set for their work with the REL program were achieved.   

Putting RELs’ capacity for rigorous R&D to work. Finally, there is one manifestly new requirement for RELs as part of the 2022 cycle, one that I am particularly excited about because it brings together the best of two NCEE programs: the RELs and the What Works Clearinghouse™ (WWC). As part of the 2022 cycle, each REL will be required to develop—and then evaluate—a comprehensive toolkit based on a WWC Practice Guide, helping educators instantiate evidence-based practices in the classroom. RELs already have experience taking the content from Practice Guides and transforming them into tools for educators. Two examples include Professional Learning Community guides for both foundational reading and English learners. Similarly, North Carolina State University’s Friday Institute has looked to Practice Guides for inspiration to develop massive open online courses (MOOCs), including foundational reading and fractions. None have been evaluated for efficacy. Of course, the development and testing of these new toolkits will follow the expectations set above, including the expectation that strong and inclusive partnerships are at the root of all high-leverage work.

My NCEE colleagues and I are excited about the possibilities that REL 2022 represents. The REL program has a proud history and a strong track record of service to local, state, and regional stakeholders. We hope that, as you review the REL 2022 RFI, you’ll find the next iteration of the program continues in that tradition. As always, I welcome your feedback.

Matthew Soldner

Commissioner, National Center for Education Evaluation and Regional Assistance