NCEE Blog

National Center for Education Evaluation and Regional Assistance

Data-Driven Decision-Making in Education: How REL Work Makes a Difference

Photo of a workshop REL Appalachia conducted as part of their Strengthening Students’ Preparation for College and Careers in Northeastern Tennessee partnership
Photo of a workshop REL Appalachia conducted as part of their Strengthening Students’ Preparation for College and Careers in Northeastern Tennessee partnership

The Regional Educational Labs (REL) program, operated by the U.S. Department of Education’s Institute of Education Sciences (IES), supports state education agencies, schools and school districts, and institutions of higher education nationwide in using data and evidence-based practice to improve opportunities and outcomes for learners. Operating in all 50 states, the District of Columbia, Puerto Rico, and the U.S. Territories and Freely Associated States of the Pacific region, the REL program brings together the expertise of local communities, top-tier education researchers, and education scientists at IES’s National Center for Education Evaluation and Regional Assistance (NCEE) to address the most vexing problems of education policy and practice in states and regions—on demand and free of charge.

Data-driven decision-making can be a critical tool to address resource disparities, enhance student success, and promote equitable outcomes. Collecting and analyzing relevant data gives insights into student performance, attendance patterns, disciplinary actions, and more. Rather than relying solely on assumptions or "how we've always done things," educators can use these insights to more effectively tailor policy and practice to better meet the unique needs of their students and communities.

In this third installation of our blog series, we'll explore three case studies showcasing the significance of data-driven decision-making and highlightig the REL Program’s pivotal role in shaping the future of education.

REL Mid-Atlantic: Data-Driven Decision-Making to Improving School Accountability Measures in the Wake of COVID-19

The suspension of standardized testing and accountability measures during the pandemic posed challenges for schools and districts. The interruption in assessments meant there was no baseline data from the 2020-21 school year against which to compare future performance. The sudden shifts between in-person and remote learning, disruptions in curricula, and variations in student participation made it difficult to interpret school performance data and introduced additional instability to school performance indicators. This uncertainty made it more important than ever to ensure that accountability measures were as accurate and reliable as possible to avoid mislabeling schools and educators.

To address these challenges, Pennsylvania's Department of Education (PDE) turned to the expertise of REL Mid-Atlantic. The state recognized the need to reduce measurement error and increase the statistical reliability of performance measures, particularly for subgroups of students, who are critical in identifying schools for targeted support and improvement under the Every Student Succeeds Act (ESSA). REL Mid-Atlantic and PDE embarked on a pioneering effort to use Bayesian statistical methods to reduce random error and stabilize performance measures. One focus was the potential instability that small sizes of student subgroups introduced. By minimizing statistical fluctuations, the new approach aims to ensure that schools are not wrongly identified for improvement based on temporary fluctuations in data. This represents a significant advancement in educational accountability and can have far-reaching implications for how states evaluate school performance.

This innovative work is groundbreaking in multiple ways. It addresses an immediate need to ensure that schools are not unfairly labeled as underperforming due to the unpredictability of data during the pandemic. This helps to maintain the credibility of the accountability system. Moreover, this effort extends beyond the immediate crisis. By introducing more accurate and reliable accountability measures, educators can be more confident that performance evaluations are based on solid, consistent data. This, in turn, can lead to greater buy-in and cooperation from educators and stakeholders.

REL Appalachia: Strengthening Preparation for College and Careers in Northeastern Tennessee

Recently, educators from a consortium of districts in northeastern Tennessee report having experienced a wake-up call when they reflected on their own experience preparing for their own college and careers and compared it with feedback from interviews with current students. Educators learned that students are still facing the same challenges, like not receiving enough guidance on navigating the college application process or finding and applying for scholarships.

Since early 2022, district leaders and staff from the Niswonger Foundation have joined with REL Appalachia in the Strengthening Students Preparation for College and Careers partnership. Together, they reflect on districts’ college and career readiness data and identify improvements to programs and services that better prepare students for life after college. Educators have participated in coaching and technical assistance workshops led by REL Appalachia where they look at quantitative data such as their college enrollment and career technical education (CTE) attainment rates. They have supplemented these numbers with student voices through interviews to better understand the whole picture.

One key takeaway from the analysis of outcome data and student interviews was an increased awareness that everyone in their system has the potential to affect postsecondary trajectories. Partners are now considering what changes they can make to help foster social-emotional preparation for college. For example, one partner stated that these changes could be as simple as having counselors support lunch duty as an opportunity for them to build relationships with students.

With a better understanding of their data on college and career preparation, partners are now asking deeper questions about how they can improve their systems to better support students. In the coming years, REL Appalachia will help the partnership address research questions that will allow them to understand better how their programs and services are strengthening preparation for college and careers.

REL Northwest: Accelerating Literacy Outcomes in Montana Through Evidence and Data Use

Despite having a dedicated leadership team that had implemented multi-tiered systems of support to improve literacy, Laurel Public Schools in Montana faced a pressing challenge: only 50 percent of students in grades 3-8 were proficient in reading. For students from economically disadvantaged backgrounds, this dropped to 25 percent. Moreover, achievement gaps persisted between White students and students from diverse racial and ethnic backgrounds, ranging from 10 to 35 percentage points. The district’s own data made it evident that a significant intervention was needed to uplift literacy instruction and outcomes in the district.

Laurel Public Schools and REL Northwest collaborated to address the immediate issue while creating a sustainable solution that would transform literacy instruction and student outcomes over the long term. Laurel wanted to take a close and critical eye to their existing multi-tiered systems of support in reading (MTSS-R) and revise their practices to align better with evidence-based methods to effectively tailor reading instruction and assessment practices. Their goal was to ensure that classroom instruction and interventions were appropriately differentiated for all learners, leading to improved reading achievement and reduced achievement disparities.

REL Northwest was pivotal in guiding the district toward using data effectively. In the first year of the project, REL Northwest worked with literacy leadership teams at Laurel to create a rubric that asked Laurel educators to reflect on evidence-based practices within their MTSS-R and describe practices in their district that are aligned with evidence-based practices and practices in need of improvement.

Using the data they collected, the literacy leadership teams identified previously undetected problems of practice. For example, while evidence-based targeted Tier 2 reading interventions and processes were in place, the data suggested that teachers skipped these targeted interventions and moved directly to Tier 3, referring struggling readers for special education. To support long-term improvement, the literacy leadership teams have begun work on action and monitoring plans as part of a “Plan-Do-Study-Act” continuous improvement cycle. 

The partnership between Laurel Public Schools and REL Northwest showcases how the partnership between RELs and dedicated state and local leaders can lead to educational transformation. It underscores the power of data as a catalyst for change and highlights the importance of evidence-based practices in driving educational excellence.

Looking Ahead

Data-driven decision-making can help states and districts deliver on their commitments to equity, evidence-based classroom practice, enhanced student outcomes, and informed policymaking. As these case studies demonstrate, the REL program can support these states and districts in effectively harnessing data to shape a brighter future for our students and the whole of our educational system.


This blog was written by Nicassia Belton (Nicassia.Belton@ed.gov), contracting officer’s representative with the REL Program at NCEE.

The Regional Educational Lab Program: Making a Difference in Educator Recruitment and Retention

Torrence Williams, Director of Teacher Advancement at the Associated Professional Educators of Louisiana, leads a professional learning module training for Louisiana’s New Teacher Experience program.
Torrence Williams, director of teacher advancement at the Associated Professional Educators of Louisiana, leads a professional learning module training for Louisiana’s New Teacher Experience program.

The Regional Educational Labs (REL) program, operated by the U.S. Department of Education’s Institute of Education Sciences (IES), supports state education agencies, schools and school districts, and institutions of higher education nationwide in using data and evidence-based practice to improve opportunities and outcome for learners. Operating in all 50 states, the District of Columbia, Puerto Rico, and the U.S. Territories and Freely Associated States of the Pacific region, the REL program brings together the expertise of local communities, top-tier education researchers, and education scientists at IES’s National Center for Education Evaluation and Regional Assistance (NCEE) to address the most vexing problems of education policy and practice in states and regions—on demand and free of charge.

It's not exactly breaking news that many schools in our country struggle to fill vacancies in their teacher workforce. This past fall, IES’s National Center for Education Statistics surveyed public school leaders about staffing challenges as they began the 2022-23 school year. The statistics were sobering: 45 percent of schools reported having at least one vacant position more than one month into the school year, and more than 25 percent of schools reported multiple vacancies. Worryingly, our most underserved students were experiencing this crisis most acutely, with roughly 60 percent of schools in high-poverty neighborhoods or with a high-minority student body reporting at least one vacancy. While all of us anxiously await data on the 2023-24 school year—which should be available later this year—RELs and their state and local partners are working to strengthen all aspects of the teacher pipeline.

In the second of a four-part blog series, we highlight four REL research and development projects that address educator recruitment and retention. Each demonstrates how RELs are leveraging their distinct capacity for innovation, rigorous research, and authentic partnership to deliver locally focused and evidence-based supports to the regions, states, and communities they serve.

REL Northwest: Examining Strategies to Improve Teacher Recruitment and Retention in Rural Alaska

Like many rural school districts across the nation, Alaska’s Lower Kuskokwim School District (LKSD) is experiencing a persistent and pressing need to attract and retain educators. Near the start of the 2022-23 school year, 78 positions—nearly one-quarter of all teaching positions in the district—remained unfilled. New teachers were often recruited to the district from other countries such as the Philippines, leading to low retention rates and a constant churn of new educators. Faced with this persistent, high-stakes problem, leadership at LKSD decided to partner with REL Northwest to discover research-based solutions to their teacher recruitment and retention crisis.

REL Northwest is partnering with LKSD to identify evidence-based strategies and tools to continuously monitor and improve working conditions with the goal of increasing teacher retention. To do that, LKSD plans to implement the recommendation of strengthening teacher working conditions from the state's Teacher Retention and Recruitment Plan. As a first step in this partnership, REL Northwest staff reviewed and summarized research on working conditions and teacher retention to identify eight factors that may influence a teacher's decision to stay or leave.  

Lower Kuskokwim leaders decided to focus on activities to identify how changes related to three of those factors—supportive school leadership, available time for teachers, and community engagement—may improve working conditions. The first activity will adapt existing LKSD data sources and develop a research-informed tool to monitor teacher perceptions of school leadership and collect further data to inform the district’s action plan.

Staff also identified a school leadership responsibility unique to their district that may affect teacher trust: managing teacher housing. Housing is a major challenge, not only in remote villages without road access, but in many areas of the country. REL Northwest led partners through an activity to brainstorm how school leaders could improve housing and develop a theory of action for how those strategies would improve working conditions and promote retention. The strategies included establishing realistic housing expectations for new teachers, revising the process for leader evaluation of housing needs, and changing the budgeting process to make maintenance needs and upgrades easier. As a result of the work, district leaders are designing a program that allows teachers to apply for district funding to make simple housing upgrades, such as changes in lighting or painting.

REL Central: Strengthening the Teacher Pipeline in South Dakota to alleviate Teacher Shortages

Like many states, South Dakota is experiencing a teacher shortage that has worsened in recent years. Late last year, SDDOE partnered with REL Central to support one component of their response to this challenge: developing new pathways into teaching for candidates such as paraprofessionals and other South Dakota residents who have interest in entering the teacher workforce. Initially, the work focused on the design and implementation of a teacher apprenticeship program designed to support paraprofessionals as they acquire their teaching degrees and as they are mentored to become certified teachers.

In March, REL Central began work with SDDOE on a fast-turnaround project to support the development of a survey for paraprofessionals about their interest in the pilot apprenticeship program and the types of supports they seek from mentor teachers. Within a matter of weeks, the survey was developed and administered to paraprofessionals statewide. With survey data in hand indicating that hundreds of South Dakota paraprofessionals were interested in such a program, the pilot was expanded by SDDOE to support additional slots starting in fall 2023. In the coming months, REL Central will work with SDDOE to further refine this program by incorporating research evidence from other “Grow Your Own” teacher workforce programs on the components of effective mentoring and by helping the state to generate, collect, analyze, and use data from participants to inform further improvements to the pilot apprenticeship program.

REL Pacific: Strengthening the teacher workforce in Palau

The Republic of Palau, like many school systems, has experienced challenges in recruiting and retaining enough teachers to provide every student with a high-quality education. The geographic isolation of Palau compounds these challenges. Many local Palauan teachers do not have a four-year college degree or are teaching outside of their area of certification. As was the case in Alaska’s Lower Kuskokwim School District, one solution has been to bring in teachers from out of the country to fill vacancies; however, the turnover rate of these teachers is very high. The Palau Partnership to Support Teacher Effectiveness–– a collaboration with the Palau Ministry of Education (MOE), Palau Community College (PCC), six private schools, and REL Pacific ––is focused on building more sustainable solutions. 

The long-term goals of the partnership include Palau private schools adopting a teacher effectiveness measurement system to support, develop, and retain effective teachers; Palau private schools adopting a systemwide instructional coaching process for improving teacher effectiveness; and Palau MOE and PCC reviewing data on the effects of teacher education programs and making implementation adjustments so that their available resources may be used more effectively and efficiently. 

REL Pacific is supporting partner schools to realize their goals by drawing from resources on indicators of successful teacher recruitment and retention as well as best practices of effective teaching. REL Pacific is providing schools training and coaching on data-driven decision-making and plan-do-study-act (PDSA) cycles in the context of the schools’ goals to improve literacy instruction through developing systemic supports for teachers. By incorporating ways of measuring best practices of effective teaching into the schools’ instructional coaching processes, each school will be better able to enact systemic change to address its specific teacher development and retention needs. Additionally, an applied research study is underway that will describe teacher pathways and certification patterns. The findings from this descriptive study will inform future efforts of Palau’s education community to address the new teacher certification requirements and overall educator shortage crisis.

REL Southwest: Partnering to support early career and aspiring teachers in Louisiana

Louisiana’s educator shortage is compounded with low retention rates for early-career teachers. Teachers with 2–5 years of experience left public schools in 2020 at a rate of 30 percent, compared with 17 percent of teachers with 6–10 years of experience. REL Southwest and the Louisiana Department of Education (LDOE) have formed the Supporting Early Career and Aspiring Teachers (SECAT) partnership to improve supports for novice and aspiring teachers. Through the SECAT partnership, REL Southwest aims to strengthen LDOE’s capacity to use evidence to refine new initiatives that support early career and aspiring teachers, such as the New Teacher Experience program and the Louisiana Pre-Educator Pathway, the state’s largest “Grow Your Own” program. This work builds from a previous partnership between REL Southwest and LDOE focused on exploring and evaluating the early impacts of Louisiana’s Believe and Prepare teacher residency program.  

The work of the SECAT partnership kicked off this past spring. Over the next five years, REL Southwest will work with LDOE and school systems in Louisiana to strengthen their capacity to generate and use evidence to refine existing programs for early career and aspiring teachers. In the first year of the partnership, REL Southwest and LDOE partners plan to focus on technical assistance that builds LDOE’s capacity for evaluating the New Teacher Experience. In future years of the partnership, REL Southwest will study LDOE’s efforts to support new and aspiring teachers. Along the way, REL Southwest will share important takeaways, resources, and policy implications related to teacher recruitment and retention learned through the partnership with Louisiana.

Stay tuned for part three of our “Making a Difference” series, focused on school accountability systems. As always, my (virtual) door is open if you have questions about the work highlighted in this blog, or anything else on REL Program. Just email me at chris.boccanfuso@ed.gov.

Getting to Know ED: My Journey as a STEM Next Fellow at IES

This guest blog was contributed by Dr. Holly Miller, who currently serves as a STEM Next Opportunity Fund Fellow at the Institute of Education Science’s National Center for Education Evaluation.

Since August 2022, I’ve been serving as the STEM Next Opportunity Afterschool and Summer Learning Fellow at the U.S. Department of Education (ED). More specifically, I work within the Department’s Institute of Education Sciences (IES).

Upon arriving at IES, I was charged with a specific challenge: amplify how evidence-based practice in out-of-school time (OST) can support student learning and development. This mission was made all the more relevant by the need for states and districts to respond to the consequences of the COVID pandemic which, at the time, remained an official national emergency.

Perhaps naively, I hoped to walk in on Day One and find “The Official Compendium of Evidence-based Practices in Global Pandemics and Related Crises” that I could pull off the shelf and hand to educators. Unfortunately, I quickly discovered no such tome existed. And I began to realize that one of the biggest challenges I’d face in my new role was getting to know ED itself! To an outsider, the Department can seem like a huge machine. Getting to know it, though, can pay incredible dividends. As I came to learn, there are tons of great resources—if only you know where to look.

One of OST educators’ first stops in getting to know ED should be IES. For the uninitiated, IES is the Department’s statistics, research, and evaluation arm. The mission of IES is to provide scientific evidence on which to ground education practice and policy and to share this information in formats that are useful and accessible to educators, parents, policymakers, researchers, and the public. It is independent and non-partisan.

Across its four centers—the National Centers for Education Statistics, Education Evaluation, Education Research, and Special Education Research—IES conducts six broad types of work (http://ies.ed.gov):

1. Providing data to describe the “condition of education,” including students’ academic proficiency.

2. Conducting surveys and sponsoring research projects to understand where education needs improvement and how these improvements might be made.

3. Funding development and rigorous testing of new approaches for improving education outcomes for all students.

4. Conducting large-scale evaluations of federal education programs and policies.

5. Providing resources to increase the use of data and research in education decision-making, including independent reviews of research on “what works” in education through the What Works Clearinghouse.

6. Supporting the advancement of statistics and research through specialized training and development of methods and measures.

I could see that this work had the potential to benefit a variety of stakeholders—teachers, administrators, students, researchers, and policymakers. Still, I had so many unanswered questions. As a middle school teacher, I frequently told students, “The only dumb question is the one you don’t ask.” Therefore, as I surveyed the education research landscape at IES, I asked lots and lots of questions. My presence at IES was akin to a toddler at the zoo for the first time: “What are those? Why is that so big? Why don’t we have more of these? When do we eat?” Months of asking and I find my queries have been distilled into two essential questions:

  1. What has been the impact of the COVID pandemic on students and educators; and

 

  1. How can education research, like that conducted or sponsored by IES, help us understand—and address—those impacts?

What has been the impact of the COVID-19 pandemic?

The pandemic disrupted nearly every aspect of daily life in the United States, including the education system. One of the most alarming impacts of the pandemic on education has been the widening of pre-existing gaps in student achievement and the resources that students need to be successful.

We all know the statistics … students have lost tons of learning. The "Report on the Condition of Education" is a congressionally mandated annual report from the National Center for Education Statistics (NCES). Using the most recent data available from NCES and other sources, the report contains key indicators on the condition of education in the United States at all levels, from prekindergarten through postsecondary, as well as labor force outcomes and international comparisons. For example, the report on the condition of education 2023 recently released shares that on both the 4th- and 8th-grade NAEP mathematics assessments, higher percentages of students performed below NAEP Basic in 2022 than in 2019 (Irwin et al., 2023).  This has been particularly bad among students who have historically been underserved. The average NAEP mathematics scores in 2022 were generally lower for English Learners (EL) students than for non-EL students; lower for those identified as students with disabilities than for their peers without disabilities; and higher for students in low-poverty schools than for students in high-poverty schools. These patterns were similar to those observed for reading (Irwin et al., 2023).

This is surely due, at least in part, to differences in the resources students have access to. Even before the pandemic, huge gaps in resources existed. The pandemic only made matters worse. According to a report by the U.S. Department of Education’s Office for Civil Rights (OCR) (2021), low-income students and students of color have been disproportionately negatively impacted by school shutdowns and remote learning practices. These students often lack access to reliable technology and internet resources, making it difficult for them to participate fully in online classes and complete assignments. Additionally, many students rely on meals provided by schools, so the closure of physical school buildings has led to food insecurity for some.

Also of note: the dramatic effect on student wellbeing. During the pandemic, mental health concerns such as fear, anxiety, and depression were common among the general public, especially children and older adults (Brooks et al., 2020; Pfefferbaum & North, 2020).  Research on the pandemic’s impact on mental health among students finds that “they showed increased fear, stress, and decreased happiness, and these were associated with their learning quality change.” (Hu et al., 2022).

Furthermore, the impact of COVID on educators is increasingly well-known. Educators had to make changes in short order, often with limited resources. This had consequences. Educators faced increased stress levels due to the shift to remote instruction, and many reported struggling to maintain a work-life balance while working from home. Findings indicate teachers reported greater mental health concerns than those in many other professions, and that remote teachers reported significantly higher levels of distress than those teaching in person (Kush et al., 2021). For some, it was too much, and they made the decision to leave the profession. Forty percent of public schools hiring for open teaching positions in special education in 2020–21 reported having difficulties filling the opening, compared with 17 percent in 2011–12 (Irwin et al., 2023) Not only were teachers leaving the workforce, but potential teachers were second-guessing their career choice. The number of persons enrolled in traditional teacher preparation programs decreased by 30 percent between 2012–13 and 2019–20, and the number of persons completing such programs decreased by 28 percent between 2012–13 and 2019–20 (Irwin et al., 2023).  

All of us are looking for solutions to all these problems. Given that I entered IES during the pandemic, I wanted to know how I could leverage its resources to help.

How can education research help?

First, I had to understand how IES, as a science agency, was structured to do the work of education research. My college textbook on education research (Newby, 2010) asserted that it should have three objectives: to explore issues and find answers to questions, to collect and disseminate information that shapes policy and decision-making, and to improve practice for practitioners.

It’s easy to see how the six broad areas of work at IES I listed above fit within those three objectives. For example, in normal (that is, pre-COVID) times, it’s the job of the National Center for Education Statistics (NCES) to collect and disseminate education-related statistics and information about student achievement to inform the work of researchers, policymakers, and other education decision-makers. IES’ two research Centers, the National Centers for Education Research (NCER) and Special Education Research (NCSER) support researchers’ exploration of a wide range of education topics and their use of high-quality methods to answer important questions of policy and practice. Finally, the National Center for Education Evaluation and Regional Assistance (NCEE) conducts its own rigorous evaluations of federal policies and programs; supports states and districts in the use of data, evidence, and applied research to improve local practice; and disseminates information about “what works” through its What Works Clearinghouse (WWC). In the wake of the pandemic, IES had to quickly focus its activities and resources to meet new demands across the education system. Here are just a few of the new questions that IES had to address amid the pandemic.

  • What’s happening in schools, and who is learning in-person versus virtually or in hybrid settings? In late 2021, NCES leveraged work being done as part of the National Assessment of Educational Progress (NAEP) to meet an immediate need to better understand schools’ policies about learning mode, masking, and social distancing. In the weeks that followed, the School Pulse Panel was created (https://ies.ed.gov/schoolsurvey/spp/). Initially, the School Pulse focused on collecting monthly information on the impact of the COVID-19 pandemic from a national sample of elementary, middle, high, and combined-grade public schools. Over time, its focus has broadened. While some survey questions are asked repeatedly to observe trends over time, others are unique each month. IES is now able to provide regular and near-real-time snapshots into “what’s happening” in the nation’s schools on a wide range of topics that matter to educators, policymakers, and families.

 

 

  • How can educators and caregivers support student learning in online, hybrid, and at-home settings? With schools closed and remote learning becoming the norm, educators and caregivers had to adapt their teaching methods and find new ways to engage students. As part of a mandate to provide assistance about “what works” in education, NCEE supported a series of efforts to bring together information for teachers navigating online and hybrid teaching environments and for caregivers who were providing instruction at home. NCEE commissioned work leading to the development of the “Best Practice in K-12 Online Teaching” minicourse (here), freely available from North Carolina State University, to support teachers new to online education in their transition to the medium. (The literature review on which the mini-course is based can be found here). NCEE’s Regional Educational Laboratories developed nearly 200 pandemic-related resources. Notable examples include “Supporting Your Child’s Reading at Home” (https://ies.ed.gov/ncee/rel/Products/Region/southeast/Resource/100679), which focuses on the development of early literacy skills, and “Teaching Math to Young Children for Families and Caregivers” (https://ies.ed.gov/ncee/rel/Products/Region/central/Resource/100652).

   

Since its inception in 2002, IES and its Centers have supported decision-makers—be they federal, state, or local—and educators in making use of high-quality evidence in their practice. The pandemic showed just important IES, its resources, and its infrastructure, can be.

In the pandemic’s wake, though, it seems to me that building even more evidence about “what works” is vital. The American Rescue Plan (ARP) provided historic levels of resources to expand educational opportunities and to ensure that education is better able to address the wide-ranging needs of students and their families – especially those who were disproportionately impacted by the pandemic. Many ARP investments, including those related to OST, have the requirement that programs be rooted in evidence-based practices. Because there are still things to learn about what makes strong programs, we can strengthen the field by building evidence that can address key problems of practice.

Conclusion

When I came to ED and IES, searching for information on how to use evidence-based practices to support COVID recovery within the context of OST, I was lost. As I’ve come to better understand the organization, I’ve learned that vast resources are available. Half of the battle was just figuring out “what lives where” within the Department! I hope this blog has given OST practitioners a bit of a roadmap to make their own process of discovery easier.

In Part Two of this series, I will explore how OST learning fits into ED, education research, and the post-pandemic education system. The latter has been profoundly affected, creating an opportunity for innovation and transformation in the delivery of education. The value of research cannot be underestimated in this context. As a result, my next blog will pose two questions. First, I’ll ask what the role of OST in learning recovery can be in the years ahead.  Then I’ll consider what evidence needs to be built to make the most of what OST can offer. I hope you’ll read it!

I’d love to hear your thoughts on this blog. Send them my way at holly.miller@ed.gov.

 

Citations

Brooks S.K., Webster R.K., Smith L.E., Woodland L., Wessely S., Greenberg N., Rubin G.J. (2020). The psychological impact of quarantine and how to reduce it: rapid review of the evidence. Lancet North Am. Ed. 395(10227):912–920.

Education in a Pandemic: The Disparate Impacts of COVID-19 on America's Students 2021 U.S. Department of Education’s Office for Civil Rights (OCR).

Hu K, Godfrey K, Ren Q, Wang S, Yang X, Li Q. (2022). The impact of the COVID-19 pandemic on college students in USA: Two years later. Psychiatry Res. Sep; 315:114685.

Huck, C., & Zhang, J. (2021). Effects of the COVID-19 Pandemic on K-12 Education: A Systematic Literature Review. New Waves-Educational Research and Development Journal, 24(1), 53-84.

Irwin, V., Wang, K., Tezil, T., Zhang, J., Filbey, A., Jung, J., ... & Parker, S. (2023). Report on the Condition of Education 2023. NCES 2023-144. National Center for Education Statistics.

Kush, J. M., Badillo-Goicoechea, E., Musci, R. J., & Stuart, E. A. (2021). Teacher mental health during the COVID-19 pandemic: informing policies to support teacher well-being and effective teaching practices.

Newby, P. (2010). Research Methods for Education. Pearson Education.

Pfefferbaum B., North C.S. (2020). Mental health and the Covid-19 pandemic. N. Engl. J. Med.;383(6):510–512.

The Regional Educational Lab Program: Making a Difference in Literacy and Math Outcomes

by Chris Boccanfuso, REL Program Branch Chief

REL Midwest ENACT Coach, Katie Rich, works with Milwaukee Public Schools’ grade 6 teachers and math coaches during the ENACT Summer Institute.
REL Midwest ENACT Coach, Katie Rich, works with Milwaukee Public Schools’ grade 6 teachers and math coaches during the ENACT Summer Institute.

When educators at Harts PreK-8 and Omar Elementary School in southwestern West Virginia wanted to pioneer a new approach that supported families to engage in their child's math learning, they turned to a partner who had supported their State for more than 50 years: Regional Educational Laboratory (REL) Appalachia. The resulting program, known in the Appalachian region and elsewhere as community math nights, has now served hundreds of families in West Virginia and Kentucky. And thanks in part to the attention of education writers and the national media, it is poised to help even more.

The Regional Educational Labs program, operated by the U.S. Department of Education’s Institute of Education Sciences (IES), supports state education agencies, schools and school districts, and institutions of higher education nationwide in using data and evidence-based practice to improve opportunities and outcome for learners. Operating in all 50 states, the District of Columbia, Puerto Rico, and the U.S. Territories and Freely Associated States of the Pacific region, the REL program brings together the expertise of local communities, top-tier education researchers, and education scientists at the IES National Center for Education Evaluation and Regional Assistance (NCEE) to address the most vexing problems of education policy and practice in states and regions—on demand and free of charge.

In this first of a four-part blog series, we highlight four REL research and development projects focused on strengthening math and literacy outcomes. Each demonstrates how RELs have leveraged their distinct capacity for innovation, rigorous research, and authentic partnership to deliver locally-focused and evidence-based supports to the regions, states, and communities they serve.

REL Appalachia: Engaging families for math success in Kentucky and West Virginia

A community math night (CMN) brings together educators, students, family members, and other caring adults to learn about, talk about, and have fun with math. CMNs are designed to reinforce positive math mindsets, help family members participate in their child's learning, and build a sense of community and partnership around a subject that many caregivers find daunting. As part of its support for a series of CMNs across Kentucky and West Virginia, REL Appalachia published materials and a facilitators guide so that any school can host their own event.

True to the program’s emphasis on continuous improvement and innovation, REL Appalachia has begun a new partnership with Logan County (WV) schools that builds from the foundations of the existing CMN approach as a jumping-off point to further accelerate improvement in middle school math achievement using a more comprehensive, year-round set of supports. This new effort involves coaching school-based teams to build their capacity to promote positive math attitudes with students and families, implement research-based instructional practices in the classroom, and employ inclusive family engagement strategies. Materials used to coach school teams will be pulled together into a single resource when the project is complete, but you can begin to use early content now! (See, for example, resources for “Promoting Positive Mathematics Attitudes.”)  

REL Midwest: Inspiring Milwaukee students to be lifelong STEM learners

What if some of the principles powering today’s most innovative technologies—such as artificial intelligence—could be taught to elementary students to encourage lifelong learning and success in science, technology, engineering, and mathematics (STEM)? The ENgagement and Achievement through Computational Thinking (ENACT) partnership, which includes Milwaukee Public Schools (MPS) and REL Midwest, is working with teachers to do just that.

Together, MPS and REL Midwest are developing, evaluating, and refining an approach for integrating computational thinking—a set of skills related to problem solving and explaining one’s reasoning—into MPS’ 6th grade math curriculum.

Laura Maly, an MPS mathematics teacher-leader, shared how integrating computational thinking strategies has influenced her work this year. "The [ENACT] computational thinking strategies are very prevalent throughout all strands of mathematics and across all of the grade levels that I work with," she said. "It is easy to bring the CT [computational thinking] strategies to life when working in mathematics classrooms."

REL Southeast: Improving Literacy in Mississippi –The Journey Continues

After many years at the bottom in student performance when compared with other states, the Mississippi Department of Education (MDE) made a concerted effort to improve the foundational literacy skills of students in kindergarten through grade 3. The results of that effort speak for themselves, with many referring to the state’s transformation as the “Mississippi Miracle.” Now the state is turning its attention to the next leg of its journey: improving the literacy skills of all students, focusing on grade 4 and beyond.

MDE and four school districts in the state–Canton, Columbus, Laurel, and George counties–formed the Mississippi Improving Adolescent Literacy Partnership with REL Southeast to ensure educators can integrate literacy strategies into a wide range of academic courses, from social studies to the sciences. To do so, partners are relying on research conducted elsewhere within NCEE: its What Works Clearinghouse. (Check out the WWC’s Improving Adolescent Literacy: Effective Classroom and Intervention Practices and Teaching Secondary Students to Write Effectively practice guides to learn more!)

After completing a REL Southeast training session, one teacher wrote a message to her principal saying the training gave her tips on how to improve student literacy in her social studies class.  “The training helped prepare me to serve my students in the classroom better and has opened my eyes to a broader range of techniques and skills. The training was so informational and engaging that it gave me the confidence level needed to help my students." Two months after the last session, a follow-up survey of participants showed that over half of the teachers reported changing (or being in the process of changing) their approach to supporting based on the training.

Now, REL Southeast is developing tools that will allow districts across Mississippi—and across the country—to replicate these early successes.

REL Southwest: Studying an enhanced approach to literacy instruction for English learner students in New Mexico

Supporting English learner students is a priority in New Mexico and for the state’s Public Education Department (NMPED). Two recent studies from REL Southwest found that significant numbers of both American Indian and Spanish-speaking English learners in New Mexico are struggling to meet grade-level standards and be reclassified as “Fluent English Proficient.” NMPED has developed a strategic plan designed to strengthen equitable educational opportunities and achievement for English learner students. To do so, the plan emphasizes supporting the whole child through evidence-based literacy instruction that is culturally and linguistically responsive (CLRI).

REL Southwest, NMPED, and several regional education cooperatives and school districts in New Mexico formed the Southwest English Learner Literacy (SWELL) partnership to help turn the State’s new plan into a reality. Over the next two school years, partners will enhance, implement, refine, and test Write to Succeed, a research-based professional learning program to help grades 4–8 teachers implement high-quality literacy instruction that includes appropriate supports for English learner students. After working with school and district coaches and NMPED staff to refine and deliver the Write to Succeed program to teachers statewide, REL Southwest will rigorously evaluate the impact of the program on teacher practice and student outcomes. What is learned will help to inform NMPED’s further adoption—or modification—of this program.

Looking Ahead

These four projects demonstrate RELs working in partnership with educators and policymakers to improve opportunities and outcomes for students in mathematics and literacy. Stay tuned for upcoming blogs about REL work on other important topics such as teacher mentorship and induction, accountability, and trauma-informed supports!

Have questions about anything you read here, or other work within the REL Program? Just email me at chris.boccanfuso@ed.gov.

Regional Educational Laboratories Develop New Tools for Educators Based on WWC Practice Guides

Whenever we get the chance to share information about the Regional Educational Laboratory (REL) program with the public, we’re often asked, “How are the 2022-2027 RELs different from past REL cycles?” In this blog, we focus on one major new effort that each REL is undertaking: the creation of a toolkit for educators based on one of the What Works Clearinghouse (WWC) Practice Guides. Each REL toolkit will include a set of resources for educators to implement and institutionalize evidence-based recommendations from a WWC Practice Guide. Importantly, each REL is co-developing their resources with educators, school, and district leaders or with postsecondary faculty and staff to ensure the toolkits’ relevance and actionability. Following the toolkit development phase, RELs will partner with educators not involved in developing the toolkits to test the usability of each toolkit and its efficacy in improving student and teacher outcomes. The RELs have current partners for toolkit development and usability testing but are looking for partner schools, districts, and postsecondary institutions in which to test the efficacy of the toolkits. These efficacy-testing partners will be among the first to benefit from the evidence-based toolkits.

Why this investment of REL and partner time and resources? WWC Practice Guides are among IES’ premier resources for translating evidence on effective practice into accessible and usable strategies for educators. Each Guide is based on a synthesis of the most rigorous research on teaching a particular subject or achieving a particular education goal. Each Guide is also based on the input of a panel of expert practitioners and researchers and includes—

  • Key recommendations for educational practice based on a synthesis of rigorous research
  • Supporting evidence for each recommendation
  • Steps to carry out each recommendation
  • Examples of the practices
  • Discussions of common implementation challenges and strategies for overcoming those challenges

WWC Practice Guide Associated with Each REL Toolkit:

REL

Practice Guide

Appalachia

Teaching Math to Young Children

Central

Teaching Strategies for Improving Algebra Knowledge in Middle and School Students

Mid-Atlantic

Teaching Elementary School Students to Be Effective Writers

Midwest

Developing Effective Fractions Instruction for Kindergarten through 8th Grade

Northeast & Islands

Assisting Students Struggling with Math: Intervention in the Elementary Grades

Northwest

Using Technology to Support Postsecondary Student Learning 

Pacific

Teaching Secondary Students to Write Effectively

Southwest

Providing Reading Interventions for Students in Grades 4 – 9

Southeast

Assisting Students Struggling with Reading: Response to Intervention (RtI) and Multi-Tier Intervention in the Primary Grades

West

Improving Reading Comprehension in Kindergarten Through 3rd Grade

RELs Emphasize Active Learning to Support Implementation of Evidence-Based Practices

Although WWC Practice Guides are some of IES’ most popular products, we also know that teachers and leaders cannot simply read about a new practice to master it. Instead, they need to engage in active learning by observing the new practice, discussing it, implementing it, receiving feedback on the practice, and continuing to improve. The REL toolkits are designed to support educators in the creation and implementation of a professional learning community (PLC) focused on the evidence-based practices outlined in a WWC Practice Guide. In these PLCs, educators will learn about the Practice Guide recommendations by reading about the practices, discussing them with colleagues, and by developing plans for implementing the practices in their classrooms. Educators will also put those plans into action and then debrief on those implementation experiences. To support this work, the toolkits will include PLC guides, workbooks, self-study guides, and rubrics. Some toolkits will also include videos of teachers effectively implementing the practices.

Each toolkit will also include the following:

  • An initial diagnostic and ongoing monitoring instrument for assessing instructional practices against the practices recommended in the WWC Practice Guide
  • A tool that enables teachers, teacher leaders, and administrators to assess the extent to which their school, district or postsecondary institution supports the implementation and ongoing monitoring of the evidence-based practice recommendations
  • A discussion of implementation steps for institutionalizing supports that help educators, building leaders, and other administrators adopt the evidence-based practices and sustain them over time

Some RELs have already started usability testing of their toolkits. Across 2025 and 2026, nine of our 10 RELs will publish final versions of their toolkits and efficacy studies on their toolkit. Both will be freely available on the REL website.[1] Visit our Newsflash page and sign up to receive newsflashes from the RELs and the IES center that houses the program—the National Center for Education Evaluation and Regional Assistance (NCEE).[2]

Partner with RELs: Help IES Study REL Toolkits

RELs will soon recruit partner schools, districts, and postsecondary institutions in their regions to conduct the toolkit efficacy studies. If you are interested in having your school, district, or institution participate in an efficacy study and benefit from being one of the first users of these toolkits, please email us at Elizabeth.Eisner@ed.gov or Chris.Boccanfuso@ed.gov. The efficacy study for each REL’s toolkit must take place within each REL’s region. Not sure which REL region is yours? Check out the “About the RELs page” on the IES website or the map visualization on our program homepage.

If you have other questions, concerns, or ideas about this work, please reach out to us. We welcome your input so that you can help IES and the RELs make the toolkits as useful and effective as possible.

Past REL Professional Development Resources based on WWC Practice Guides:

The RELs have a successful track record of creating professional development resources that complement WWC Practice Guides. For example, see:

Professional Learning Community: Improving Mathematical Problem Solving for Students in Grades 4 Through 8 Facilitator’s Guide (REL Southeast).

Professional learning communities facilitator’s guide for the What Works Clearinghouse practice guide: Foundational skills to support reading for understanding in kindergarten through 3rd grade (REL Southeast).

Professional Learning Communities Facilitator's Guide for the What Works Clearinghouse Practice Guide Teaching Academic Content and Literacy to English Learners in Elementary and Middle School (REL Southwest).

The new toolkits will expand the number of WWC Practice Guides for which the RELs develop professional development resources and will also provide instruments for assessing instructional practice and implementing institutional supports. 

Liz Eisner, Associate Commissioner for Knowledge Use

Chris Boccanfuso, REL Branch Chief


[1] REL Southwest’s contract started 11 months after the contracts of the other 9 RELs, so the REL Southwest toolkit will be released in 2027.

[2] You can also sign up for Newsflashes from IES and its other three centers—NCES, NCER, & NCSER.

Taking a pause…

In a first for ERIC, the supply of education research content the program aspires to index is regularly exceeding its capacity to do so. For the past 15 years, ERIC has consistently indexed 4,000 records a month. This pace has allowed us to index all approved sources without significant delays. However, over the past two years, the volume of content published in our approved sources has doubled. This has resulted in a backlog of publishable content and, as a result, ERIC cannot index new work in a timely fashion.

As part of our periodic collection analyses, we have been investigating potential causes of this backlog. We have found that several journals are publishing far more content than when they were originally selected to be included in ERIC. For example, one journal was publishing fewer than 20 articles per year when approved, but now is publishing over 850 per year. This is close to a 5000% increase in production.

An increased volume of published work in the education sciences—as a whole—is a good thing. However, when an individual journal dramatically increases the number of articles it publishes, it is noteworthy. In those instances, ERIC wants to ensure that the journal is still adhering to the standards and criteria it met when originally included in the index, including rigorously applying the peer-review process, if applicable, and maintaining its original aim and scope. Both are important to ensuring that work contained in ERIC is of high quality and that a wide range of key topics in education can be indexed.

ERIC’s Selection Policy already requires an ongoing review of currently indexed sources, including identifying sources that may no longer meet the Policy’s standard and criteria. As part of that review, ERIC will now identify journals that have published over twice as many articles from the year it was accepted and flag them for further review. As part of that review, ERIC will assess whether the increase is temporary and associated with a unique event, like a special issue, or if the increase reflects an ongoing trend. If the review indicates the increase is persistent, the ERIC team will recommend that indexing of that journal be paused. If a pause is approved, ERIC will stop indexing subsequent issues for a two-year period. The journal will also be removed from ERIC’s Journals List, because this list only contains actively indexed journals.

After the two-year pause, ERIC will re-review the journal. If ERIC reinstates the journal, ERIC will notify the journal and will index any content published during the two-year pause. If the journal is not reinstated, ERIC will not index any issues published during the pause.

This decision has led to a few questions:

  • Why can’t ERIC index everything? ERIC is funded by the U.S. Department of Education’s Institute of Education Sciences and has limited resources. It must prioritize indexing the highest quality education research, include content for all topic areas, and can only index a set number of records per month.
  • What is the concern with the increased volume? Increased volume may signal that the journal has changed scope to accommodate a broader set of articles or that quality assurance processes have been affected in a rush-to-publish environment. Particularly among journals that were considered to be peer reviewed when originally accepted into ERIC, the publication of articles shortly after submission may signal a substantive change in a journal’s quality assurance process such that they no longer meet the criteria needed to receive that designation. There is also a concern that if a journal greatly produces more records than estimated, the collection will get skewed in a way which would favor one topic area over another.
  • How will ERIC identify the journals to pause? As part of the source selection process, ERIC will monitor two years of current publishing and compare the number of articles published to the number published during the year the journal was selected for ERIC.
  • Why is the pause for two years? The ERIC Selection Policy says that sources may be reviewed after 2 years (24 months). To be consistent with this policy, we will automatically review paused sources after this same time frame.
  • How will I know if my journal is paused? ERIC will email the journal representatives to inform them of this decision in the coming weeks.
  • Can my journal appeal the decision to be paused? Journals may not appeal the decision to pause indexing. However, if at the end of the pause period the journal is not automatically reinstated, journals may apply for re-review 24 months later.
  • What is ERIC looking for in the automatic review at the end of the pause period? ERIC will conduct a full review of the journal and consider the two years of published content against the criteria set forth in the ERIC Selection Policy.
  • Can authors submit their article published in a “paused” journal via online submission during this period? No, authors must wait for a re-review of the journal to be conducted after the two-year pause. If the journal is selected again, the articles from the paused issues will be indexed in ERIC.

How State Education Agencies Can Leverage Their Regional Educational Laboratory to Support Students’ Academic, Social, and Mental Health Needs

(A Dear Colleague Letter sent to Chief State School Officers on February 23, 2023.)

Dear Colleague:

As state and local education agencies leaders reflect upon the successes and challenges of the 2022-2023 school year—and the opportunity that summer 2023 presents to further support students’ academic, social, and mental health needs—I am writing today to encourage you to take full advantage of the services offered by your Regional Educational Laboratory (REL).

The REL Program, sponsored by the U.S. Department of Education’s Institute of Education Sciences, supports educators and policymakers at the state and local levels in the use of data and evidence-based practices to improve student outcomes. All REL services are provided free of charge and are designed in partnership with state and local partners to meet their specific needs. Each REL is led by a Director with deep expertise in education policy, practice, and research who can help you navigate how best to leverage REL supports to address your state’s most pressing needs. A list of REL Directors, including their contact information, is attached.

Your REL can support a wide range of state and local initiatives. They include:

  • Analyzing student progress and outcome data (e.g., achievement, chronic absenteeism, graduation rate, English language proficiency) to understand the ongoing effects of the COVID-19 pandemic. Your REL can analyze longitudinal student data provided by the state or district partners to better understand the trajectory of student performance prior to the pandemic, during the pandemic, and today. When disaggregated by student group, school characteristics, or other relevant features, these analyses can support decision-makers in focusing resources, monitoring improvement, and adjusting implementation efforts. RELs Midwest and Mid-Atlantic recently provided similar services for their state and district-level partners.
  • Supporting the identification of existing, or the design of new, evidence-based practices to meet students’ academic, social, and mental health needs. Your REL can support state and local efforts to identify practices that prior evidence suggests can promote learning and development. REL Southeast recently published a review on the effectiveness of early literacy interventions across several domains in response to a request from partners regionwide. When high-quality evidence does not exist, or existing practices are not well-aligned to state or local needs, RELs can support efforts to design and pilot research-based innovations.
  • Coaching state and local education agency staff on the use of data to improve the ongoing implementation of education policies, programs, and practices. Your REL offers coaching and training services for state and local leaders on data-driven approaches to continuous quality improvement. These services are particularly beneficial when a program is relatively new to a state or district and leaders are focused on timely feedback to ensure an evidence-based practice is well-implemented at scale. REL Southwest recently supported the Oklahoma State Department of Education’s (OSDE) rollout of Oklahoma Excel, a data-driven and job-embedded professional development program for educators in participating districts. A 2-part video series provides background on the program, and the supports REL Southwest provided to OSDE staff who administer the program.
  • Evaluating the impact of state or local interventions on important student outcomes. Your REL can support the rigorous evaluation of well-implemented policies, programs, or practices to document those efforts’ impacts on important student outcomes. For example, a 2021 REL Northwest study examined the implementation and impact of full-day kindergarten in Oregon in light of a funding structure shift that incentivized districts to offer the programming. When a rigorous evaluation is not feasible, your REL can advise you on credible, alternative approaches to understanding the outcomes associated with a policy or program.
  • Coaching state or local education agency staff on the use of existing REL tools and resources. Through their work with state and local partners, RELs have developed a wide range of actionable resources designed to support the implementation of evidence-based practices. Your REL can coach state and local education agency staff on how to  customize and use tools developed elsewhere to meet your needs. Examples include REL Appalachia’s Community Math Night Facilitators’ Toolkit and REL Southeast’s Professional Learning Community on Emergent Literacy.
  • Providing expert guidance to senior state or local education agency leaders. Finally, your REL can leverage its network of internal and external experts to offer guidance on data- and evidence-driven approaches to addressing problems of policy and practice. This “Ask-an-Expert” service is available to senior leaders of both state and local education agencies. A recent REL Appalachia “Ask an Expert” response to a Tennessee-based partner shared best practices for administering and using data from Kindergarten readiness screeners.

REL Directors are routinely in contact with senior education agency leadership as part of their on-going work to better understand the kinds of supports that might benefit states in their region. However, if you or senior members of your leadership team have not yet had the opportunity to meet with your REL Director (or have not done so recently), please consider contacting them at your convenience. I am also glad to facilitate that connection at your request.

Finally, I would be remiss if I did not mention the critical relationship between your REL and the Regional Comprehensive Center (RCC) that serves your state. Sponsored by the Department’s Office of Elementary and Secondary Education, RCCs support state education agencies in their efforts to implement evidence-based policy and practice and realize the goals set in their Consolidated State Plans.

If you have any questions about the REL Program, please do not hesitate to contact me or a senior member of my team.

Sincerely,

Matthew Soldner
Commissioner, National Center for Education Evaluation and Regional Assistance
Matthew.Soldner@ed.gov 

Note: This blog reflects slight edits to the letter sent to Chief State School Officers. References to an attached brochure and a contact list for REL Program staff have been removed.

Leading with Evidence: Celebrating Evidence-Based Policymaking at the U.S. Department of Education

Matthew Soldner,

Commissioner, National Center for Education Evaluation and Regional Assistance &

Chief Evaluation Officer, U.S. Department of Education

(Remarks made on November 17, 2022, at Results for America’s 2022 Invest in What Works Federal Standards of Excellence events.)

If you walk into almost any kindergarten classroom today, you’ll almost always find a “Star Chart.” It’s usually displayed with pride next to the chalkboard in the front of the room. On it, the name of every student in the class. And next to each name, a long row of stars, each signifying a task earnestly mastered by a young learner. (More than forty years later, I can still remember the pride associated with finally getting a gold star for “can tie shoes,” which I was seriously delayed in getting due to an overabundance of shoes secured with Velcro in my youth.)

Today, I’m very proud to acknowledge that the U.S. Department of Education has received its own gold star. It was bestowed by Results for America, which advocates for the use of evidence in federal, state, and policymaking to improve outcomes for students, families, and communities, as part of its 2022 Federal Standards of Excellence program. Each year, participating federal agencies are evaluated on their progress in using high-quality evidence as a “north star” in policymaking. This year, Education was recognized alongside two other agencies – the Millennium Challenge Corporation and the U.S. Agency for International Development – as a top-scorer, earning the “Gold” designation.

ED’s national leadership in using evidence to inform policymaking has been a journey that now spans more than two decades. Its roots can be traced to the 107th Congress, which in 2001 reauthorized the Elementary and Secondary Education Act of 1965 as the No Child Left Behind Act and, in 2002, passed the Education Sciences Reform Act. The latter authorized the founding of a group toward which I’m somewhat partial: the Institute of Education Sciences (IES). (I would invite you to join my colleagues and me in celebrating #IESat20, now through mid-2023!) But no single event, and no distinct component of the Department, is individually responsible for our success building evidence about “what works” in education and putting that evidence to work to better serve learners, educators, and communities. I often say “evidence-building is a team sport at the Department of Education,” and it truly does take the commitment of talented professionals from across the organization to make it a reality.

This year, that team has been particularly busy. Department-wide, we have supported states, school districts, and institutions of higher education in their continued efforts to meet the challenge of pandemic recovery. Much of that work has focused on the use of evidence-based practices to accelerate learning for all students, making the most of historic investments in education such as the Coronavirus Aid, Relief, and Economic Security (CARES) and American Rescue Plan (ARP) Acts. Key partners in that work include IES’ Regional Educational Laboratories, operated by the National Center for Education Evaluation and Regional Assistance (NCEE);  the Office of Elementary and Secondary Education’s Comprehensive Centers; the Office of Special Education and Rehabilitative Services’ Technical Assistance and Research Centers; and the Office of Planning, Evaluation, and Policy Development’s Grant Policy Office.

Elsewhere in the Department, the emphasis has been on evidence-building. Here, IES has taken a particular leadership role. The National Center for Education Statistics’ (NCES’) School Pulse Panel is a critical new component in our evidence-building infrastructure. The Pulse allows us to more rapidly collect and report descriptive information about conditions on the ground in our nation’s schools, addressing topics from the extent of staffing shortages to the programs schools are offering to support learning acceleration. That and other information supports a vibrant research and development infrastructure, led by the National Centers for Education Research (NCER) and Special Education Research (NCSER). In addition to their regular education grant programs, both Centers ran special competitions in Fiscal Year 2022 specifically designed to support pandemic recovery, including those aimed at better leveraging longitudinal data to support state recovery policymaking and building evidence about the approaches states and districts used to address the pandemic, and, when possible, their effectiveness.  

In Fiscal Year 2023, more good work is already underway.

First, I would be remiss if I did not acknowledge an important investment this most recent Congress has made in the evidence-building work of the Department: authority as part of the Consolidated Appropriations Act of 2022 to reserve up to one-half of one percent from selected programs authorized by the Higher Education Act of 1965, as amended, to support high-quality research and evaluation related to the Department’s postsecondary programs. A similar set-aside for the Department’s K-12 programs dramatically catalyzed our ability to build and use evidence there – and I have every confidence this new authority, if continued, would do the same for our postsecondary portfolio.

Second, and consistent with my belief that “evidence building is a team sport,” I want to take a moment to encourage you to join the team! As an initial step, I’d like to invite you to join me and special guests from the Department in our new webinar series creatively entitled “Evidence-based Policymaking at ED: Introducing the U.S. Department of Education’s Inaugural Learning Agenda.” Across three installments, we’ll discuss the Department’s evidence-building priorities in three areas: the educator workforce; meeting students’ social, emotional, and academic needs; and increasing postsecondary value. In each, we’ll dig a bit deeper into each topic and its nexus with an equitable recovery from the COVID-19 pandemic.

I hope – but cannot promise – that Education will, this time next year, report that we’ve earned another gold star for building and using evidence in service of smart policymaking. What I can promise is that, because both educators and education policymakers will continue to need high-quality evidence to do their best work on behalf of the nation’s learners, we will do our best to help them meet challenges both old and new. Thanks to Results for America for today’s recognition, and to all those who support our nation’s students, educators, and communities every day.

 

How the 2017-2022 Cohort of RELs Supported the Use of Evidence in Education

Three adults discuss a chart that is displayed on a laptop.

This winter is a special season that comes along once every five years for the Regional Educational Laboratory (REL) program at IES. It’s a winter when the REL team manages the end of five-year REL contracts and oversees the launch of nine new REL contracts.[i]  During this exciting time, we actively reflect on the successes and lessons of the previous contracts—the 2017-2022 REL cohort—and channel those reflections into our work with the 2022-2027 REL cohort. 

As I collaborate with the REL team on the new RELs, I want to share some of the successes of the RELs that finished their work early this year. We expect the new RELs to build on these successes and to engage in new, innovative work that I will discuss in a future blog.

As we look back at the large body of work that the last cohort of RELs produced, I want to share some exciting results. Over three-quarters of participants in REL researcher-practitioner partnerships who responded to the REL Stakeholder Feedback Survey (SFS) reported that they used or were in the process of using the research or data that they learned about through the REL partnerships. On average across the last three years, an additional 17 percent reported that they were making plans to use research or data presented by the REL:

Image of a chart entitled “Responses to REL Partnership Stakeholder Feedback Survey (SFS).” The chart shows that in 2019, 77 percent of 695 respondents reported that they used or were in the process of using the research data they learned through REL partnerships, 19 percent said they were making plans to use the research, and 4 percent said they had no plans to use the research; in 2020, 81 percent of 397 respondents reported that they used or were in the process of using the research data they learned through REL partnerships, 17 percent said they were making plans to use the research, and 2 percent said they had no plans to use the research; and in 2021, 82 percent of 582 respondents reported that they used or were in the process of using the research data they learned through REL partnerships, 15 percent said they were making plans to use the research, and 3 percent said they had no plans to use the research.

While these survey results are promising, I want to provide a more vivid picture of how the RELs partnered with stakeholders to use evidence to improve teaching and learning. Read on to learn how REL work has been integral to education policy and practice across the country.

REL Mid-Atlantic and REL Southeast both engaged in projects that supported efforts to safely educate students during the pandemic:

  • In Pennsylvania, REL Mid-Atlantic helped the Pennsylvania Department of Education (PDE) provide evidence to inform the reopening of schools in the state during the first year of the COVID-19 pandemic. REL Mid-Atlantic worked with PDE to produce an extensive memo that included (1) a rapid review of existing evidence on public-health and educational issues relevant to the reopening of schools, (2) findings from interviews with a cross-section of stakeholders from across Pennsylvania to assess concerns and challenges related to reopening, and (3) agent-based modeling simulations of the potential spread of COVID-19 under alternative approaches to reopening schools.  The two largest school districts in the state—the School District of Philadelphia and the Pittsburgh Public School District—along with at least 25 other school districts and one Catholic archdiocese drew on the findings in the memo to make decisions about whether and how to reopen schools. 
  • Shortly after two of four of REL Southeast's teacher guides were released in early 2020, schools across the country shut down due to the COVID-19 pandemic. The REL realized that the content of the guides—originally created to support teachers in working with families to facilitate their children’s literacy development—would be immediately useful to parents across the county who were suddenly thrust into the role of teacher for their children at home. The content of the guides was based on the What Works Clearinghouse Practice Guide: Foundational Skills to Support Reading for Understanding in Kindergarten Through 3rd Grade.

REL Southeast made all the content, which included videos and activities, available on the REL website so that parents could easily access them and use them to support their children during that difficult time.The landing page for the content—Supporting Your Child's Reading at Home—has been visited nearly 130,000 times since April of 2020. And landing pages for the four guides for teachers—A Kindergarten Teacher's Guide, A First Grade Teacher's Guide, A Second Grade Teacher's Guide and A Third Grade Teacher's Guide—have each been accessed between 1,300 and 7,500 times since their release. 

REL West and REL Midwest both worked with states in their regions to support student health and the need to identify and recruit more teachers.  These topics proved to be particularly  important as a result of the pandemic:

  • Robla Elementary School District (RESD) and several other districts in California’s Central Valley began offering telemedicine services during the 2017/18 school year as part of a broader “whole-child” strategy for improving student health, well-being, and attendance. Telemedicine is the remote evaluation, diagnosis and treatment of patients using telecommunications technology. RESD contracted with and paid Hazel Health, a telemedicine provider that operates virtual health clinics in school settings.  The telemedicine visits were free to students and families and did not require scheduled appointments. To learn more about the implementation of the program and whether it was associated with students staying in school throughout the day, RESD enlisted REL West for assistance.

REL West's study of the telemedicine services found that districtwide, a little over one-quarter of students used the services at least once over two years, with nine percent of students using telemedicine multiple times. Non-communicable physical illnesses/conditions such as stomach aches, headaches, allergies, and asthma were consistently the most common reason for school-based telemedicine visits across the two years of implementation. Ninety-four percent of all telemedicine visits resulted in students returning to class and receiving, on average, three more hours of instruction in the school day. Approximately 39 percent of Black students used telemedicine services compared with 17 percent of Asian students. Due to these findings, the district decided to continue with the program. The telemedicine provider is working to identify possible reasons for the differences in use by different student groups to ensure that all students are comfortable accessing the services.

  • Even before the COVID-19 pandemic, Michigan was experiencing teacher shortages in several subjects and geographic areas. This led Michigan members of the REL Midwest Alliance for Teacher Preparation to partner with the REL to examine nonteaching-certified teachers’ reasons for not teaching and incentives that could motivate them to return to the classroom. The REL Midwest study found that salary and certification/recertification requirements were among the most frequent barriers to teachers entering or remaining in the teaching profession.

As a result, the Michigan Department of Education launched the “Welcome Back Proud Michigan Educator” campaign, which seeks to recruit nonteaching educators into the teacher workforce. The first wave of the campaign, which began in April 2021, recruited educators with expired teaching certificates by reducing—and in some cases eliminating—professional learning requirements for recertification. The second wave, which began in October 2021, recruited teachers who had a valid certificate but were not teaching in public schools. As of January 2022, 218 educators have been recertified or issued a teaching permit, and 27 educators are in the pipeline to reinstate their teaching credentials. Of those with valid certificates, 123 educators started in a teaching position in fall 2021 and an additional 244 educators took a non-teaching assignment, such as day-to-day substitute teaching.

Concerns about the lack of equity in educational opportunities and in disciplinary practices led stakeholders to partner with REL Appalachia and REL Northwest:

  • Throughout the country, students are often encouraged to study Algebra I in middle school so that they can take more advanced math courses in high school and can graduate with a college-ready diploma. Concerned that economically disadvantaged students and English learners might be taking Algebra I later than their peers and earning college preparatory diplomas at lower rates than other students, Virginia’s Department of Education asked REL Appalachia for assistance analyzing the state’s data. The REL researchers found that the Department of Education’s hypotheses were correct. They found that, among all 5th graders rated as “advanced proficient” on the state’s math assessment, economically disadvantaged and English learner students were less likely take Algebra before 9th grade and less likely to earn a college preparatory diploma. As a result of these findings, the Virginia Department of Education asked the REL to work with school districts across the state to analyze data to identify student course-taking patterns and to further examine district-level policies and practices that may be contributing to the inequitable course-taking patterns and outcomes. 
  • REL Northwest undertook several projects with the Equity in School Discipline (ESD) collaborative: a cross-state collaborative of districts, state education agencies, community-based organizations, and institutions of higher education in Oregon and Washington committed to increasing equity in school climate and discipline policies and practices. ESD sought to reduce the use of exclusionary discipline practices and to eliminate disproportionately high rates of exclusion for students who identify as American Indian, Black, and Hispanic. REL Northwest and ESD district leaders in four districts partnered to co-design and pilot training materials to help school and district teams increase equity in school discipline practices. REL Northwest also produced a tool so other districts and states can identify discipline disproportionality.

REL Pacific helped to make a language program more evidence-based:

  • Recognizing the role of the school in sustaining Kosrae’s cultural and linguistic heritage and preparing students for the global world, Kosrae Department of Education (KDOE) leaders reached out to REL Pacific for support in creating a new language immersion policy and program that better supports the goal of building student proficiency in both Kosraean and English. REL Pacific supported KDOE by providing coaching on the research behind effective bilingual education models, policy implementation frameworks, and language assessments. REL Pacific and Region 18 Comprehensive Center (RC18) subsequently collaborated to provide complementary supports to ensure KDOE had increased capacity to implement its bilingual language policy in schools across the island. As REL Pacific continued support in best practices in bilingual instruction, classroom observation, and teacher professional learning, RC18 provided supports such as bilingual materials development and financing options for the new policy. KDOE began piloting the new policy in two elementary schools in the fall of 2021.

REL Central supported Nebraska by providing evidence-based resources and training to support the implementation of new legislation:

  • In 2018, the Nebraska Reading Improvement Act was passed to decrease the number of struggling readers in grade 3 across the state. The Nebraska State Board of Education (NSBE) and the Nebraska Department of Education enlisted REL Central’s support in providing the state’s elementary school teachers with evidence-based practices for the teaching of reading. To meet this need, REL Central reviewed strategies in eight What Works Clearinghouse practice guides on reading, writing, and literacy instruction and distilled the information into summary documents that were aligned with the state’s initiative. Each document is featured on NDE’s NebraskaREADS website and each describes a practice guide recommendation, how it should be implemented, and discusses the appropriate grade level or target student population (for example, English learners). REL Central also provided trainings to support regional education service unit staff and school-based educators in reviewing, selecting, and testing evidence-based reading strategies.

Finally, through applied research studies, REL Northeast and Islands and REL Southwest helped education leaders answer important questions about whether students in certain localities had equitable access to important services. These studies informed leaders’ decisions about state programs or indicators:

  • In an effort to increase the percentage of children ready for kindergarten, Vermont passed Act 166 in 2014 that provided access to high-quality prekindergarten (pre-K) for all 3- and 4-year-olds and for 5-year-olds not yet in kindergarten. As universal pre-K began in the 2016/17 school year, officials were concerned about unequal distribution and availability of high-quality pre-K programs across the state. The Vermont Agency of Education, the Agency of Human Services’ Department for Children and Families, and Building Bright Futures (Vermont’s early childhood advisory council) participated in the Vermont Universal PreK Research Partnership with REL Northeast & Islands to answer these important questions. Through one study, the REL found that although the majority of pre-K children were enrolled in the highest quality programs, some children had less access to high quality programs in their home districts. These findings led the Vermont legislature to maintain a provision that allows families to enroll their children in programs outside their home district.
  • Texas House Bill 3 (HB3), a comprehensive reform of the state’s school finance system passed in 2019, established a college, career, and military readiness outcomes bonus, which provides extra funding to districts for each annual graduate demonstrating college, career, or military readiness under the state accountability system. Leaders at the Texas Education Agency (TEA) were concerned that it may be hard for small and rural districts to demonstrate career readiness through the required accountability measure. Through a partnership with TEA, REL Southwest conducted a study that found that there were no substantive differences by district size or locale with respect to the percentage of students meeting the career readiness standard. Further, the study found that students who fell into two of the alternative career readiness options—CTE completers and work-based learners—had higher rates of college enrollment than graduates who met the existing career readiness accountability standard. The study also indicated that CTE completers had higher rates of either college persistence or of credential attainment after high school than graduates who met the existing career readiness accountability standard. These findings led the Commissioner of Education to recommend, and the Texas legislature to create, a new measure of career readiness in the state accountability system that met the needs of the districts across the state.

From these examples, one takeaway is clear: REL work can make a difference. RELs supported educators’ and policymakers’ efforts to improve educational programs, policies, and outcomes through use of research and evidence-based practice between 2017 and 2022. The new RELs will continue this work and, as I will write about in a future blog, they will also undertake some new types of projects. Until then, please visit the new REL website or reach out to me at Elizabeth.Eisner@ed.gov  if you have questions about the REL program and how it can help your community.

Liz Eisner is the associate commissioner of the Knowledge Use Division at the National Center for Education Evaluation and Regional Assistance


[i] One REL contract—REL Southwest (REL SW)—is on a different schedule. The current REL SW contract ends in late November of 2022 and the next REL SW contract will begin the day after the current contract ends. The contracts that just ended were the 2017-2022 contracts and the contracts that just started are the 2022-2027 contracts.

Is believing in yourself enough? Growth mindset and social belonging interventions for postsecondary students

The WWC recently reviewed the strength of evidence for two types of interventions designed to help students succeed in college: one report focuses on growth mindset interventions and another on social belonging. The WWC found that (1) neither type of intervention had a discernible effect on full-time college enrollment, (2) social belonging interventions had mixed effects on progressing in college and academic achievement, and (3) growth mindset interventions had potentially positive effects on academic achievement. We asked Greg Walton, an Associate Professor at Stanford University, IES-funded researcher, and expert on these kinds of interventions, to discuss what college faculty, administrators, and students should make of these findings.  

Can you walk through how growth mindset interventions and social belonging interventions with postsecondary students work? Were the interventions reviewed by the WWC typical interventions in this space?

Growth mindset interventions focus on the underlying “implicit” beliefs students have about the nature of intelligence: Is intelligence fixed or can it grow? These beliefs inform how students make sense of everyday academic challenges in school. If you think that intelligence is fixed, that you either have it or you don’t, then a setback like a poor grade can seem to be evidence that you don’t have what it takes. That can make students avoid academic challenges, withdraw, and ultimately learn and achieve less. Growth mindset interventions offer students the view that intelligence can grow with effort, hard work, good strategies, and help from others. The theory is that that mindset can help students see setbacks simply as evidence that they haven’t learned the material yet, or that their strategies haven’t been successful yet, and thus to sustain their efforts. These interventions typically start by sharing information from neuroscience about how the brain grows “like a muscle” during learning, especially when students work on challenging material. Then students might read stories from older students who used a growth mindset to persist through challenges. Finally, they may be asked to describe this idea to help younger students struggling in school, a technique termed “saying-is-believing.” That makes the experience active rather than passive and positions students as benefactors rather than beneficiaries, which would be potentially stigmatizing.

Social-belonging interventions target “belonging uncertainty,” a persistent doubt students can feel about whether “people like me” can belong in a school setting. This doubt arises most strongly for people who belong to groups that have historically faced exclusion in school settings, negative stereotypes that pose them as less able and less deserving of educational opportunities, or who are underrepresented in a school context. When students experience this doubt, everyday challenges such as feeling lonely, being excluded, or getting critical feedback can seem like evidence that they don’t belong in general. Social-belonging interventions share stories from older students who describe how they worried at first about whether they belonged in a new school and how these worries dissipated with time as they developed friendships and study partners, joined student groups, and formed mentor relationships. Belonging interventions offer students the view that it’s normal to worry about belonging at first in a new school but this gets better with time. Like growth mindset interventions, belonging interventions use written exercises to give students the opportunity to reflect on the intervention message and advocate for it to younger students. The theory is that this message can help students sustain a sense of belonging and stay engaged in school even when they face challenges, and that that helps students develop friendships and mentor relationships that support higher rates of achievement.

Social-belonging interventions were designed specifically to address circumstances in which people face underrepresentation or negative stereotypes in school. Even if all students have reasons to worry whether they belong in school, only some students have reason to question whether “people like me” belong. I am a White person whose parents both graduated from college. So, when I went to college, I felt homesick but I didn’t wonder whether “people like me” could belong.

That said, belonging concerns are felt by almost everyone, and in some cases belonging interventions have produced main effects (benefits for all students) rather than interactions predicated on group identity (e.g., Borman et al., 2019 for evidence from students in grade 6). However, most trials find greater benefits for students who face underrepresentation or negative stereotypes in specific settings. One study found that women in more gender-diverse engineering majors (averaging 33% women) showed no achievement gap with men in the first year and no benefit from a belonging intervention. But women in male-dominated majors (averaging 10% women) showed a large achievement gap in first year performance, but that gap was closed by the intervention (Walton et al., 2015; see also Binning et al., 2020) [Editor’s note: These two latter studies did not meet WWC standards for internal validity. Although this suggests caution in drawing conclusions from the studies, failing to meet WWC standards does not imply that an intervention is ineffective.]

Taken together, a fixed-mindset of intelligence and belonging uncertainty can be like a toxic tornado for students, swirling into each other and creating cascading self-doubt. I’m describing these interventions separately because they grew up independently in the literature, and the WWC’s two reports look at each separately. But for students, they are often experienced together.

It’s also important to state that, although the interventions reviewed by the WWC are typical of those conducted with postsecondary students, these are highly active areas with new trials reported regularly. Studies have explored new populations and college contexts (e.g., Murphy et al., 2020) and are increasingly focused on identifying boundary conditions that determine where we should and should not predict effects (see Bryan, Tipton, & Yeager, 2020). It is also noteworthy how few studies have examined the critical question of progress in college (3 in each report). We need much more research here, exploring effectiveness, implementation strategies, and boundary conditions. Further, research is increasingly complementing direct-to-student interventions by exploring how we can support practices in school that support growth mindset and belonging (Murphy et al., 2021). For example, recent research shows that highlighting pro-diversity peer norms—namely that most students endorse diversity—can facilitate more inclusive treatment among college students and, in turn, reduce achievement gaps between privileged and marginalized students (Murrar et al., 2020).

What are the key components that are needed for a social belonging or growth mindset intervention to have a good chance of working? What elements need to be in place to help students improve academically or to stay enrolled in college?

I would distinguish two layers of this question.

One layer is what does it take for a discrete exercise focused on belonging or growth-mindset—such as the focus of the trials reviewed by WWC—to help students. In general, we should consider what, how, when, and where.

What is it you want to offer students? It should give students an authentic and adaptive way to make sense of common challenges they face, a way of thinking they can use to achieve their goals in college. Simple exhortations such as, “I know you can do it” or “You belong!” do not effectively impart a growth mindset or a sense of belonging, as Carol Dweck and I have written. Instead, it is useful to use high-quality materials developed and validated in research. Examples of materials available online are here and here.

How will you convey this? The goal of these interventions is to address foundational beliefs students have about school, such as “Can I do it?” and “Can people like me belong here?” It’s not to do something else, like to build a skill. That means the experience need not take long—typically, interventions last 30-60 minutes—but it should be immersive and interactive. You want students to deeply reflect on the ideas you present and connect these ideas to their lived experience.

That said, the more you can implement approaches that are scalable within an institutional context the more students you can potentially help. That’s one reason recent trials that reach large samples have focused on online modules (e.g., LaCosse et al., 2020; Yeager, Walton, & Brady et al., 2016). Students can log-on individually and complete materials at near-zero marginal cost. However, these approaches also have challenges, as online modules may not be as engrossing as in-person experiences. As we have moved from delivering these interventions in one-on-one, in-person experiences to larger studies with materials delivered online, we have found that students spend less time on the same materials and write less in response to prompts. Another alternative is having students meet in-person in groups to participate in these interventions or discuss their content (see Binning et al., 2020; Murphy et al., 2020), but that may be more difficult to implement on a large scale. So, there can be trade-offs between reaching scale and creating deep and impactful experiences.

When should you do this? In general, it is valuable if an intervention happens earlier rather than later, so it can alter trajectories going forward. However, it may be optimal to deliver interventions soon after students have encountered some challenges, but before they have taken steps in response to those challenges that are hard to reverse (e.g., dropping out). In general, social-psychological interventions are more sensitive to timing than to dosage. Growth mindset and belonging interventions have been delivered from the summer before college (Yeager, Walton, Brady, et al., 2016), to the first academic term (Walton et al., 2015), to the second (Walton & Cohen, 2011).

Where should you deliver interventions? This brings us to the second layer. So far, I’ve addressed the first layer, where you are focused on a discrete experience or set of experiences. But the second layer is that, growth mindset and belonging interventions will be most effective in contexts in which (1) the message offered is legitimate and authentic (locally true) and (2) students have real opportunities to get academic support and to develop a sense of belonging. In the end, to produce the most robust change, we must create cultures in schools in which adaptive ideas about ability and belonging are normal and reinforced. There are many ways that institutions signal to students, even inadvertently, messages about the nature of intelligence and who belongs. In welcoming a new class to campus, do we extol the past achievements of a few, which may only heighten imposter syndrome among everyone else? Can we instead talk about what students can do in the future and who they can become? In welcoming students to class, do faculty communicate that they expect to weed out large numbers of students? Or do they design assignments and evaluations to support students’ learning and growth (Canning et al., 2019)? Another question involves how well colleges foster opportunities for students to develop in-group pride and identity. Tiffany Brannon at UCLA finds that African American students do better in college when they have more opportunities to participate in events that celebrate and explore Black culture (Brannon & Lin, 2021). Some resources to help researchers and practitioners create cultures of growth and belonging for all students are available at the Student Experience Project, co-led by the College Transition Collaborative (https://collegetransitioncollaborative.org/student-experience/).

Recently, you and your colleagues have distinguished between people with different characteristics - and environments with different characteristics. You’ve argued that researchers should be looking more closely at the contexts, or what you’ve called “psychological affordances” in which these interventions might have different effects. Why is this work important? Why should educators be paying attention?

Social-psychological interventions operate within complex systems. Those systems invariably determine the specific effect any intervention has. To understand this, my colleagues and I have found it useful to consider the affordances of a school context: What does a context make possible (Walton & Yeager, 2020)? For instance, no psychological intervention will help English-language speakers learn Chinese if they aren’t receiving instruction in Chinese.

We distinguish two kinds of affordances. One is structural: What is it that different institutions make possible for students to do? As an example, in a forthcoming study, Shannon Brady, Parker Goyer, David Yeager, and I tracked college outcomes of students randomly assigned to a social belonging intervention or a control condition at the end of high school. The intervention raised the rate of bachelor’s degree completion for students who first enrolled in more selective 4-year institutions from 26% to 43%. These are institutions that tend to have higher retention and graduation rates and tend to spend more per student on instruction and student services than less selective 4-year institutions. They thus afford higher 4-year completion rates. At the same time, the same belonging intervention had no effect on bachelor’s degree completion rates for students who first enrolled in less selective 4-year institutions.

The second kind of affordance is psychological: What is it that students can believe in a school context? Does the cultural context in which an intervention is delivered one in which the way of thinking offered by the intervention can take hold and thrive? Or is it one that makes that way of thinking illegitimate, inauthentic, or not useful?  A large-scale social-belonging intervention delivered online to students in 21 diverse colleges and universities increased first-year full-time completion rates for students from historically underperforming groups, but only in colleges that afforded, or fostered, a sense of belonging to members of those groups. Let’s break this down: In some college contexts, students from historically underperforming groups (who were not exposed to the intervention) realized a high sense of belonging by the end of the first year. Here the belonging message was “locally true” (true here, for people like me). Although we don’t know exactly why this was the case, presumably in these schools students from the given group had more opportunities to develop friendships, to join student groups, and to form meaningful relationships with instructors. In other colleges, students did not attain this high sense of belonging by the end of the first year. Only in the first case did the belonging intervention raise first-year completion rates (Walton, Murphy et al., in prep; described in Walton & Yeager, 2020).

In both cases, the belonging intervention helped students take advantage of opportunities available to them, whether to graduate or to belong. An important implication is that it may be necessary to address both students’ beliefs and whether contexts support more positive beliefs. That’s helpful, because it gives us a precise way to think about how to make contexts more supportive: To what extent do they make adaptive beliefs about intelligence and belonging legitimate and authentic and, if they do not, what can we do about this?

It sounds like you’re saying postsecondary leaders who want to foster greater student success and reduce gaps in retention and academic performance may want to consider these kinds of interventions, in part because they are relatively inexpensive to deliver to large numbers of students. But they should also consider how hospitable their campus is to students who might initially struggle in college.

For example, to reinforce a growth mindset, universities need to make academic support resources truly accessible; to reinforce a sense of belonging, universities might look for multiple ways to communicate that successful students of all kinds of backgrounds have initially experienced self-doubt, and that feeling like you don’t belong is a fairly normal and temporary part of adjusting to college.

That’s right. Growth mindset and belonging are about both student beliefs or ways of thinking and institutional practices—either alone may not be enough. So, to support a growth mindset, institutions should both (1) convey that all students can learn and grow with effort, good strategies, and support from others and (2) back that up by creating learning environments designed to support growth, including adequate academic supports, and classes that focus on fostering growth rather than identifying who is allegedly smart and who is not. To support belonging, institutions should (1) acknowledge that nearly all new college students worry at first about whether they belong, that this is normal and improves with time and (2) create classroom and out-of-classroom environments in which all of the diverse students we serve can develop strong friendships and mentoring relationships and find communities in which they belong.

Thanks very much, Greg.

 

Read the WWC’s summary of evidence for these interventions in the Growth Mindset Intervention Report and the Social Belonging Intervention Report. Find related resources at the The College Transition Collaborative (https://collegetransitioncollaborative.org/) or the Project for Education Research That Scales (https://www.perts.net/)

 

Carter Epstein, Senior Associate at Abt Associates, produced this blog with Greg Walton, Associate Professor of Psychology at Stanford University.

 

Note: The discussion above reflects the opinions of Greg Walton and does not necessarily reflect the opinions of the Institute of Education Sciences or the What Works Clearinghouse. Some of the studies cited above have not been reviewed by the What Works Clearinghouse.

 

REFERENCES

Binning, K.R., Kaufmann, N., McGreevy, E.M., Fotuhi, O., Chen, S., Marshman, E., Kalender, Z.Y., Limeri, L., Betancur, L., & Singh, C. (2020). Changing social contexts to foster equity in college science courses: An ecological-belonging intervention. Psychological Science, 31,1059-1070. https://doi.org/10.1177/0956797620929984

Borman, G.D., Rozek, C.S., Pyne, J., & Hanselman, P. (2019). Reappraising academic and social adversity improves middle school students’ academic achievement, behavior, and well-being. Proceedings of the National Academy of Sciences 116 (33), 16286-16291. https://doi.org/10.1073/pnas.1820317116

Brady, S. T., Walton, G. M., Goyer, J. P., & Yeager, D. S. (in prep). [Where does a brief belonging intervention increase the attainment of a college degree? The role of institutional affordances.] Manuscript in preparation.

Bryan, C. J., Tipton, E., & Yeager, D. S. (2021). Behavioural science is unlikely to change the world without a heterogeneity revolution. Nature human behaviour, 5(8), 980–989. https://doi.org/10.1038/s41562-021-01143-3

 Bryk, A. S., Grunow, A., Gomez, L. M., & LeMahieu, P. G. (2015). Learning to improve: How America’s schools can get better at getting better.  Harvard Education Press.

Canning, E. A., Muenks, K. ,Green, D.J., & Murphy, M.C. (2019). STEM faculty who believe ability is fixed have larger racial achievement gaps and inspire less student motivation in their classes. Science Advances, 5(2). https://www.science.org/doi/10.1126/sciadv.aau4734 

Dweck, C. (2016, January 11). Recognizing and overcoming false growth mindset. Edutopia. https://www.edutopia.org/blog/recognizing-overcoming-false-growth-mindset-carol-dweck

Murphy, M.C., Fryberg, S.A., Brady, L.M, Canning, E.A., & Hecht, C.A. ( 2021, August 25). Global Mindset Initiative Paper 1: Growth mindset cultures and teacher practices. https://ssrn.com/abstract=3911594

Murrar, S., Campbell, M.R. & Brauer, M. (2020). Exposure to peers’ pro-diversity attitudes increases inclusion and reduces the achievement gap. Nature Human Behavior 4, 889–897 . https://doi.org/10.1038/s41562-020-0899-5

Walton, G.M. (2021, November 9). Stop telling students, “You belong!” Three ways to make a sense of belonging real and valuable. Education Week. https://www.edweek.org/leadership/opinion-stop-telling-students-you-belong/2021/11

Walton, G. M., Logel, C., Peach, J. M., Spencer, S. J., & Zanna, M. P. (2015). Two brief interventions to mitigate a “chilly climate” transform women’s experience, relationships, and achievement in engineering. Journal of Educational Psychology, 107(2), 468–485. https://eric.ed.gov/?id=EJ1061905

Walton, G. M., Murphy, M. C., Logel, C., Yeager, D. S., Goyer, J. P., Brady, S. T., . . . Krol, N. (in preparation). Where and with whom does a brief social-belonging intervention raise college achievement? Manuscript in preparation.

Walton, G. M. & Yeager, D. S. (2020). Seed and soil: Psychological affordances in contexts help to explain where wise interventions succeed or fail. Current Directions in Psychological Science, 29, 219-226. http://gregorywalton-stanford.weebly.com/uploads/4/9/4/4/49448111/waltonyeager_2020.pdf

Yeager, D. S., Walton, G. M., Brady, S. T., Akcinar, E. N., Paunesku, D., Keane, L., Kamentz, D., Ritter, G., Duckworth, A. L., Urstein, R., Gomez, E. M., Markus, H. R., Cohen, G. L., & Dweck, C. S. (2016). Teaching a lay theory before college narrows achievement gaps at scale. Proceedings of the National Academy of Sciences of the United States of America, 113(24), E3341-E3348. https://doi.org/10.1073/pnas.1524360113