NCEE Blog

National Center for Education Evaluation and Regional Assistance

Meeting the Moment: The Role of Evidence-Based Practice in Innovation and Improvement

On March 5, 2024, the U.S. Department of Education convened state education agency (SEA) leaders and their teams to an event called “Meeting the Moment: How State Leaders are Using Innovation for Impact.” As part of this meeting, my colleagues and I presented IES initiatives that support the use of evidence-based practices in states and districts. Below are my remarks that introduce this important work.

Thank you – and good morning, everyone. It truly is a privilege to be with you today.

If you don’t know IES, we are the Department’s independent research, statistics, and evaluation office. Perhaps many of you know us better by two of our signature programs: the What Works Clearinghouse and the Regional Educational Labs.

I’m excited to be here to learn alongside each of you as you share the good work your states are doing.

In just a few minutes, we’re going to hear from my IES colleagues about work they’re doing, and resources they’re creating, on improving teaching and learning. I’m hopeful that you’ll see they can be used across many of the initiatives you’re pioneering.

As you might imagine, coming from an outfit named IES, what they’ll be sharing falls into the category of “evidence-based practices.” That is, practices that high-quality research has shown improved student outcomes in the past, or at the very least has suggested are likely to do so in the future.

Before they do, I want to talk briefly about evidence-based practice more generally. In this cone of silence that is a ballroom filled with 250 people, I’m going to offer one statement that might be a little provocative, and then I’m going to make an ask of each of you.

Here’s the provocative bit: Increasingly, I worry that “evidence-based practice” is becoming a buzzword. And that, like many buzzwords, people are translating it into “yada, yada, yada” when they hear the phrase. Or, worse, the phrase “evidence-based practice” is evoking skepticism such that people dismiss the concept when they hear it. Often, this is paired with concerns that the available evidence is misaligned to what educators and policymakers most need, or that it doesn’t feel like a fit to their context.  

Unfortunately, not leaning into high-quality evidence is, I think, a huge risk for us all.

It’s my absolute belief that, as a nation, we will not reach the goal of every student having the opportunity to achieve their full potential and then realizing that potential, and we will not ensure that every educator has the preparation and support to do their best work and then see that work done, without a sustained commitment to evidence-based practice. Without a commitment to supporting high-quality evidence use for the long-haul and a commitment to learn from that use by building high-quality evidence.

Absent a belief in, the use of, and efforts to sustain evidence-based practice, I do not believe we will “meet the moment” or “raise the bar” as the Secretary has just challenged us to do.  

This brings me to my ask of you. Here it is: use the high-quality evidence like that which my colleagues are about to show you to ground your initiatives, to support the professional development of the educators in your state, and to make smart policy. Indeed, I know many of you already are.

But as you do expect, dare I say demand (in a collegial way), more and better from the process of evidence building and use so that it is truly meeting your needs and the needs of the districts and communities you serve. You can do so in at least five ways:

  • Expect that you should be a full partner in the process of identifying the issues around which you need more evidence;
  • Expect that as we build new evidence we do so together, and with the authentic engagement of the communities that we hope to benefit;
  • Expect that, together, we find ways to make meaning of what we’re learning, and then communicating those learnings, in ways that aren’t just “right” when viewed through the lens of rigorous research but also “rich” in terms of the depth of content and the nuance needed by those who we expect to use that evidence;
  • Expect that we’re collaborating to place what we’ve learned into the hands of educators, policymakers, and those who support them ... and in ways they can most effectively use those learnings; and finally
  • Expect that you will be able to get the assistance you need implementing and sustaining evidence-based practices—and, if you’re willing, get assistance building evidence about your own programs.

I believe unfailingly in the potential of evidence-based practice, and of our work together, to transform students’ lives. As a leader of evidence builders, let me say we will expect these same five things of ourselves in service of that belief becoming a reality. We would welcome accountability from you, as users of that evidence, as we move forward.

With that, let me cede the floor to my colleagues. Thank you again for the opportunity to be here.

Have questions about these remarks? Please email me at matthew.soldner@ed.gov.

Changes to the ERIC Selection Policy

ERIC’s selection policy guides the types of materials it catalogs. It gives the ERIC team guidelines to determine if a source is a good fit for the ERIC collection. On a periodic basis, the team reviews the selection policy to ensure it is transparent and accurate. We often find that we have developed working policies to supplement the official selection policy and use the review process to formalize our processes.

As part of the latest update, ERIC proposes four major changes: (1) improvements in providing public access to federally funded work, (2) periodically re-reviewing all sources, (3) re-prioritization of international content, and (4) changes to online submission. ERIC is asking the community to provide feedback on these changes to ensure the policy is clear and effective. To learn more about the ERIC Selection Policy, please access the draft policy and watch a recorded webinar explaining changes. Please submit all questions and feedback to ericrequests@ed.gov by March 11, 2024.

Embracing new public access policies

The Institute of Education Sciences (IES) recently released a new plan to improve public access to federally funded articles. This is part of a government-wide initiative to ensure that taxpayers can access the research that they funded. This policy expands upon the 2012 public access policy that required all IES funded research be available to the public via ERIC. There are several key elements of the policy, listed below, that will impact how federally funded research will be displayed in ERIC.

First, all work funded under this new policy will be available in a machine-readable format. This means that the text and figures in any work will be displayed as website text. This will enable users to access the article without downloading the PDF. Additionally, data scientists will be able to bulk download and use the full text for their analysis. It will also make ERIC easier to use on mobile and tablets.

Second, all work will be in ERIC shortly after becoming publicly available. Under the current public access policy, the full text becomes available 12 months after the official publication date. The new public access plan states that awardees must submit the full text to ERIC immediately after the article is made publicly available (that is, when it is released as an “online first” article). This means the public will get access to the full text approximately 18 months earlier than under the current policy.

Lastly, ERIC will be providing increased metadata for these articles. All awardees must have an ORCID iD and author affiliation linked in their records to provide transparency on who authored the research. There will be a DOI permalink to all articles to ensure that users can find the articles in perpetuity. We will change the way that we link to grant funding information to be transparent about what IES is funding. And lastly, all work will have a link to the underlying data. This will help users make connections about who funded the work and for what purposes.

To implement the new policy, ERIC is changing how we catalog grantee submissions. Currently ERIC has two entries for many articles: the journal version of the article and the grantee submission record. Going forward, we will catalog the full text article and with increased metadata under the journal’s source name. There will only be one accession number for the article. Procedurally, this means that we will be updating many articles after they are formally published in a journal and have complete metadata. Awardee articles in sources not regularly cataloged in ERIC will still be cataloged under the grantee submission source name.

Re-review process

The current selection policy for ERIC states that once a source is approved for ERIC, it will remain in ERIC as long as it continues to publish content. Historically, the majority of our sources continue to publish content that meets ERIC’s standards and mission and remains a good fit for ERIC. However, for other sources we have seen a dramatic change in scope, quality, or quantity of articles.

In this new selection policy, we will re-review all new sources three years after initial acceptance in ERIC. After the initial re-review, we will re-review all sources every 5 years to make sure all of our content continues to meet ERIC’s standards. These reviews will happen automatically, and publishers will only be notified if their source is not selected to go forward. Additional re-reviews may happen more frequently for situations such as a change in scope, a source name change, current content is not provided, or if there is a dramatic increase in journal frequency of publication or a dramatic increase in published journal content. If a source has been acquired by a new publisher, a re-review will automatically be conducted, and a new agreement established to continue cataloging in ERIC.

To ensure that review decisions are not arbitrary, all decisions will be reviewed by multiple individuals and publishers will be notified if their source is not selected to continue. We expect that these decisions will be final, but also know that there may be cases where the ERIC team does not have full information. We will consider additional information if an editor or publisher believed our decision was incorrect. If the source is not reinstated, then they are eligible to be re-reviewed after 36 months.

Re-thinking our approach to foreign content

In the almost 60 years of ERIC’s existence, ERIC has become a repository of education research from around the world. Over half of ERIC’s users are international and much of our content is published outside of the United States. While international content is valued by ERIC, there is a tension that ERIC is a digital library funded by the Institute of Education Sciences, an agency within the US Department of Education. IES’s mission is to provide scientific evidence on which to ground education practice and policy and to share this information in formats that are useful and accessible to educators, parents, policymakers, researchers, and the public across the United States. While ERIC has always had a large international audience, the primary mission of ERIC is to share information for a US-based audience.

ERIC has a limited budget to catalog content and there is more high-quality content that meets our selection standards than IES can afford to regularly catalog. As we are looking at how to prioritize our content, we need to focus on our mission. This means that we need to prioritize the needs of educators in the United States.

Going forward, to prioritize the cataloging of content relevant to US educators, ERIC is going to select international sources from Organization for Economic Cooperation and Development (OECD) member countries where English is the primary or most used language both within schools and within society – that is, Australia, Canada, Ireland, New Zealand, and the United Kingdom. Sources from these countries will go through the same ERIC review process. That is, we plan to catalog all content from these countries that meet our selection standards.

Additionally, ERIC will review sources from outside of these countries that are directly relevant to the work of education in the US, its territories and freely associated states, and military bases overseas. For example, if a publisher based in the Netherlands publishes a journal on early childhood education in the US, we review it against the selection standards and catalog it if it meets the standards. Similarly, if a source from Japan publishes a journal on comparative education policy, it will be selected if it meets the standards.

Lastly, ERIC is interested in sources to balance the collection in terms of geographic and topical diversity. This means that there may be international sources that meet ERIC’s standards but are not selected for ERIC because we either have too many sources on that topic or too many sources from that country.

So, what does that mean for our international users? First, ERIC will continue to be available to our international audience. Our website and metadata will continue to be free and open to use. We want to encourage international users to keep using ERIC. But users may see a decrease in the number of sources from countries outside of the US that only publish international content or limited comparative content.

Changing our approach to online submission

Under this new policy, ERIC proposes to no longer accept journal articles through online submission. We found that some journal editors would use the online submission tool to bypass the formal review process. We believe it is more valuable for our users to have journal articles cataloged with the appropriate metadata and source name. If a publisher or editor believes they are a good candidate for ERIC, we want to encourage them to reach out at ERICRequests@ed.gov and go through the formal review process.

ERIC will still accept other types of content through the online submission. Users can submit reports, white papers, conference papers, and other research meeting ERIC’s selection policy. These will continue to be cataloged in PDF format.

The ERIC team is excited about these changes and wants to hear any question, comments, or concerns. Please email any feedback to ericrequests@ed.gov by March 11, 2024.

Data-Driven Decision-Making in Education: How REL Work Makes a Difference

Photo of a workshop REL Appalachia conducted as part of their Strengthening Students’ Preparation for College and Careers in Northeastern Tennessee partnership
Photo of a workshop REL Appalachia conducted as part of their Strengthening Students’ Preparation for College and Careers in Northeastern Tennessee partnership

The Regional Educational Labs (REL) program, operated by the U.S. Department of Education’s Institute of Education Sciences (IES), supports state education agencies, schools and school districts, and institutions of higher education nationwide in using data and evidence-based practice to improve opportunities and outcomes for learners. Operating in all 50 states, the District of Columbia, Puerto Rico, and the U.S. Territories and Freely Associated States of the Pacific region, the REL program brings together the expertise of local communities, top-tier education researchers, and education scientists at IES’s National Center for Education Evaluation and Regional Assistance (NCEE) to address the most vexing problems of education policy and practice in states and regions—on demand and free of charge.

Data-driven decision-making can be a critical tool to address resource disparities, enhance student success, and promote equitable outcomes. Collecting and analyzing relevant data gives insights into student performance, attendance patterns, disciplinary actions, and more. Rather than relying solely on assumptions or "how we've always done things," educators can use these insights to more effectively tailor policy and practice to better meet the unique needs of their students and communities.

In this third installation of our blog series, we'll explore three case studies showcasing the significance of data-driven decision-making and highlighting the REL Program’s pivotal role in shaping the future of education.

REL Mid-Atlantic: Data-Driven Decision-Making to Improving School Accountability Measures in the Wake of COVID-19

The suspension of standardized testing and accountability measures during the pandemic posed challenges for schools and districts. The interruption in assessments meant there was no baseline data from the 2020-21 school year against which to compare future performance. The sudden shifts between in-person and remote learning, disruptions in curricula, and variations in student participation made it difficult to interpret school performance data and introduced additional instability to school performance indicators. This uncertainty made it more important than ever to ensure that accountability measures were as accurate and reliable as possible to avoid mislabeling schools and educators.

To address these challenges, Pennsylvania's Department of Education (PDE) turned to the expertise of REL Mid-Atlantic. The state recognized the need to reduce measurement error and increase the statistical reliability of performance measures, particularly for subgroups of students, who are critical in identifying schools for targeted support and improvement under the Every Student Succeeds Act (ESSA). REL Mid-Atlantic and PDE embarked on a pioneering effort to use Bayesian statistical methods to reduce random error and stabilize performance measures. One focus was the potential instability that small sizes of student subgroups introduced. By minimizing statistical fluctuations, the new approach aims to ensure that schools are not wrongly identified for improvement based on temporary fluctuations in data. This represents a significant advancement in educational accountability and can have far-reaching implications for how states evaluate school performance.

This innovative work is groundbreaking in multiple ways. It addresses an immediate need to ensure that schools are not unfairly labeled as underperforming due to the unpredictability of data during the pandemic. This helps to maintain the credibility of the accountability system. Moreover, this effort extends beyond the immediate crisis. By introducing more accurate and reliable accountability measures, educators can be more confident that performance evaluations are based on solid, consistent data. This, in turn, can lead to greater buy-in and cooperation from educators and stakeholders.

REL Appalachia: Strengthening Preparation for College and Careers in Northeastern Tennessee

Recently, educators from a consortium of districts in northeastern Tennessee report having experienced a wake-up call when they reflected on their own experience preparing for their own college and careers and compared it with feedback from interviews with current students. Educators learned that students are still facing the same challenges, like not receiving enough guidance on navigating the college application process or finding and applying for scholarships.

Since early 2022, district leaders and staff from the Niswonger Foundation have joined with REL Appalachia in the Strengthening Students Preparation for College and Careers partnership. Together, they reflect on districts’ college and career readiness data and identify improvements to programs and services that better prepare students for life after college. Educators have participated in coaching and technical assistance workshops led by REL Appalachia where they look at quantitative data such as their college enrollment and career technical education (CTE) attainment rates. They have supplemented these numbers with student voices through interviews to better understand the whole picture.

One key takeaway from the analysis of outcome data and student interviews was an increased awareness that everyone in their system has the potential to affect postsecondary trajectories. Partners are now considering what changes they can make to help foster social-emotional preparation for college. For example, one partner stated that these changes could be as simple as having counselors support lunch duty as an opportunity for them to build relationships with students.

With a better understanding of their data on college and career preparation, partners are now asking deeper questions about how they can improve their systems to better support students. In the coming years, REL Appalachia will help the partnership address research questions that will allow them to understand better how their programs and services are strengthening preparation for college and careers.

REL Northwest: Accelerating Literacy Outcomes in Montana Through Evidence and Data Use

Despite having a dedicated leadership team that had implemented multi-tiered systems of support to improve literacy, Laurel Public Schools in Montana faced a pressing challenge: only 50 percent of students in grades 3-8 were proficient in reading. For students from economically disadvantaged backgrounds, this dropped to 25 percent. Moreover, achievement gaps persisted between White students and students from diverse racial and ethnic backgrounds, ranging from 10 to 35 percentage points. The district’s own data made it evident that a significant intervention was needed to uplift literacy instruction and outcomes in the district.

Laurel Public Schools and REL Northwest collaborated to address the immediate issue while creating a sustainable solution that would transform literacy instruction and student outcomes over the long term. Laurel wanted to take a close and critical eye to their existing multi-tiered systems of support in reading (MTSS-R) and revise their practices to align better with evidence-based methods to effectively tailor reading instruction and assessment practices. Their goal was to ensure that classroom instruction and interventions were appropriately differentiated for all learners, leading to improved reading achievement and reduced achievement disparities.

REL Northwest was pivotal in guiding the district toward using data effectively. In the first year of the project, REL Northwest worked with literacy leadership teams at Laurel to create a rubric that asked Laurel educators to reflect on evidence-based practices within their MTSS-R and describe practices in their district that are aligned with evidence-based practices and practices in need of improvement.

Using the data they collected, the literacy leadership teams identified previously undetected problems of practice. For example, while evidence-based targeted Tier 2 reading interventions and processes were in place, the data suggested that teachers skipped these targeted interventions and moved directly to Tier 3, referring struggling readers for special education. To support long-term improvement, the literacy leadership teams have begun work on action and monitoring plans as part of a “Plan-Do-Study-Act” continuous improvement cycle. 

The partnership between Laurel Public Schools and REL Northwest showcases how the partnership between RELs and dedicated state and local leaders can lead to educational transformation. It underscores the power of data as a catalyst for change and highlights the importance of evidence-based practices in driving educational excellence.

Looking Ahead

Data-driven decision-making can help states and districts deliver on their commitments to equity, evidence-based classroom practice, enhanced student outcomes, and informed policymaking. As these case studies demonstrate, the REL program can support these states and districts in effectively harnessing data to shape a brighter future for our students and the whole of our educational system.


This blog was written by Nicassia Belton (Nicassia.Belton@ed.gov), contracting officer’s representative with the REL Program at NCEE.

The Regional Educational Lab Program: Making a Difference in Educator Recruitment and Retention

Torrence Williams, Director of Teacher Advancement at the Associated Professional Educators of Louisiana, leads a professional learning module training for Louisiana’s New Teacher Experience program.
Torrence Williams, director of teacher advancement at the Associated Professional Educators of Louisiana, leads a professional learning module training for Louisiana’s New Teacher Experience program.

The Regional Educational Labs (REL) program, operated by the U.S. Department of Education’s Institute of Education Sciences (IES), supports state education agencies, schools and school districts, and institutions of higher education nationwide in using data and evidence-based practice to improve opportunities and outcome for learners. Operating in all 50 states, the District of Columbia, Puerto Rico, and the U.S. Territories and Freely Associated States of the Pacific region, the REL program brings together the expertise of local communities, top-tier education researchers, and education scientists at IES’s National Center for Education Evaluation and Regional Assistance (NCEE) to address the most vexing problems of education policy and practice in states and regions—on demand and free of charge.

It's not exactly breaking news that many schools in our country struggle to fill vacancies in their teacher workforce. This past fall, IES’s National Center for Education Statistics surveyed public school leaders about staffing challenges as they began the 2022-23 school year. The statistics were sobering: 45 percent of schools reported having at least one vacant position more than one month into the school year, and more than 25 percent of schools reported multiple vacancies. Worryingly, our most underserved students were experiencing this crisis most acutely, with roughly 60 percent of schools in high-poverty neighborhoods or with a high-minority student body reporting at least one vacancy. While all of us anxiously await data on the 2023-24 school year—which should be available later this year—RELs and their state and local partners are working to strengthen all aspects of the teacher pipeline.

In the second of a four-part blog series, we highlight four REL research and development projects that address educator recruitment and retention. Each demonstrates how RELs are leveraging their distinct capacity for innovation, rigorous research, and authentic partnership to deliver locally focused and evidence-based supports to the regions, states, and communities they serve.

REL Northwest: Examining Strategies to Improve Teacher Recruitment and Retention in Rural Alaska

Like many rural school districts across the nation, Alaska’s Lower Kuskokwim School District (LKSD) is experiencing a persistent and pressing need to attract and retain educators. Near the start of the 2022-23 school year, 78 positions—nearly one-quarter of all teaching positions in the district—remained unfilled. New teachers were often recruited to the district from other countries such as the Philippines, leading to low retention rates and a constant churn of new educators. Faced with this persistent, high-stakes problem, leadership at LKSD decided to partner with REL Northwest to discover research-based solutions to their teacher recruitment and retention crisis.

REL Northwest is partnering with LKSD to identify evidence-based strategies and tools to continuously monitor and improve working conditions with the goal of increasing teacher retention. To do that, LKSD plans to implement the recommendation of strengthening teacher working conditions from the state's Teacher Retention and Recruitment Plan. As a first step in this partnership, REL Northwest staff reviewed and summarized research on working conditions and teacher retention to identify eight factors that may influence a teacher's decision to stay or leave.  

Lower Kuskokwim leaders decided to focus on activities to identify how changes related to three of those factors—supportive school leadership, available time for teachers, and community engagement—may improve working conditions. The first activity will adapt existing LKSD data sources and develop a research-informed tool to monitor teacher perceptions of school leadership and collect further data to inform the district’s action plan.

Staff also identified a school leadership responsibility unique to their district that may affect teacher trust: managing teacher housing. Housing is a major challenge, not only in remote villages without road access, but in many areas of the country. REL Northwest led partners through an activity to brainstorm how school leaders could improve housing and develop a theory of action for how those strategies would improve working conditions and promote retention. The strategies included establishing realistic housing expectations for new teachers, revising the process for leader evaluation of housing needs, and changing the budgeting process to make maintenance needs and upgrades easier. As a result of the work, district leaders are designing a program that allows teachers to apply for district funding to make simple housing upgrades, such as changes in lighting or painting.

REL Central: Strengthening the Teacher Pipeline in South Dakota to alleviate Teacher Shortages

Like many states, South Dakota is experiencing a teacher shortage that has worsened in recent years. Late last year, SDDOE partnered with REL Central to support one component of their response to this challenge: developing new pathways into teaching for candidates such as paraprofessionals and other South Dakota residents who have interest in entering the teacher workforce. Initially, the work focused on the design and implementation of a teacher apprenticeship program designed to support paraprofessionals as they acquire their teaching degrees and as they are mentored to become certified teachers.

In March, REL Central began work with SDDOE on a fast-turnaround project to support the development of a survey for paraprofessionals about their interest in the pilot apprenticeship program and the types of supports they seek from mentor teachers. Within a matter of weeks, the survey was developed and administered to paraprofessionals statewide. With survey data in hand indicating that hundreds of South Dakota paraprofessionals were interested in such a program, the pilot was expanded by SDDOE to support additional slots starting in fall 2023. In the coming months, REL Central will work with SDDOE to further refine this program by incorporating research evidence from other “Grow Your Own” teacher workforce programs on the components of effective mentoring and by helping the state to generate, collect, analyze, and use data from participants to inform further improvements to the pilot apprenticeship program.

REL Pacific: Strengthening the teacher workforce in Palau

The Republic of Palau, like many school systems, has experienced challenges in recruiting and retaining enough teachers to provide every student with a high-quality education. The geographic isolation of Palau compounds these challenges. Many local Palauan teachers do not have a four-year college degree or are teaching outside of their area of certification. As was the case in Alaska’s Lower Kuskokwim School District, one solution has been to bring in teachers from out of the country to fill vacancies; however, the turnover rate of these teachers is very high. The Palau Partnership to Support Teacher Effectiveness–– a collaboration with the Palau Ministry of Education (MOE), Palau Community College (PCC), six private schools, and REL Pacific ––is focused on building more sustainable solutions. 

The long-term goals of the partnership include Palau private schools adopting a teacher effectiveness measurement system to support, develop, and retain effective teachers; Palau private schools adopting a systemwide instructional coaching process for improving teacher effectiveness; and Palau MOE and PCC reviewing data on the effects of teacher education programs and making implementation adjustments so that their available resources may be used more effectively and efficiently. 

REL Pacific is supporting partner schools to realize their goals by drawing from resources on indicators of successful teacher recruitment and retention as well as best practices of effective teaching. REL Pacific is providing schools training and coaching on data-driven decision-making and plan-do-study-act (PDSA) cycles in the context of the schools’ goals to improve literacy instruction through developing systemic supports for teachers. By incorporating ways of measuring best practices of effective teaching into the schools’ instructional coaching processes, each school will be better able to enact systemic change to address its specific teacher development and retention needs. Additionally, an applied research study is underway that will describe teacher pathways and certification patterns. The findings from this descriptive study will inform future efforts of Palau’s education community to address the new teacher certification requirements and overall educator shortage crisis.

REL Southwest: Partnering to support early career and aspiring teachers in Louisiana

Louisiana’s educator shortage is compounded with low retention rates for early-career teachers. Teachers with 2–5 years of experience left public schools in 2020 at a rate of 30 percent, compared with 17 percent of teachers with 6–10 years of experience. REL Southwest and the Louisiana Department of Education (LDOE) have formed the Supporting Early Career and Aspiring Teachers (SECAT) partnership to improve supports for novice and aspiring teachers. Through the SECAT partnership, REL Southwest aims to strengthen LDOE’s capacity to use evidence to refine new initiatives that support early career and aspiring teachers, such as the New Teacher Experience program and the Louisiana Pre-Educator Pathway, the state’s largest “Grow Your Own” program. This work builds from a previous partnership between REL Southwest and LDOE focused on exploring and evaluating the early impacts of Louisiana’s Believe and Prepare teacher residency program.  

The work of the SECAT partnership kicked off this past spring. Over the next five years, REL Southwest will work with LDOE and school systems in Louisiana to strengthen their capacity to generate and use evidence to refine existing programs for early career and aspiring teachers. In the first year of the partnership, REL Southwest and LDOE partners plan to focus on technical assistance that builds LDOE’s capacity for evaluating the New Teacher Experience. In future years of the partnership, REL Southwest will study LDOE’s efforts to support new and aspiring teachers. Along the way, REL Southwest will share important takeaways, resources, and policy implications related to teacher recruitment and retention learned through the partnership with Louisiana.

Stay tuned for part three of our “Making a Difference” series, focused on school accountability systems. As always, my (virtual) door is open if you have questions about the work highlighted in this blog, or anything else on REL Program. Just email me at chris.boccanfuso@ed.gov.

Getting to Know ED: My Journey as a STEM Next Fellow at IES

This guest blog was contributed by Dr. Holly Miller, who currently serves as a STEM Next Opportunity Fund Fellow at the Institute of Education Science’s National Center for Education Evaluation.

Since August 2022, I’ve been serving as the STEM Next Opportunity Afterschool and Summer Learning Fellow at the U.S. Department of Education (ED). More specifically, I work within the Department’s Institute of Education Sciences (IES).

Upon arriving at IES, I was charged with a specific challenge: amplify how evidence-based practice in out-of-school time (OST) can support student learning and development. This mission was made all the more relevant by the need for states and districts to respond to the consequences of the COVID pandemic which, at the time, remained an official national emergency.

Perhaps naively, I hoped to walk in on Day One and find “The Official Compendium of Evidence-based Practices in Global Pandemics and Related Crises” that I could pull off the shelf and hand to educators. Unfortunately, I quickly discovered no such tome existed. And I began to realize that one of the biggest challenges I’d face in my new role was getting to know ED itself! To an outsider, the Department can seem like a huge machine. Getting to know it, though, can pay incredible dividends. As I came to learn, there are tons of great resources—if only you know where to look.

One of OST educators’ first stops in getting to know ED should be IES. For the uninitiated, IES is the Department’s statistics, research, and evaluation arm. The mission of IES is to provide scientific evidence on which to ground education practice and policy and to share this information in formats that are useful and accessible to educators, parents, policymakers, researchers, and the public. It is independent and non-partisan.

Across its four centers—the National Centers for Education Statistics, Education Evaluation, Education Research, and Special Education Research—IES conducts six broad types of work (http://ies.ed.gov):

1. Providing data to describe the “condition of education,” including students’ academic proficiency.

2. Conducting surveys and sponsoring research projects to understand where education needs improvement and how these improvements might be made.

3. Funding development and rigorous testing of new approaches for improving education outcomes for all students.

4. Conducting large-scale evaluations of federal education programs and policies.

5. Providing resources to increase the use of data and research in education decision-making, including independent reviews of research on “what works” in education through the What Works Clearinghouse.

6. Supporting the advancement of statistics and research through specialized training and development of methods and measures.

I could see that this work had the potential to benefit a variety of stakeholders—teachers, administrators, students, researchers, and policymakers. Still, I had so many unanswered questions. As a middle school teacher, I frequently told students, “The only dumb question is the one you don’t ask.” Therefore, as I surveyed the education research landscape at IES, I asked lots and lots of questions. My presence at IES was akin to a toddler at the zoo for the first time: “What are those? Why is that so big? Why don’t we have more of these? When do we eat?” Months of asking and I find my queries have been distilled into two essential questions:

  1. What has been the impact of the COVID pandemic on students and educators; and

 

  1. How can education research, like that conducted or sponsored by IES, help us understand—and address—those impacts?

What has been the impact of the COVID-19 pandemic?

The pandemic disrupted nearly every aspect of daily life in the United States, including the education system. One of the most alarming impacts of the pandemic on education has been the widening of pre-existing gaps in student achievement and the resources that students need to be successful.

We all know the statistics … students have lost tons of learning. The "Report on the Condition of Education" is a congressionally mandated annual report from the National Center for Education Statistics (NCES). Using the most recent data available from NCES and other sources, the report contains key indicators on the condition of education in the United States at all levels, from prekindergarten through postsecondary, as well as labor force outcomes and international comparisons. For example, the report on the condition of education 2023 recently released shares that on both the 4th- and 8th-grade NAEP mathematics assessments, higher percentages of students performed below NAEP Basic in 2022 than in 2019 (Irwin et al., 2023).  This has been particularly bad among students who have historically been underserved. The average NAEP mathematics scores in 2022 were generally lower for English Learners (EL) students than for non-EL students; lower for those identified as students with disabilities than for their peers without disabilities; and higher for students in low-poverty schools than for students in high-poverty schools. These patterns were similar to those observed for reading (Irwin et al., 2023).

This is surely due, at least in part, to differences in the resources students have access to. Even before the pandemic, huge gaps in resources existed. The pandemic only made matters worse. According to a report by the U.S. Department of Education’s Office for Civil Rights (OCR) (2021), low-income students and students of color have been disproportionately negatively impacted by school shutdowns and remote learning practices. These students often lack access to reliable technology and internet resources, making it difficult for them to participate fully in online classes and complete assignments. Additionally, many students rely on meals provided by schools, so the closure of physical school buildings has led to food insecurity for some.

Also of note: the dramatic effect on student wellbeing. During the pandemic, mental health concerns such as fear, anxiety, and depression were common among the general public, especially children and older adults (Brooks et al., 2020; Pfefferbaum & North, 2020).  Research on the pandemic’s impact on mental health among students finds that “they showed increased fear, stress, and decreased happiness, and these were associated with their learning quality change.” (Hu et al., 2022).

Furthermore, the impact of COVID on educators is increasingly well-known. Educators had to make changes in short order, often with limited resources. This had consequences. Educators faced increased stress levels due to the shift to remote instruction, and many reported struggling to maintain a work-life balance while working from home. Findings indicate teachers reported greater mental health concerns than those in many other professions, and that remote teachers reported significantly higher levels of distress than those teaching in person (Kush et al., 2021). For some, it was too much, and they made the decision to leave the profession. Forty percent of public schools hiring for open teaching positions in special education in 2020–21 reported having difficulties filling the opening, compared with 17 percent in 2011–12 (Irwin et al., 2023) Not only were teachers leaving the workforce, but potential teachers were second-guessing their career choice. The number of persons enrolled in traditional teacher preparation programs decreased by 30 percent between 2012–13 and 2019–20, and the number of persons completing such programs decreased by 28 percent between 2012–13 and 2019–20 (Irwin et al., 2023).  

All of us are looking for solutions to all these problems. Given that I entered IES during the pandemic, I wanted to know how I could leverage its resources to help.

How can education research help?

First, I had to understand how IES, as a science agency, was structured to do the work of education research. My college textbook on education research (Newby, 2010) asserted that it should have three objectives: to explore issues and find answers to questions, to collect and disseminate information that shapes policy and decision-making, and to improve practice for practitioners.

It’s easy to see how the six broad areas of work at IES I listed above fit within those three objectives. For example, in normal (that is, pre-COVID) times, it’s the job of the National Center for Education Statistics (NCES) to collect and disseminate education-related statistics and information about student achievement to inform the work of researchers, policymakers, and other education decision-makers. IES’ two research Centers, the National Centers for Education Research (NCER) and Special Education Research (NCSER) support researchers’ exploration of a wide range of education topics and their use of high-quality methods to answer important questions of policy and practice. Finally, the National Center for Education Evaluation and Regional Assistance (NCEE) conducts its own rigorous evaluations of federal policies and programs; supports states and districts in the use of data, evidence, and applied research to improve local practice; and disseminates information about “what works” through its What Works Clearinghouse (WWC). In the wake of the pandemic, IES had to quickly focus its activities and resources to meet new demands across the education system. Here are just a few of the new questions that IES had to address amid the pandemic.

  • What’s happening in schools, and who is learning in-person versus virtually or in hybrid settings? In late 2021, NCES leveraged work being done as part of the National Assessment of Educational Progress (NAEP) to meet an immediate need to better understand schools’ policies about learning mode, masking, and social distancing. In the weeks that followed, the School Pulse Panel was created (https://ies.ed.gov/schoolsurvey/spp/). Initially, the School Pulse focused on collecting monthly information on the impact of the COVID-19 pandemic from a national sample of elementary, middle, high, and combined-grade public schools. Over time, its focus has broadened. While some survey questions are asked repeatedly to observe trends over time, others are unique each month. IES is now able to provide regular and near-real-time snapshots into “what’s happening” in the nation’s schools on a wide range of topics that matter to educators, policymakers, and families.

 

 

  • How can educators and caregivers support student learning in online, hybrid, and at-home settings? With schools closed and remote learning becoming the norm, educators and caregivers had to adapt their teaching methods and find new ways to engage students. As part of a mandate to provide assistance about “what works” in education, NCEE supported a series of efforts to bring together information for teachers navigating online and hybrid teaching environments and for caregivers who were providing instruction at home. NCEE commissioned work leading to the development of the “Best Practice in K-12 Online Teaching” minicourse (here), freely available from North Carolina State University, to support teachers new to online education in their transition to the medium. (The literature review on which the mini-course is based can be found here). NCEE’s Regional Educational Laboratories developed nearly 200 pandemic-related resources. Notable examples include “Supporting Your Child’s Reading at Home” (https://ies.ed.gov/ncee/rel/Products/Region/southeast/Resource/100679), which focuses on the development of early literacy skills, and “Teaching Math to Young Children for Families and Caregivers” (https://ies.ed.gov/ncee/rel/Products/Region/central/Resource/100652).

   

Since its inception in 2002, IES and its Centers have supported decision-makers—be they federal, state, or local—and educators in making use of high-quality evidence in their practice. The pandemic showed just important IES, its resources, and its infrastructure, can be.

In the pandemic’s wake, though, it seems to me that building even more evidence about “what works” is vital. The American Rescue Plan (ARP) provided historic levels of resources to expand educational opportunities and to ensure that education is better able to address the wide-ranging needs of students and their families – especially those who were disproportionately impacted by the pandemic. Many ARP investments, including those related to OST, have the requirement that programs be rooted in evidence-based practices. Because there are still things to learn about what makes strong programs, we can strengthen the field by building evidence that can address key problems of practice.

Conclusion

When I came to ED and IES, searching for information on how to use evidence-based practices to support COVID recovery within the context of OST, I was lost. As I’ve come to better understand the organization, I’ve learned that vast resources are available. Half of the battle was just figuring out “what lives where” within the Department! I hope this blog has given OST practitioners a bit of a roadmap to make their own process of discovery easier.

In Part Two of this series, I will explore how OST learning fits into ED, education research, and the post-pandemic education system. The latter has been profoundly affected, creating an opportunity for innovation and transformation in the delivery of education. The value of research cannot be underestimated in this context. As a result, my next blog will pose two questions. First, I’ll ask what the role of OST in learning recovery can be in the years ahead.  Then I’ll consider what evidence needs to be built to make the most of what OST can offer. I hope you’ll read it!

I’d love to hear your thoughts on this blog. Send them my way at holly.miller@ed.gov.

 

Citations

Brooks S.K., Webster R.K., Smith L.E., Woodland L., Wessely S., Greenberg N., Rubin G.J. (2020). The psychological impact of quarantine and how to reduce it: rapid review of the evidence. Lancet North Am. Ed. 395(10227):912–920.

Education in a Pandemic: The Disparate Impacts of COVID-19 on America's Students 2021 U.S. Department of Education’s Office for Civil Rights (OCR).

Hu K, Godfrey K, Ren Q, Wang S, Yang X, Li Q. (2022). The impact of the COVID-19 pandemic on college students in USA: Two years later. Psychiatry Res. Sep; 315:114685.

Huck, C., & Zhang, J. (2021). Effects of the COVID-19 Pandemic on K-12 Education: A Systematic Literature Review. New Waves-Educational Research and Development Journal, 24(1), 53-84.

Irwin, V., Wang, K., Tezil, T., Zhang, J., Filbey, A., Jung, J., ... & Parker, S. (2023). Report on the Condition of Education 2023. NCES 2023-144. National Center for Education Statistics.

Kush, J. M., Badillo-Goicoechea, E., Musci, R. J., & Stuart, E. A. (2021). Teacher mental health during the COVID-19 pandemic: informing policies to support teacher well-being and effective teaching practices.

Newby, P. (2010). Research Methods for Education. Pearson Education.

Pfefferbaum B., North C.S. (2020). Mental health and the Covid-19 pandemic. N. Engl. J. Med.;383(6):510–512.

The Regional Educational Lab Program: Making a Difference in Literacy and Math Outcomes

by Chris Boccanfuso, REL Program Branch Chief

REL Midwest ENACT Coach, Katie Rich, works with Milwaukee Public Schools’ grade 6 teachers and math coaches during the ENACT Summer Institute.
REL Midwest ENACT Coach, Katie Rich, works with Milwaukee Public Schools’ grade 6 teachers and math coaches during the ENACT Summer Institute.

When educators at Harts PreK-8 and Omar Elementary School in southwestern West Virginia wanted to pioneer a new approach that supported families to engage in their child's math learning, they turned to a partner who had supported their State for more than 50 years: Regional Educational Laboratory (REL) Appalachia. The resulting program, known in the Appalachian region and elsewhere as community math nights, has now served hundreds of families in West Virginia and Kentucky. And thanks in part to the attention of education writers and the national media, it is poised to help even more.

The Regional Educational Labs program, operated by the U.S. Department of Education’s Institute of Education Sciences (IES), supports state education agencies, schools and school districts, and institutions of higher education nationwide in using data and evidence-based practice to improve opportunities and outcome for learners. Operating in all 50 states, the District of Columbia, Puerto Rico, and the U.S. Territories and Freely Associated States of the Pacific region, the REL program brings together the expertise of local communities, top-tier education researchers, and education scientists at the IES National Center for Education Evaluation and Regional Assistance (NCEE) to address the most vexing problems of education policy and practice in states and regions—on demand and free of charge.

In this first of a four-part blog series, we highlight four REL research and development projects focused on strengthening math and literacy outcomes. Each demonstrates how RELs have leveraged their distinct capacity for innovation, rigorous research, and authentic partnership to deliver locally-focused and evidence-based supports to the regions, states, and communities they serve.

REL Appalachia: Engaging families for math success in Kentucky and West Virginia

A community math night (CMN) brings together educators, students, family members, and other caring adults to learn about, talk about, and have fun with math. CMNs are designed to reinforce positive math mindsets, help family members participate in their child's learning, and build a sense of community and partnership around a subject that many caregivers find daunting. As part of its support for a series of CMNs across Kentucky and West Virginia, REL Appalachia published materials and a facilitators guide so that any school can host their own event.

True to the program’s emphasis on continuous improvement and innovation, REL Appalachia has begun a new partnership with Logan County (WV) schools that builds from the foundations of the existing CMN approach as a jumping-off point to further accelerate improvement in middle school math achievement using a more comprehensive, year-round set of supports. This new effort involves coaching school-based teams to build their capacity to promote positive math attitudes with students and families, implement research-based instructional practices in the classroom, and employ inclusive family engagement strategies. Materials used to coach school teams will be pulled together into a single resource when the project is complete, but you can begin to use early content now! (See, for example, resources for “Promoting Positive Mathematics Attitudes.”)  

REL Midwest: Inspiring Milwaukee students to be lifelong STEM learners

What if some of the principles powering today’s most innovative technologies—such as artificial intelligence—could be taught to elementary students to encourage lifelong learning and success in science, technology, engineering, and mathematics (STEM)? The ENgagement and Achievement through Computational Thinking (ENACT) partnership, which includes Milwaukee Public Schools (MPS) and REL Midwest, is working with teachers to do just that.

Together, MPS and REL Midwest are developing, evaluating, and refining an approach for integrating computational thinking—a set of skills related to problem solving and explaining one’s reasoning—into MPS’ 6th grade math curriculum.

Laura Maly, an MPS mathematics teacher-leader, shared how integrating computational thinking strategies has influenced her work this year. "The [ENACT] computational thinking strategies are very prevalent throughout all strands of mathematics and across all of the grade levels that I work with," she said. "It is easy to bring the CT [computational thinking] strategies to life when working in mathematics classrooms."

REL Southeast: Improving Literacy in Mississippi –The Journey Continues

After many years at the bottom in student performance when compared with other states, the Mississippi Department of Education (MDE) made a concerted effort to improve the foundational literacy skills of students in kindergarten through grade 3. The results of that effort speak for themselves, with many referring to the state’s transformation as the “Mississippi Miracle.” Now the state is turning its attention to the next leg of its journey: improving the literacy skills of all students, focusing on grade 4 and beyond.

MDE and four school districts in the state–Canton, Columbus, Laurel, and George counties–formed the Mississippi Improving Adolescent Literacy Partnership with REL Southeast to ensure educators can integrate literacy strategies into a wide range of academic courses, from social studies to the sciences. To do so, partners are relying on research conducted elsewhere within NCEE: its What Works Clearinghouse. (Check out the WWC’s Improving Adolescent Literacy: Effective Classroom and Intervention Practices and Teaching Secondary Students to Write Effectively practice guides to learn more!)

After completing a REL Southeast training session, one teacher wrote a message to her principal saying the training gave her tips on how to improve student literacy in her social studies class.  “The training helped prepare me to serve my students in the classroom better and has opened my eyes to a broader range of techniques and skills. The training was so informational and engaging that it gave me the confidence level needed to help my students." Two months after the last session, a follow-up survey of participants showed that over half of the teachers reported changing (or being in the process of changing) their approach to supporting based on the training.

Now, REL Southeast is developing tools that will allow districts across Mississippi—and across the country—to replicate these early successes.

REL Southwest: Studying an enhanced approach to literacy instruction for English learner students in New Mexico

Supporting English learner students is a priority in New Mexico and for the state’s Public Education Department (NMPED). Two recent studies from REL Southwest found that significant numbers of both American Indian and Spanish-speaking English learners in New Mexico are struggling to meet grade-level standards and be reclassified as “Fluent English Proficient.” NMPED has developed a strategic plan designed to strengthen equitable educational opportunities and achievement for English learner students. To do so, the plan emphasizes supporting the whole child through evidence-based literacy instruction that is culturally and linguistically responsive (CLRI).

REL Southwest, NMPED, and several regional education cooperatives and school districts in New Mexico formed the Southwest English Learner Literacy (SWELL) partnership to help turn the State’s new plan into a reality. Over the next two school years, partners will enhance, implement, refine, and test Write to Succeed, a research-based professional learning program to help grades 4–8 teachers implement high-quality literacy instruction that includes appropriate supports for English learner students. After working with school and district coaches and NMPED staff to refine and deliver the Write to Succeed program to teachers statewide, REL Southwest will rigorously evaluate the impact of the program on teacher practice and student outcomes. What is learned will help to inform NMPED’s further adoption—or modification—of this program.

Looking Ahead

These four projects demonstrate RELs working in partnership with educators and policymakers to improve opportunities and outcomes for students in mathematics and literacy. Stay tuned for upcoming blogs about REL work on other important topics such as teacher mentorship and induction, accountability, and trauma-informed supports!

Have questions about anything you read here, or other work within the REL Program? Just email me at chris.boccanfuso@ed.gov.

Regional Educational Laboratories Develop New Tools for Educators Based on WWC Practice Guides

Whenever we get the chance to share information about the Regional Educational Laboratory (REL) program with the public, we’re often asked, “How are the 2022-2027 RELs different from past REL cycles?” In this blog, we focus on one major new effort that each REL is undertaking: the creation of a toolkit for educators based on one of the What Works Clearinghouse (WWC) Practice Guides. Each REL toolkit will include a set of resources for educators to implement and institutionalize evidence-based recommendations from a WWC Practice Guide. Importantly, each REL is co-developing their resources with educators, school, and district leaders or with postsecondary faculty and staff to ensure the toolkits’ relevance and actionability. Following the toolkit development phase, RELs will partner with educators not involved in developing the toolkits to test the usability of each toolkit and its efficacy in improving student and teacher outcomes. The RELs have current partners for toolkit development and usability testing but are looking for partner schools, districts, and postsecondary institutions in which to test the efficacy of the toolkits. These efficacy-testing partners will be among the first to benefit from the evidence-based toolkits.

Why this investment of REL and partner time and resources? WWC Practice Guides are among IES’ premier resources for translating evidence on effective practice into accessible and usable strategies for educators. Each Guide is based on a synthesis of the most rigorous research on teaching a particular subject or achieving a particular education goal. Each Guide is also based on the input of a panel of expert practitioners and researchers and includes—

  • Key recommendations for educational practice based on a synthesis of rigorous research
  • Supporting evidence for each recommendation
  • Steps to carry out each recommendation
  • Examples of the practices
  • Discussions of common implementation challenges and strategies for overcoming those challenges

WWC Practice Guide Associated with Each REL Toolkit:

REL

Practice Guide

Appalachia

Teaching Math to Young Children

Central

Teaching Strategies for Improving Algebra Knowledge in Middle and School Students

Mid-Atlantic

Teaching Elementary School Students to Be Effective Writers

Midwest

Developing Effective Fractions Instruction for Kindergarten through 8th Grade

Northeast & Islands

Assisting Students Struggling with Math: Intervention in the Elementary Grades

Northwest

Using Technology to Support Postsecondary Student Learning 

Pacific

Teaching Secondary Students to Write Effectively

Southwest

Providing Reading Interventions for Students in Grades 4 – 9

Southeast

Assisting Students Struggling with Reading: Response to Intervention (RtI) and Multi-Tier Intervention in the Primary Grades

West

Improving Reading Comprehension in Kindergarten Through 3rd Grade

RELs Emphasize Active Learning to Support Implementation of Evidence-Based Practices

Although WWC Practice Guides are some of IES’ most popular products, we also know that teachers and leaders cannot simply read about a new practice to master it. Instead, they need to engage in active learning by observing the new practice, discussing it, implementing it, receiving feedback on the practice, and continuing to improve. The REL toolkits are designed to support educators in the creation and implementation of a professional learning community (PLC) focused on the evidence-based practices outlined in a WWC Practice Guide. In these PLCs, educators will learn about the Practice Guide recommendations by reading about the practices, discussing them with colleagues, and by developing plans for implementing the practices in their classrooms. Educators will also put those plans into action and then debrief on those implementation experiences. To support this work, the toolkits will include PLC guides, workbooks, self-study guides, and rubrics. Some toolkits will also include videos of teachers effectively implementing the practices.

Each toolkit will also include the following:

  • An initial diagnostic and ongoing monitoring instrument for assessing instructional practices against the practices recommended in the WWC Practice Guide
  • A tool that enables teachers, teacher leaders, and administrators to assess the extent to which their school, district or postsecondary institution supports the implementation and ongoing monitoring of the evidence-based practice recommendations
  • A discussion of implementation steps for institutionalizing supports that help educators, building leaders, and other administrators adopt the evidence-based practices and sustain them over time

Some RELs have already started usability testing of their toolkits. Across 2025 and 2026, nine of our 10 RELs will publish final versions of their toolkits and efficacy studies on their toolkit. Both will be freely available on the REL website.[1] Visit our Newsflash page and sign up to receive newsflashes from the RELs and the IES center that houses the program—the National Center for Education Evaluation and Regional Assistance (NCEE).[2]

Partner with RELs: Help IES Study REL Toolkits

RELs will soon recruit partner schools, districts, and postsecondary institutions in their regions to conduct the toolkit efficacy studies. If you are interested in having your school, district, or institution participate in an efficacy study and benefit from being one of the first users of these toolkits, please email us at Elizabeth.Eisner@ed.gov or Chris.Boccanfuso@ed.gov. The efficacy study for each REL’s toolkit must take place within each REL’s region. Not sure which REL region is yours? Check out the “About the RELs page” on the IES website or the map visualization on our program homepage.

If you have other questions, concerns, or ideas about this work, please reach out to us. We welcome your input so that you can help IES and the RELs make the toolkits as useful and effective as possible.

Past REL Professional Development Resources based on WWC Practice Guides:

The RELs have a successful track record of creating professional development resources that complement WWC Practice Guides. For example, see:

Professional Learning Community: Improving Mathematical Problem Solving for Students in Grades 4 Through 8 Facilitator’s Guide (REL Southeast).

Professional learning communities facilitator’s guide for the What Works Clearinghouse practice guide: Foundational skills to support reading for understanding in kindergarten through 3rd grade (REL Southeast).

Professional Learning Communities Facilitator's Guide for the What Works Clearinghouse Practice Guide Teaching Academic Content and Literacy to English Learners in Elementary and Middle School (REL Southwest).

The new toolkits will expand the number of WWC Practice Guides for which the RELs develop professional development resources and will also provide instruments for assessing instructional practice and implementing institutional supports. 

Liz Eisner, Associate Commissioner for Knowledge Use

Chris Boccanfuso, REL Branch Chief


[1] REL Southwest’s contract started 11 months after the contracts of the other 9 RELs, so the REL Southwest toolkit will be released in 2027.

[2] You can also sign up for Newsflashes from IES and its other three centers—NCES, NCER, & NCSER.

Taking a pause…

In a first for ERIC, the supply of education research content the program aspires to index is regularly exceeding its capacity to do so. For the past 15 years, ERIC has consistently indexed 4,000 records a month. This pace has allowed us to index all approved sources without significant delays. However, over the past two years, the volume of content published in our approved sources has doubled. This has resulted in a backlog of publishable content and, as a result, ERIC cannot index new work in a timely fashion.

As part of our periodic collection analyses, we have been investigating potential causes of this backlog. We have found that several journals are publishing far more content than when they were originally selected to be included in ERIC. For example, one journal was publishing fewer than 20 articles per year when approved, but now is publishing over 850 per year. This is close to a 5000% increase in production.

An increased volume of published work in the education sciences—as a whole—is a good thing. However, when an individual journal dramatically increases the number of articles it publishes, it is noteworthy. In those instances, ERIC wants to ensure that the journal is still adhering to the standards and criteria it met when originally included in the index, including rigorously applying the peer-review process, if applicable, and maintaining its original aim and scope. Both are important to ensuring that work contained in ERIC is of high quality and that a wide range of key topics in education can be indexed.

ERIC’s Selection Policy already requires an ongoing review of currently indexed sources, including identifying sources that may no longer meet the Policy’s standard and criteria. As part of that review, ERIC will now identify journals that have published over twice as many articles from the year it was accepted and flag them for further review. As part of that review, ERIC will assess whether the increase is temporary and associated with a unique event, like a special issue, or if the increase reflects an ongoing trend. If the review indicates the increase is persistent, the ERIC team will recommend that indexing of that journal be paused. If a pause is approved, ERIC will stop indexing subsequent issues for a two-year period. The journal will also be removed from ERIC’s Journals List, because this list only contains actively indexed journals.

After the two-year pause, ERIC will re-review the journal. If ERIC reinstates the journal, ERIC will notify the journal and will index any content published during the two-year pause. If the journal is not reinstated, ERIC will not index any issues published during the pause.

This decision has led to a few questions:

  • Why can’t ERIC index everything? ERIC is funded by the U.S. Department of Education’s Institute of Education Sciences and has limited resources. It must prioritize indexing the highest quality education research, include content for all topic areas, and can only index a set number of records per month.
  • What is the concern with the increased volume? Increased volume may signal that the journal has changed scope to accommodate a broader set of articles or that quality assurance processes have been affected in a rush-to-publish environment. Particularly among journals that were considered to be peer reviewed when originally accepted into ERIC, the publication of articles shortly after submission may signal a substantive change in a journal’s quality assurance process such that they no longer meet the criteria needed to receive that designation. There is also a concern that if a journal greatly produces more records than estimated, the collection will get skewed in a way which would favor one topic area over another.
  • How will ERIC identify the journals to pause? As part of the source selection process, ERIC will monitor two years of current publishing and compare the number of articles published to the number published during the year the journal was selected for ERIC.
  • Why is the pause for two years? The ERIC Selection Policy says that sources may be reviewed after 2 years (24 months). To be consistent with this policy, we will automatically review paused sources after this same time frame.
  • How will I know if my journal is paused? ERIC will email the journal representatives to inform them of this decision in the coming weeks.
  • Can my journal appeal the decision to be paused? Journals may not appeal the decision to pause indexing. However, if at the end of the pause period the journal is not automatically reinstated, journals may apply for re-review 24 months later.
  • What is ERIC looking for in the automatic review at the end of the pause period? ERIC will conduct a full review of the journal and consider the two years of published content against the criteria set forth in the ERIC Selection Policy.
  • Can authors submit their article published in a “paused” journal via online submission during this period? No, authors must wait for a re-review of the journal to be conducted after the two-year pause. If the journal is selected again, the articles from the paused issues will be indexed in ERIC.

How State Education Agencies Can Leverage Their Regional Educational Laboratory to Support Students’ Academic, Social, and Mental Health Needs

(A Dear Colleague Letter sent to Chief State School Officers on February 23, 2023.)

Dear Colleague:

As state and local education agencies leaders reflect upon the successes and challenges of the 2022-2023 school year—and the opportunity that summer 2023 presents to further support students’ academic, social, and mental health needs—I am writing today to encourage you to take full advantage of the services offered by your Regional Educational Laboratory (REL).

The REL Program, sponsored by the U.S. Department of Education’s Institute of Education Sciences, supports educators and policymakers at the state and local levels in the use of data and evidence-based practices to improve student outcomes. All REL services are provided free of charge and are designed in partnership with state and local partners to meet their specific needs. Each REL is led by a Director with deep expertise in education policy, practice, and research who can help you navigate how best to leverage REL supports to address your state’s most pressing needs. A list of REL Directors, including their contact information, is attached.

Your REL can support a wide range of state and local initiatives. They include:

  • Analyzing student progress and outcome data (e.g., achievement, chronic absenteeism, graduation rate, English language proficiency) to understand the ongoing effects of the COVID-19 pandemic. Your REL can analyze longitudinal student data provided by the state or district partners to better understand the trajectory of student performance prior to the pandemic, during the pandemic, and today. When disaggregated by student group, school characteristics, or other relevant features, these analyses can support decision-makers in focusing resources, monitoring improvement, and adjusting implementation efforts. RELs Midwest and Mid-Atlantic recently provided similar services for their state and district-level partners.
  • Supporting the identification of existing, or the design of new, evidence-based practices to meet students’ academic, social, and mental health needs. Your REL can support state and local efforts to identify practices that prior evidence suggests can promote learning and development. REL Southeast recently published a review on the effectiveness of early literacy interventions across several domains in response to a request from partners regionwide. When high-quality evidence does not exist, or existing practices are not well-aligned to state or local needs, RELs can support efforts to design and pilot research-based innovations.
  • Coaching state and local education agency staff on the use of data to improve the ongoing implementation of education policies, programs, and practices. Your REL offers coaching and training services for state and local leaders on data-driven approaches to continuous quality improvement. These services are particularly beneficial when a program is relatively new to a state or district and leaders are focused on timely feedback to ensure an evidence-based practice is well-implemented at scale. REL Southwest recently supported the Oklahoma State Department of Education’s (OSDE) rollout of Oklahoma Excel, a data-driven and job-embedded professional development program for educators in participating districts. A 2-part video series provides background on the program, and the supports REL Southwest provided to OSDE staff who administer the program.
  • Evaluating the impact of state or local interventions on important student outcomes. Your REL can support the rigorous evaluation of well-implemented policies, programs, or practices to document those efforts’ impacts on important student outcomes. For example, a 2021 REL Northwest study examined the implementation and impact of full-day kindergarten in Oregon in light of a funding structure shift that incentivized districts to offer the programming. When a rigorous evaluation is not feasible, your REL can advise you on credible, alternative approaches to understanding the outcomes associated with a policy or program.
  • Coaching state or local education agency staff on the use of existing REL tools and resources. Through their work with state and local partners, RELs have developed a wide range of actionable resources designed to support the implementation of evidence-based practices. Your REL can coach state and local education agency staff on how to  customize and use tools developed elsewhere to meet your needs. Examples include REL Appalachia’s Community Math Night Facilitators’ Toolkit and REL Southeast’s Professional Learning Community on Emergent Literacy.
  • Providing expert guidance to senior state or local education agency leaders. Finally, your REL can leverage its network of internal and external experts to offer guidance on data- and evidence-driven approaches to addressing problems of policy and practice. This “Ask-an-Expert” service is available to senior leaders of both state and local education agencies. A recent REL Appalachia “Ask an Expert” response to a Tennessee-based partner shared best practices for administering and using data from Kindergarten readiness screeners.

REL Directors are routinely in contact with senior education agency leadership as part of their on-going work to better understand the kinds of supports that might benefit states in their region. However, if you or senior members of your leadership team have not yet had the opportunity to meet with your REL Director (or have not done so recently), please consider contacting them at your convenience. I am also glad to facilitate that connection at your request.

Finally, I would be remiss if I did not mention the critical relationship between your REL and the Regional Comprehensive Center (RCC) that serves your state. Sponsored by the Department’s Office of Elementary and Secondary Education, RCCs support state education agencies in their efforts to implement evidence-based policy and practice and realize the goals set in their Consolidated State Plans.

If you have any questions about the REL Program, please do not hesitate to contact me or a senior member of my team.

Sincerely,

Matthew Soldner
Commissioner, National Center for Education Evaluation and Regional Assistance
Matthew.Soldner@ed.gov 

Note: This blog reflects slight edits to the letter sent to Chief State School Officers. References to an attached brochure and a contact list for REL Program staff have been removed.

Leading with Evidence: Celebrating Evidence-Based Policymaking at the U.S. Department of Education

Matthew Soldner,

Commissioner, National Center for Education Evaluation and Regional Assistance &

Chief Evaluation Officer, U.S. Department of Education

(Remarks made on November 17, 2022, at Results for America’s 2022 Invest in What Works Federal Standards of Excellence events.)

If you walk into almost any kindergarten classroom today, you’ll almost always find a “Star Chart.” It’s usually displayed with pride next to the chalkboard in the front of the room. On it, the name of every student in the class. And next to each name, a long row of stars, each signifying a task earnestly mastered by a young learner. (More than forty years later, I can still remember the pride associated with finally getting a gold star for “can tie shoes,” which I was seriously delayed in getting due to an overabundance of shoes secured with Velcro in my youth.)

Today, I’m very proud to acknowledge that the U.S. Department of Education has received its own gold star. It was bestowed by Results for America, which advocates for the use of evidence in federal, state, and policymaking to improve outcomes for students, families, and communities, as part of its 2022 Federal Standards of Excellence program. Each year, participating federal agencies are evaluated on their progress in using high-quality evidence as a “north star” in policymaking. This year, Education was recognized alongside two other agencies – the Millennium Challenge Corporation and the U.S. Agency for International Development – as a top-scorer, earning the “Gold” designation.

ED’s national leadership in using evidence to inform policymaking has been a journey that now spans more than two decades. Its roots can be traced to the 107th Congress, which in 2001 reauthorized the Elementary and Secondary Education Act of 1965 as the No Child Left Behind Act and, in 2002, passed the Education Sciences Reform Act. The latter authorized the founding of a group toward which I’m somewhat partial: the Institute of Education Sciences (IES). (I would invite you to join my colleagues and me in celebrating #IESat20, now through mid-2023!) But no single event, and no distinct component of the Department, is individually responsible for our success building evidence about “what works” in education and putting that evidence to work to better serve learners, educators, and communities. I often say “evidence-building is a team sport at the Department of Education,” and it truly does take the commitment of talented professionals from across the organization to make it a reality.

This year, that team has been particularly busy. Department-wide, we have supported states, school districts, and institutions of higher education in their continued efforts to meet the challenge of pandemic recovery. Much of that work has focused on the use of evidence-based practices to accelerate learning for all students, making the most of historic investments in education such as the Coronavirus Aid, Relief, and Economic Security (CARES) and American Rescue Plan (ARP) Acts. Key partners in that work include IES’ Regional Educational Laboratories, operated by the National Center for Education Evaluation and Regional Assistance (NCEE);  the Office of Elementary and Secondary Education’s Comprehensive Centers; the Office of Special Education and Rehabilitative Services’ Technical Assistance and Research Centers; and the Office of Planning, Evaluation, and Policy Development’s Grant Policy Office.

Elsewhere in the Department, the emphasis has been on evidence-building. Here, IES has taken a particular leadership role. The National Center for Education Statistics’ (NCES’) School Pulse Panel is a critical new component in our evidence-building infrastructure. The Pulse allows us to more rapidly collect and report descriptive information about conditions on the ground in our nation’s schools, addressing topics from the extent of staffing shortages to the programs schools are offering to support learning acceleration. That and other information supports a vibrant research and development infrastructure, led by the National Centers for Education Research (NCER) and Special Education Research (NCSER). In addition to their regular education grant programs, both Centers ran special competitions in Fiscal Year 2022 specifically designed to support pandemic recovery, including those aimed at better leveraging longitudinal data to support state recovery policymaking and building evidence about the approaches states and districts used to address the pandemic, and, when possible, their effectiveness.  

In Fiscal Year 2023, more good work is already underway.

First, I would be remiss if I did not acknowledge an important investment this most recent Congress has made in the evidence-building work of the Department: authority as part of the Consolidated Appropriations Act of 2022 to reserve up to one-half of one percent from selected programs authorized by the Higher Education Act of 1965, as amended, to support high-quality research and evaluation related to the Department’s postsecondary programs. A similar set-aside for the Department’s K-12 programs dramatically catalyzed our ability to build and use evidence there – and I have every confidence this new authority, if continued, would do the same for our postsecondary portfolio.

Second, and consistent with my belief that “evidence building is a team sport,” I want to take a moment to encourage you to join the team! As an initial step, I’d like to invite you to join me and special guests from the Department in our new webinar series creatively entitled “Evidence-based Policymaking at ED: Introducing the U.S. Department of Education’s Inaugural Learning Agenda.” Across three installments, we’ll discuss the Department’s evidence-building priorities in three areas: the educator workforce; meeting students’ social, emotional, and academic needs; and increasing postsecondary value. In each, we’ll dig a bit deeper into each topic and its nexus with an equitable recovery from the COVID-19 pandemic.

I hope – but cannot promise – that Education will, this time next year, report that we’ve earned another gold star for building and using evidence in service of smart policymaking. What I can promise is that, because both educators and education policymakers will continue to need high-quality evidence to do their best work on behalf of the nation’s learners, we will do our best to help them meet challenges both old and new. Thanks to Results for America for today’s recognition, and to all those who support our nation’s students, educators, and communities every day.