NCEE Blog

National Center for Education Evaluation and Regional Assistance

The Role of RELs in Making WWC Practice Guides Actionable for Educators

Earlier this year, I wrote a short blog about how I envisioned the Regional Educational Laboratories (REL) Program, The What Works Clearinghouse™ (WWC), and the Comprehensive Center Program could work together to take discovery to scale. In it, I promised I would follow-up with more thoughts on a specific—and critically important—example: making WWC Practice Guides actionable for educators. I do so below. At the end of this blog, I pose a few questions on which I welcome comments.

The challenge. The single most important resources the WWC produces are its Practice Guides. Practice Guides evaluate the research on a given topic—say, teaching fractions in elementary and middle school—and boil study findings down to a handful of evidence-based practices for educators. Each practice is given a rating to indicate the WWC’s confidence in the underlying evidence, along with tips for how practices can be implemented in the classroom. In many ways, Practice Guides are IES’s most specific and definitive statements about what works to improve education practice and promote student achievement.

Despite their importance, the amount of effort IES has intentionally dedicated to producing high-quality resources that support educators in implementing Practice Guide recommendations has been uneven. (By most measures, it has been on the decline.) Why? Although we have confidence that the materials we have already produced are high-quality, we cannot prove it. Rigor is part of our DNA, and the absence of systematic efficacy tests demonstrating tools’ contribution to improved teacher practice has made us hesitant to dramatically expand IES-branded resources.

To their credit, several organizations have stepped in to address the “last mile problem” between Practice Guides and classroom practice. Some, like RELs, are IES partners. As a result, we have seen a small number of Practice Guides turned in to professional learning community guides, massively on-line open courses, and other teacher-facing resources. Despite these efforts, similar resources have not been developed for the overwhelming majority of Practice Guides. This means many of our Guides and the dozens of recommendations for evidence-based practice they contain are languishing underused on IES’s virtual bookshelf.

An idea. IES should “back” the systematic transformation of Practice Guide recommendations from words on a page to high-quality materials that support teachers’ use of evidence-based practices in their classrooms. And because we should demonstrate our own practice works, those materials should be tested for efficacy.

From my perspective, RELs are well-suited to this task. This work unambiguously aligns with RELs’ purpose, which is to improve student achievement using scientifically-valid research. It also leverages RELs’ unique value proposition among federal technical assistance providers: the capacity to conduct rigorous research and development activities in partnership with state and local educators. If RELs took on a greater role in supporting Practice Guides in the next REL cycle—which runs from 2022 until 2027—what might it look like in practice?

One model involves RELs collaborating with state and/or district partners to design, pilot, and test a coherent set of resources (a “toolkit”) that help educators bring Practice Guide recommendations to life in the classroom. Potential products might include rubrics to audit current policy or practice, videos of high-quality instructional practice, sample classroom materials, or professional learning community facilitation guides, each linked to one or more Practice Guide recommendations.

Long-time followers of the WWC may recognize the design aspect of this work as similar to the defunct Doing What Works Program. The difference? New resources would not only be developed in collaboration with educators, they must be piloted and tested with them as well. It’s simple, really: if we expect educators to use evidence-based practices in the classroom, we need evidence-based tools to help teachers succeed when implementing them.  

Once vetted, materials must get into the hands of educators who need them. It’s here where the value of the REL-Comprehensive Center partnership becomes clear. With a mission of supporting each state education agency in its school improvement efforts, Regional Comprehensive Centers are in the ideal position to bring resources and implementation supports to state and local education leaders that meet their unique needs. Tools that are developed, piloted, and refined by a REL and educators in a single state can then be disseminated by the national network of Comprehensive Centers to meet other states’ needs.

Extensions. It isn’t hard to imagine other activities that the WWC, RELs, and Comprehensive Centers might take on to maximize this model’s potential effectiveness. Most hinge on building effective feedback loops.

Promoting continuous improvement of Practice Guide resources is an obvious example. RELs could and should be in the business of following Comprehensive Centers as they work with states and districts to implement REL-developed Practice Guide supports, looking for ways to maximize their effectiveness. Similarly, Comprehensive Centers and RELs should be regularly communicating with one another about needs-sensing, identifying areas where support for evidence-based practice is lacking and determining which partners to involve in the solution. When there is a growing body of evidence to support educator best practice, the WWC is in the best position to take the lead and develop a new Practice Guide. When that body of evidence does not exist yet—or when even the practices themselves are underdeveloped—the RELs and other parts of IES, such as the National Centers for Education and Special Education Research, should step in.  

Questions. When the WWC releases a new Practice Guide, its work may be done—at least temporarily. The work of its partners to support take-up of a Guide’s recommendations will, however, have just begun. I’d appreciate your thoughts on how to best accomplish that transition, and offer up the following additional questions for your consideration:

  1. Are we thinking about the problem correctly, and in a helpful way? Are there elements of the problem that should be redefined, and would that lead us to different solutions?

 

  1. What parts of the problem does this proposed solution address well, and where are its shortcomings? Are there other solutions—even solutions that don’t seem to fit squarely within today’s model of the REL Program—that might be more effective?

 

  1. If we proceed under a model like that which is described above:

 

  1. What sort of REL partnership models would be most effective in supporting the conceptualization, design, piloting, and testing of teacher-facing “toolkits” aligned to WWC Practice Guides?

 

  1. What research and evaluation activities—and which outcome measures—should be incorporated into this activity to give IES confidence that the resulting “toolkits” are likely to be associated with changed teacher practice and improved student outcomes?

 

  1. How does the 5-year limit on REL contracts affect the feasibility of this idea, including its scope and cost? What could be accomplished in 5 years, and what might take longer to see to completion?

 

  1. How could RELs leverage existing ED-sponsored content, such as that created by Doing What Works, in service of this new effort?

 

If you have thoughts on these questions or other feedback you would like to share, please e-mail me. I can be reached directly at matthew.soldner@ed.gov. Thanks in advance for the consideration!

by Matthew Soldner, NCEE Commissioner 

Taking Discovery to Scale

Along with my NCEE colleagues, I was excited to read the recent Notice Inviting Applications for the next cycle of Comprehensive Centers, administered by the Department’s Office of Elementary and Secondary Education.

As you can see in the notice, Regional Comprehensive Centers will “provide high-quality intensive capacity-building services to State clients and recipients to identify, implement, and sustain effective evidence-based programs, practices, and interventions that support improved educator and student outcomes,” with a special emphasis on benefitting disadvantaged students, students from low-income families, and rural populations.

With this focus on supporting implementation, Regional Comprehensive Centers (RCCs) can amplify the work of NCEE’s Regional Educational Laboratories (RELs) and What Works Clearinghouse (WWC). Learning from states, districts, and schools to understand their unique needs, and then being able to support high-quality implementation of evidence-based practices that align with those needs, has the potential to dramatically accelerate the process of improving outcomes for students.

RELs and the WWC already collaborate with today’s Comprehensive Centers, of course. But it’s easy to see how stronger and more intentional relationships between them could increase each program’s impact.

True to its name, the REL program has worked with educators to design and evaluate innovative practices – or identify, implement, and refine existing ones – to meet regional and local needs for more than 50 years. And since its inception in 2002, the WWC has systematically identified and synthesized high-quality evidence about the effectiveness of education programs, policies, and practices so that educators and other instructional leaders can put that information to use improving outcomes for students. But with more than 3.6 million teachers spread across more than 132,000 public and private schools nation-wide, making sure discoveries from education science are implemented at scale and with fidelity is no small feat. RCCs are welcome partners in that work.

This figure describes how RELs, the What Works Clearinghouse, and Regional Comprehensive Centers could most effectively collaborate across a continuum from discovery to scale.

RELs, the WWC, and Comprehensive Centers can play critical, complementary roles in taking discovery to scale (see Figure). With their analysis, design, and evaluation expertise, RELs – in partnership with states and districts, postsecondary institutions, and other stakeholders – can begin the process by designing and rigorously evaluating best practices that meet local or regional needs. (Or, as I will discuss in future messages, by developing and rigorously testing materials that support adoption of evidence-based practices.) The WWC follows, vetting causal impact studies, synthesizing their findings to better understand the strength of evidence that supports a practice and identifying its likely impact. Partners in the Comprehensive Centers can then “pick-up” those WWC-vetted practices, aligning them to needs of State and other clients, and supporting and sustaining implementation at scale. Finally, lessons learned from RCCs’ implementation efforts about what worked – and what didn’t – can be fed back to RELs, refining the practice and fueling the next cycle of discovery.

Those that follow the REL-WWC-RCC process know that what I’ve just described isn’t quite how these programs operate today. Sometimes, out of necessity, roles are more “fluid” and efforts are somewhat less well-aligned. The approach of “taking discovery to scale” depicted above provides one way of thinking about how each program can play a unique, but interdependent, role with the other two.

I have every confidence this is possible. After all, the North star of each program is the same: improving outcomes for students. And that means we have a unique opportunity. One we’d be remiss not to seize.

 

Matthew Soldner
Commissioner, National Center for Education Evaluation and Regional Assistance
Institute of Education Sciences
U.S. Department of Education

 

As always, your feedback is welcome. You can email the Commissioner at matthew.soldner@ed.gov.