NCEE Blog

National Center for Education Evaluation and Regional Assistance

The Role of RELs in Making WWC Practice Guides Actionable for Educators

Earlier this year, I wrote a short blog about how I envisioned the Regional Educational Laboratories (REL) Program, The What Works Clearinghouse™ (WWC), and the Comprehensive Center Program could work together to take discovery to scale. In it, I promised I would follow-up with more thoughts on a specific—and critically important—example: making WWC Practice Guides actionable for educators. I do so below. At the end of this blog, I pose a few questions on which I welcome comments.

The challenge. The single most important resources the WWC produces are its Practice Guides. Practice Guides evaluate the research on a given topic—say, teaching fractions in elementary and middle school—and boil study findings down to a handful of evidence-based practices for educators. Each practice is given a rating to indicate the WWC’s confidence in the underlying evidence, along with tips for how practices can be implemented in the classroom. In many ways, Practice Guides are IES’s most specific and definitive statements about what works to improve education practice and promote student achievement.

Despite their importance, the amount of effort IES has intentionally dedicated to producing high-quality resources that support educators in implementing Practice Guide recommendations has been uneven. (By most measures, it has been on the decline.) Why? Although we have confidence that the materials we have already produced are high-quality, we cannot prove it. Rigor is part of our DNA, and the absence of systematic efficacy tests demonstrating tools’ contribution to improved teacher practice has made us hesitant to dramatically expand IES-branded resources.

To their credit, several organizations have stepped in to address the “last mile problem” between Practice Guides and classroom practice. Some, like RELs, are IES partners. As a result, we have seen a small number of Practice Guides turned in to professional learning community guides, massively on-line open courses, and other teacher-facing resources. Despite these efforts, similar resources have not been developed for the overwhelming majority of Practice Guides. This means many of our Guides and the dozens of recommendations for evidence-based practice they contain are languishing underused on IES’s virtual bookshelf.

An idea. IES should “back” the systematic transformation of Practice Guide recommendations from words on a page to high-quality materials that support teachers’ use of evidence-based practices in their classrooms. And because we should demonstrate our own practice works, those materials should be tested for efficacy.

From my perspective, RELs are well-suited to this task. This work unambiguously aligns with RELs’ purpose, which is to improve student achievement using scientifically-valid research. It also leverages RELs’ unique value proposition among federal technical assistance providers: the capacity to conduct rigorous research and development activities in partnership with state and local educators. If RELs took on a greater role in supporting Practice Guides in the next REL cycle—which runs from 2022 until 2027—what might it look like in practice?

One model involves RELs collaborating with state and/or district partners to design, pilot, and test a coherent set of resources (a “toolkit”) that help educators bring Practice Guide recommendations to life in the classroom. Potential products might include rubrics to audit current policy or practice, videos of high-quality instructional practice, sample classroom materials, or professional learning community facilitation guides, each linked to one or more Practice Guide recommendations.

Long-time followers of the WWC may recognize the design aspect of this work as similar to the defunct Doing What Works Program. The difference? New resources would not only be developed in collaboration with educators, they must be piloted and tested with them as well. It’s simple, really: if we expect educators to use evidence-based practices in the classroom, we need evidence-based tools to help teachers succeed when implementing them.  

Once vetted, materials must get into the hands of educators who need them. It’s here where the value of the REL-Comprehensive Center partnership becomes clear. With a mission of supporting each state education agency in its school improvement efforts, Regional Comprehensive Centers are in the ideal position to bring resources and implementation supports to state and local education leaders that meet their unique needs. Tools that are developed, piloted, and refined by a REL and educators in a single state can then be disseminated by the national network of Comprehensive Centers to meet other states’ needs.

Extensions. It isn’t hard to imagine other activities that the WWC, RELs, and Comprehensive Centers might take on to maximize this model’s potential effectiveness. Most hinge on building effective feedback loops.

Promoting continuous improvement of Practice Guide resources is an obvious example. RELs could and should be in the business of following Comprehensive Centers as they work with states and districts to implement REL-developed Practice Guide supports, looking for ways to maximize their effectiveness. Similarly, Comprehensive Centers and RELs should be regularly communicating with one another about needs-sensing, identifying areas where support for evidence-based practice is lacking and determining which partners to involve in the solution. When there is a growing body of evidence to support educator best practice, the WWC is in the best position to take the lead and develop a new Practice Guide. When that body of evidence does not exist yet—or when even the practices themselves are underdeveloped—the RELs and other parts of IES, such as the National Centers for Education and Special Education Research, should step in.  

Questions. When the WWC releases a new Practice Guide, its work may be done—at least temporarily. The work of its partners to support take-up of a Guide’s recommendations will, however, have just begun. I’d appreciate your thoughts on how to best accomplish that transition, and offer up the following additional questions for your consideration:

  1. Are we thinking about the problem correctly, and in a helpful way? Are there elements of the problem that should be redefined, and would that lead us to different solutions?

 

  1. What parts of the problem does this proposed solution address well, and where are its shortcomings? Are there other solutions—even solutions that don’t seem to fit squarely within today’s model of the REL Program—that might be more effective?

 

  1. If we proceed under a model like that which is described above:

 

  1. What sort of REL partnership models would be most effective in supporting the conceptualization, design, piloting, and testing of teacher-facing “toolkits” aligned to WWC Practice Guides?

 

  1. What research and evaluation activities—and which outcome measures—should be incorporated into this activity to give IES confidence that the resulting “toolkits” are likely to be associated with changed teacher practice and improved student outcomes?

 

  1. How does the 5-year limit on REL contracts affect the feasibility of this idea, including its scope and cost? What could be accomplished in 5 years, and what might take longer to see to completion?

 

  1. How could RELs leverage existing ED-sponsored content, such as that created by Doing What Works, in service of this new effort?

 

If you have thoughts on these questions or other feedback you would like to share, please e-mail me. I can be reached directly at matthew.soldner@ed.gov. Thanks in advance for the consideration!

by Matthew Soldner, NCEE Commissioner 

In Nebraska, a focus on evidence-based reading instruction

Researchers have learned a lot about reading over the last few decades, but these insights don’t always make their way into elementary school classrooms. In Nebraska, a fresh effort to provide teachers with practical resources on reading instruction could help bridge the research-practice divide.

At the start of the 2019-2020 school year, the Nebraska Department of Education (NDE) implemented NebraskaREADS to help “serve the needs of students, educators, and parents along the journey to successful reading.” As part of a broader effort guided by the Nebraska Reading Improvement Act, the initiative emphasizes the importance of high-quality reading instruction and targeted, individualized support for struggling readers.

As part of NebraskaREADS, NDE is developing an online resource inventory of tools and information to support high-quality literacy instruction for all Nebraska students. After reaching out to REL Central for support, experts from both agencies identified the What Works Clearinghouse (WWC) as a prime source for evidenced-based information on instructional practices and policies and partnered with REL Central to develop a set of instructional strategies summary documents based on recommendations in eight WWC practice guides.

Each WWC practice guide presents recommendations for educators based on reviews of research by a panel of nationally recognized experts on a particular topic or challenge. NDE’s instructional strategies summary documents align the evidence-based strategies in the practice guides more directly with NebraskaREADS.

REL Central and NDE partners used the WWC practice guides to develop the 35 instructional strategies summary documents, each of which condenses one recommendation from a practice guide. For each recommendation, the instructional strategies summary document provides the associated NebraskaREADS literacy focus, implementation instructions, appropriate grade levels, potential roadblocks and ways to address them, and the strength of supporting evidence.

NDE launched the instructional strategies summary documents in spring 2019, and already educators have found them to be helpful resources. Dee Hoge, executive director of Bennington Public Schools, described the guides as “checkpoints for strong instruction” and noted her district’s plan to use them to adapt materials and provide professional development. Marissa Payzant, English language arts education specialist at NDE and a lead partner in this collaboration, shared that the “districts are excited to incorporate the strategy summaries in a variety of professional learning and other initiatives. They make the information in the practice guides much more accessible, and all the better it’s from a trustworthy source.”

Building on the positive reception and benefits of the instructional strategies summary documents, NDE and REL Central plan to expand the resource inventory into other content areas. Currently, they are working together to develop instructional strategy summaries using the WWC practice guides for math instruction.

by Douglas Van Dine‚Äč, Regional Educational Laboratory Central

Taking Discovery to Scale

Along with my NCEE colleagues, I was excited to read the recent Notice Inviting Applications for the next cycle of Comprehensive Centers, administered by the Department’s Office of Elementary and Secondary Education.

As you can see in the notice, Regional Comprehensive Centers will “provide high-quality intensive capacity-building services to State clients and recipients to identify, implement, and sustain effective evidence-based programs, practices, and interventions that support improved educator and student outcomes,” with a special emphasis on benefitting disadvantaged students, students from low-income families, and rural populations.

With this focus on supporting implementation, Regional Comprehensive Centers (RCCs) can amplify the work of NCEE’s Regional Educational Laboratories (RELs) and What Works Clearinghouse (WWC). Learning from states, districts, and schools to understand their unique needs, and then being able to support high-quality implementation of evidence-based practices that align with those needs, has the potential to dramatically accelerate the process of improving outcomes for students.

RELs and the WWC already collaborate with today’s Comprehensive Centers, of course. But it’s easy to see how stronger and more intentional relationships between them could increase each program’s impact.

True to its name, the REL program has worked with educators to design and evaluate innovative practices – or identify, implement, and refine existing ones – to meet regional and local needs for more than 50 years. And since its inception in 2002, the WWC has systematically identified and synthesized high-quality evidence about the effectiveness of education programs, policies, and practices so that educators and other instructional leaders can put that information to use improving outcomes for students. But with more than 3.6 million teachers spread across more than 132,000 public and private schools nation-wide, making sure discoveries from education science are implemented at scale and with fidelity is no small feat. RCCs are welcome partners in that work.

This figure describes how RELs, the What Works Clearinghouse, and Regional Comprehensive Centers could most effectively collaborate across a continuum from discovery to scale.

RELs, the WWC, and Comprehensive Centers can play critical, complementary roles in taking discovery to scale (see Figure). With their analysis, design, and evaluation expertise, RELs – in partnership with states and districts, postsecondary institutions, and other stakeholders – can begin the process by designing and rigorously evaluating best practices that meet local or regional needs. (Or, as I will discuss in future messages, by developing and rigorously testing materials that support adoption of evidence-based practices.) The WWC follows, vetting causal impact studies, synthesizing their findings to better understand the strength of evidence that supports a practice and identifying its likely impact. Partners in the Comprehensive Centers can then “pick-up” those WWC-vetted practices, aligning them to needs of State and other clients, and supporting and sustaining implementation at scale. Finally, lessons learned from RCCs’ implementation efforts about what worked – and what didn’t – can be fed back to RELs, refining the practice and fueling the next cycle of discovery.

Those that follow the REL-WWC-RCC process know that what I’ve just described isn’t quite how these programs operate today. Sometimes, out of necessity, roles are more “fluid” and efforts are somewhat less well-aligned. The approach of “taking discovery to scale” depicted above provides one way of thinking about how each program can play a unique, but interdependent, role with the other two.

I have every confidence this is possible. After all, the North star of each program is the same: improving outcomes for students. And that means we have a unique opportunity. One we’d be remiss not to seize.

 

Matthew Soldner
Commissioner, National Center for Education Evaluation and Regional Assistance
Institute of Education Sciences
U.S. Department of Education

 

As always, your feedback is welcome. You can email the Commissioner at matthew.soldner@ed.gov.

 

 

How can we work together to promote achievement for all students?

Matthew Soldner, Commissioner of the National Center for Education Evaluation and Regional Assistance, delivered the remarks below at Regional Educational Laboratory (REL) Midwest’s February 27, 2019 Governing Board Meeting in Chicago, Illinois. The remarks have been edited for clarity and to remove references to specific attendees or projects.

Good evening, and thank you for inviting me to share a few thoughts this evening about the Institute of Education Sciences’ vision for the REL program. I will promise to keep them brief, because I know you want to hear from REL Midwest about the work they have planned for the upcoming year, and I know they want to hear from you about the planned work and how it can be designed to meet your needs and the needs of your stakeholders.

As you meet over the course of the next day, I’d ask that you keep one question in mind throughout: how can our work together, across the various partnerships that are represented in this room, work to promote achievement for all students. I want to spend a moment on a few of those words.

First, achievement. When I talk about achievement, I’m not referring to only test scores or grades. I’m talking about measures and indicators of development and success from early childhood through adulthood, including outcomes in early childhood education, early and middle grades, high school, and college and university. This also must include indicators of success as learners move to and through the workforce.

Second, when I say all students – or, perhaps more precisely, all learners – I mean it in the most inclusive terms. We are deeply committed to ensuring each student, each learner, is well-served by our systems of education – from pre-Kindergarten to adult education, and all levels in-between.  

So what must we, as a REL program, do to work toward that goal? I think most of us would agree that nothing changes for students if adults don’t begin to do things differently and, hopefully, better.

That means our work must be focused on action. The kind of action you tell us is most needed in your states, your districts, and your communities.

Some of you are probably saying: “But I thought this work was about RESEARCH? Doesn’t the ‘L’ in ‘REL’ imply that we are out to experiment, test, and discover? Not ACTION?”

The answer is, of course, yes: Research is core to the distinctiveness of the REL program. Research, and a reliance on evidence in classroom practice and policymaking is at the foundation of everything that we do. And yes, in all of our work, we hope to inspire among our partners a desire and capacity to better use data and evidence.

But it cannot end there. The research that we do together must be in service of the action – of the change – around which you have invited us into your work. It must be part of a larger, coherent effort to improve the achievement of all students. Research is a means to an end, but it is not the end this program is meant to achieve.

I would offer one word of caution. This is not just, or even mostly, about improving dissemination. It isn’t about a better tweet, a better infographic, or a better video. We cannot be in the business of just putting research in peoples’ hands and expecting change.

Instead, this is about being in active partnership with you. And putting that relationship to work so that what we know and what we are learning can support the policy, program, and practice goals you have set to support all students.

I do not believe this is a radical departure from how this community thinks about its work. But it may call us to do our work with a different kind of intentionality.

So my ask of you, my charge to you, is that as Governing Board members and stakeholders you consistently challenge us to leverage the research, evaluation, and technical assistance skills of the REL staff in this room in ways that make a real difference in the lives of the learners you serve. Thank you being good partners with us on this journey. As always, please feel free to reach out to me directly if you have thoughts about how we might do our work better.

Bringing Evidence-based Practices to the Field

By Dr. Barbara Foorman, Director Emeritus, Florida Center for Reading Research, Florida State University

The Institute of Education Sciences recently released a What Works Clearinghouse (WWC) educator’s practice guide that has four recommendations to support the development of foundational reading skills that are critically important to every student’s success. The recommendations in Foundational Skills to Support Reading for Understanding in Kindergarten Through 3rd Grade are based on a comprehensive review of 15 years of research on reading, and guidance from a national panel of reading experts, of which I was the chair.

Recently, the Regional Educational Laboratory (REL) Southeast at Florida State University has developed a set of professional learning community (PLC) materials and videos to help teachers and other practitioners implement the guide’s recommendations in classrooms.

Over the past few months, REL Southeast has shared the practice guide and PLC materials with practitioners and policymakers in two states – North Carolina and Mississippi, which both have K-3 reading initiatives and reading coaches who assist with implementation. I’m excited by the feedback we are getting.

During these presentations, we shared the format of the ten 75-min PLC sessions and accompanying videos that demonstrate the recommendations and action steps in actual classrooms. We filmed the videos in partnership with Dr. Lynda Hayes, Director of the PK Yonge Developmental Research School at the University of Florida, and her primary grade teachers.

In North Carolina, we trained K–3 regional literacy consultants, elementary teachers and reading coaches, and higher education faculty on the PLC Facilitator’s Guide in Charlotte and Raleigh. The K-3 regional literacy consultants are organized by the North Carolina Department of Public Instruction.

In Mississippi, we trained the 90 Mississippi Department of Education reading coaches and district-supported special education specialists in Jackson. In turn, the state coaches will train the K–3 teachers who are a part of the reading initiative in the practice guide recommendations and action steps. Additionally, the coaches will work with the primary grade teachers in each of their assigned schools to implement the PLC. Having the state coaches oversee the implementation of the PLC will help ensure commitment and instill depth to the PLC sessions.

Also present at the training in Mississippi were faculty members from the University of Mississippi and Belhaven University. I accepted an invitation from the Mississippi Institutions of Higher Learning Literacy Council to speak to higher education faculty about the guide and PLC materials. The invitation is timely because Mississippi recently completed a study of teacher preparation for early literacy instruction.

I hope you will download the practice guide and PLC materials. If you have any thoughts, comments, or questions, please email Contact.IES@ed.gov. You can learn more about the work of the Regional Educational Laboratories program and REL Southeast on the IES website.  

Dr. Foorman is the Director of REL Southeast, located at Florida State University

Responding to the Needs of the Field

By Chris Boccanfuso Education Research Analyst, NCEE

One of the most commonly asked questions about the Regional Educational Laboratory, or REL, program is how we choose the applied research and evaluation studies, analytic technical assistance, and dissemination activities we provide for free to stakeholders every year. The answer is simple – we don’t!

Instead, the REL staff at the Institute of Education Sciences (IES) and our contractors who run the nation’s ten regional labs listen to the voices of teachers, administrators, policymakers, and students to identify and address high-leverage problems of practice and build the capacity of stakeholders. In other words, these groups determine where the RELs should use their resources to make the largest, most lasting impact possible on practice, policy and, ultimately, student achievement.

How do the RELs do this? Through a variety of activities we collectively refer to as needs sensing. The following are a few examples of how the RELs engage in needs sensing:

Research Alliances: Research Alliances are a type of researcher–practitioner partnership where a group of education stakeholders convene around a specific topic of concern to work collaboratively to investigate a problem and build capacity to address it. Alliances can be made up of many types of stakeholders, such as teachers, administrators, researchers and members of community organizations. Alliances can vary in size and scope and address a variety of topics. For instance, alliances have formed to address topics as broad as dropout prevention and as specific as Hispanic students’ STEM performance. The vast majority of the RELs’ work is driven by these research alliances.

While the RELs’ 79 research alliances are incredibly diverse, one thing each alliance has in common is that they collectively develop a research agenda. These agendas can change, as the alliance continually weighs the questions and needs of various groups against the types of services and the resources available to address these needs. Not every need has to be addressed through a multi-year research study. Sometimes, it can be addressed through a workshop, a literature review, or a “Bridge Event”, where national experts on certain topic work with practitioners to provide the information that alliance members need, when they need it. Sometimes, a need is state or district-specific, is related to the impact of a specific program, or covers a topic where the existing research literature is thin. In these cases, a research study may be most appropriate.

Governing Boards: Another way that RELs determine their work is through their Governing Boards.  By law, each REL is required to have a Governing Board that consists of the Chief State School Officers (or their designee) for each state, territory, or freely associated state in the region. The Board also includes carefully selected members who equitably represent each state, as well as a broad array of regional interests, such as educating rural and economically disadvantaged populations. (A 2013 REL Northeast and Islands Governing Board meeting is pictured here.)
 
Governing Boards typically include a mix of people with experience in research, policy and teaching practice. Each Governing Board meets two to three times per year to discuss, direct, advise, and approve each REL project that occurs in that region. The intent is to ensure that the work being done by the REL is timely, high-leverage, equitably distributed across the region, and not redundant with existing efforts.                          

“Ask a REL”: A third way in which the RELs engage in needs sensing is through the Ask a REL service. Ask a REL is a publicly available reference desk service that functions much like a technical reference library. It provides requestors with references, referrals, and brief responses in the form of citations on research-based education questions. RELs are able to examine trends in the topics of Ask a REL requests to verify needs determined through other methods, as well as identify new topics that may warrant additional attention.   

RELs use many additional ways to explore the needs of their region, including scans of regional education news sites, reviews of recently published research, and Stakeholder Feedback Surveys that are filled out by alliance members and attendees at REL events.

It’s a thorough and ongoing process that RELs are engaging in to address authentic, high-leverage problems of practice in a variety of ways. In the coming months, we will share stories of the many projects that were informed by this needs sensing process. Stay tuned!

 

Regional Educational Laboratories: Connecting Research to Practice

By Joy Lesnick, Acting Commissioner, NCEE

Welcome to the NCEE Blog! 

Joy Lesnick

We look forward to using this space to provide information and insights about the work of the National Center for Education Evaluation and Regional Assistance (NCEE). A part of the Institute of Education Sciences (IES), NCEE’s primary goal is providing practitioners and policymakers with research-based information they can use to make informed decisions. 

We do this in a variety of ways, including large-scale evaluations of education programs and practices supported by federal funds; independent reviews and syntheses of research on what works in education; and a searchable database of research citations and articles (ERIC) and reference searches from National Library of Education. We will explore more of this work in future blogs, but in this post I’d like to talk about an important part of NCEE—the Regional Educational Laboratories (RELs).

It’s a timely topic. Last week, the U.S. Department of Education released a solicitation for organizations seeking to become REL contractors beginning in 2017 (the five-year contracts for the current RELs will conclude at the end of 2016). The REL program is an important part of the IES infrastructure for bridging education research and practice. Through the RELs, IES seeks to ensure that research does not “sit on a shelf” but rather is broadly shared in ways that are relevant and engaging to policymakers and practitioners. The RELs also involve state and district staff in collaborative research projects focused on pressing problems of practice. An important aspect of the RELs’ work is supporting the use of research in education decision making – a charge that the Every Student Succeeds Act has made even more critical.

The RELs and their staff must be able to navigate comfortably between the two worlds of education research and education practice, and understand the norms and requirements of both.  As part of this navigating, RELs focus on: (1) balancing rigor and relevance; (2) differentiating support to stakeholders based on need; (3) providing information in the short term, and developing evidence over the long term; and (4) addressing local issues that can also benefit the nation.

While the RELs are guided by federal legislation, their work reflects – and responds to – the needs of their communities. Each REL has a governing board comprised of state and local education leaders that sets priorities for REL work. Also, nearly all REL work is conducted in collaboration with research alliances, which are ongoing partnerships in which researchers and regional stakeholders work together over time to use research to address an education problem.  

Since the current round of RELs were awarded in 2012, these labs and their partners have conducted meaningful research resulting in published reports and tools, held hundreds of online and in-person seminars and training events that have been attended by practitioners across the country, and produced videos of their work that you can find on the REL Playlist on the IES YouTube site. Currently, the RELs have more than 100 projects in progress. RELs do work in nearly every topic that is crucial to improving education—kindergarten readiness, parent engagement, discipline, STEM education, college and career readiness, teacher preparation and evaluation, and much more.

IES’s vision is that the 2017–2022 RELs will build on and extend the current priorities of high-quality research, genuine partnership, and effective communication, while also tackling high-leverage education problems.  High-leverage problems are those that: (1) if addressed could result in substantial improvements in education outcomes for many students or for key subgroups of students; (2) are priorities for regional policymakers, particularly at the state level; and (3) require research or research-related support to address well. Focusing on high-leverage problems increases the likelihood that REL support ultimately will contribute to improved student outcomes.

Visit the IES REL website to learn more about the 2012-2017 RELs and how you can connect with the REL that serves your region.  Visit the FedBizOpps website for information about the competition for the 2017-2022 RELs.