First Year Accomplishments

Mark Schneider, Director of IES   |   April 23, 2019

I am nearing the end of my first year as Director of IES. I thought I would report on some of the things we have been working on over the last year. I originally intended to write a single update, but realized that it would be too long. Instead, I plan to write a few shorter pieces over the next few weeks, as time allows.

First, let me note that for the first time in over five years, all the senior leadership positions in IES have been filled with skilled professionals appointed according to the 6-year terms specified in law. IES now has an excellent team working with me to lead the Institute into the future. I also recently published proposed priorities in the Federal Register. There is a 60-day comment period so please read and comment as you see fit.

In this piece, I want to emphasize how we are focusing on the mission of IES as an applied research agency. Certainly, IES has invested—and will continue to invest—in basic research, but that is not our core function. Rather, as a science agency housed in the U.S. Department of Education, our work focuses on improving the outcomes of learners throughout the life cycle. This means an emphasis on translating research for widespread use to improve outcomes.

To more fully realize this principle, we have undertaken several concrete steps.

First, we are changing the nature of the reports that we will issue. We all know that good writing is difficult. And, despite a government-wide push for Plain Language, IES reports are not known for pithiness. As Mark Twain is credited with saying “I didn’t have time to write you a short letter, so I wrote you a long one instead.” Except in rare cases, we are not going to publish 100-page reports that don’t even make good door stops.

With the new 15-page limit, our reports are going to have to be better thought out and more concisely written. We recognize our responsibility to document methods and present auxiliary data and analysis that some of our readers will need to access. Technical appendices and supplemental material will always be made available on the web. We are thinking about how different and more modern data analytic and data visualization tools could help in the exploration of our data.

Couple this emphasis on short reports with our new writing guide, which in its entirety reads: “Strong Verbs. Short sentences.” (With thanks to Malcolm Gladwell’s podcast Revisionist History.) This is remarkably good writing advice. It is also extremely difficult. IES has hired a new communications director, Beth Greene, and we are in the process of hiring two writer/editors to help achieve the goal of better, more understandable writing.

But more important is how we are rethinking the What Works Clearinghouse website and its Practice Guides. Both are among IES’s marquee activities, and WWC staff have done excellent work in managing the growing volume of reports and studies that are now eligible for WWC review.

That said, we are striving to make the WWC website as usable as possible for the wide range of users who come to it. All too often, our own rules and procedures have made the WWC and Practice Guides prone to burying the valuable information they contain in language that is difficult to understand.

Consider the description of the Improvement Index on the WWC’s website, a key measure of an intervention’s effectiveness: “An indicator of the size of the effect from using the intervention. It is the expected change in percentile rank for an average comparison group student if the student had received the intervention, ranging from -50 to +50. At the domain level, the improvement index is only shown if the effectiveness rating is positive, potentially positive, potentially negative, or negative; dashes are displayed for mixed or no discernible effects. At the study level, the improvement index is only shown if the findings are characterized as statistically significant or substantively important (greater than +10 or less than -10); dashes are displayed for an indeterminate effect.”

I have read this description many times and my reaction is always the same: Huh?

Look for this far simpler substitute in May: An indicator of the effect of the intervention, the improvement index can be interpreted as the expected change in percentile rank for an average comparison group student if that student had received the intervention.

To be sure, communicating information about effect size is no easy matter and staff (in consultation with many experts) have been struggling with this for a long time. But we must and we will do better if the WWC is to live up to its tag line: “Find What Works based on evidence.”

Moving from What Works to What Happens

Practice Guides are among the most important products of IES, because they are central to translating research into practice. Practice Guides are now two-year, million-dollar endeavors. That’s a significant investment. But it’s what IES does (or doesn’t do) after all that time and all that money that can make a difference in changing practice. IES hasn’t done nearly enough to translate the guides into actionable information that can transform behavior and outcomes.

Others have stepped in to fill the gap. We have found several examples of how Practice Guides (what works) can be turned into products that change how education is delivered (what happens). Here are a few that we think may serve as prototypes for how a product developed with IES resources can be “taken to market” through connections to regional partners.

Based on work from the WWC, Radford University (VA) is developing micro-credentials for teachers, especially teachers located in rural areas. Radford proudly calls attention to the fact that the training uses “evidence-based tools and resources from the Institute of Education Sciences (IES) What Works Clearinghouse (WWC) practice guides.” Moreover—and this is critical—teachers have a reason to enroll in this program since the micro-credentials can be used as professional development points for licensure renewal and potentially graduate credit. More information about their program can be found here.

North Carolina State University’s Friday Institute has turned information from two different Practice Guides into MOOCs for teacher professional development. Its Fraction Foundations MOOC is organized around the recommendations of the Practice Guide on Developing Effective Fractions Instruction for Kindergarten Through 8th Grade, published by the WWC in September 2010. The Teaching Foundational Reading Skills MOOC is organized around the recommendations of Foundational Skills to Support Reading for Understanding in Kindergarten Through 3rd Grade, published in July 2016. As with Radford’s program, both MOOCs award continuing education credit to teachers who complete the course.

IES has never sufficiently prioritized translating and testing tools and products that could flow from IES-supported research, and we have never fully mobilized our resources in this translation process. Some of you may remember the Department of Education’s Doing What Works project, which turned WWC Practice Guides into products to support their use by teachers. That project ended several years ago, not because the idea wasn’t a good one but because the follow-up materials were not sufficiently tested. Finding the balance between generating knowledge and applying it is essential as we move to turn what works into what happens.

NCEE, under the leadership of Commissioner Matt Soldner, is considering how future generations of Regional Educational Laboratories (RELs) could take center stage in IES’s translation function. There are already examples of RELs doing this work, such as teacher-focused tools developed by REL Southwest based on the Teaching Academic Content and Literacy to English Learners in Elementary and Middle School practice guide. At their best, RELs build strong partnerships with states, districts, and teachers around research, development, and evaluation capacity. How they can contribute even more to helping IES moving its work into the field is crucial. Expect to hear more from Matt on this in the coming weeks.

As always, I am more than happy to hear from you about this and other issues (including my proposed priorities). There is only a handful of us at IES and hundreds upon hundreds of smart people in the field. We have no monopoly on good ideas—so please share your good ideas with us!

Mark Schneider
Director of IES
Mark.Schneider@ed.gov