IES Blog

Institute of Education Sciences

Persistence Pays Off: Introducing the New WWC Website

By Ruth Curran Neild, Delegated Director, IES

Stick with it, teachers tell their students.  Don’t give up when the going gets tough. Important work is often difficult. Keep at it.

More than a decade ago, it took this kind of dogged determination to launch an entity that would review education research against rigorous standards—the What Works Clearinghouse (WWC).   There were some skeptics, but also many supporters. And year after year, the WWC persisted, building out new topic areas, adding systematic reviews and practice guides, and identifying more interventions with positive effects on student outcomes.

Over time, WWC standards began to influence the field, with researchers designing studies that would meet its rigorous expectations. At the same time, researchers provided input that helped us improve WWC and its standards. Now, there are about 1,000 effectiveness studies available that meet our high bar for research design.  

By 2014, there was so much content in the WWC and so many people using the WWC website that it was time to retool our online presence and make our resources easier to find.

So IES staff and WWC contractors undertook an extensive project to create a more flexible database and intuitive search engine that meets a variety of user needs. A new relational database was built from the ground up, using study review data previously kept in thousands of different spreadsheets. Focus groups and user testing allowed us to identify the most important functionalities and best design features.

After more than two years of work, we launched the new WWC website today (Sept. 13), with a much-improved Find What Works feature that makes it easier to identify interventions, programs, and policies have improved student outcomes. (The video below can help you learn how to navigate the new site). The new WWC site has something for all types of users:

Do you want just top-level information about a program’s effects?  We can give you that.

Do you want to dig into the details of a particular study, including outcome domains, population, geographic context, and implementation?  We can give you that, too.

Do you want a quick assessment of whether a study was conducted with students like yours?  Find What Works can do that.

Do you want an easy way to compare the research on interventions?  Yes, absolutely – that’s a new feature.  

Do you want to see a list of randomized controlled trials (RCTs) that meet WWC standards and found at least one statistically significant positive effect?  No problem.

This is a proud day indeed for the What Works Clearinghouse, but there’s more to come.  We are always working on new resources for our users. In the next few months, look for a new practice guide on teaching writing in grades 6-12, as well as online WWC reviewer training, which we hope will help to meet the high demand for this credential. And we are continuing to expand our topic areas, including reviews of studies of postsecondary education programs to identify those that are showing promise for improving student outcomes.  

A lot of hard work and persistence went into this project from many people, including IES staff and the teams from Mathematica Policy Research and Sanametrix that worked on the site. I want to say “thank you” to them all. I also want to thank the many people who have been a part of the WWC over the years; including IES team members, study reviewers, and contractors. They have helped get us to this point today.  

Most of all, I want to thank the hundreds of thousands of educators, researchers, and decision makers who visit and use the WWC each year. We are glad that the What Works Clearinghouse has been a part of your work and hope it will continue to be a go-to resource for many years to come. 

 

Recognizing Our Outstanding Predoctoral Fellows

Each year, the Institute of Education Sciences (IES) recognizes an outstanding fellow from its Predoctoral Interdisciplinary Research Training Programs in the Education Sciences for academic accomplishments and contributions to education research. For the first time, IES has selected joint recipients for the 2015 award: Meghan McCormick and Eric Taylor. They will receive their awards and present their research at the annual IES Principal Investigators meeting in Washington, D.C. in December 2016.

Meghan completed her Ph.D. in Applied Psychology at New York University and wrote her dissertation on the efficacy of INSIGHTS, a social emotional learning intervention aimed at improving low-income urban students’ academic achievement. She is currently a research associate at MDRC. Eric completed his PhD in the Economics of Education from Stanford University and wrote his dissertation on the contributions of the quality and quantity of classroom instruction to student learning. Eric is currently an assistant professor at the Harvard University Graduate School of Education.

We asked Meghan and Eric how participating in an IES predoctoral training program helped their development as researchers.  For more information about the IES predoctoral training program, visit our website.

Meghan McCormick

Having the opportunity to be part of the IES Predoctoral Training Program helped me to develop a set of theoretical, quantitative, and practical skills that I would not have had the opportunity to develop otherwise. 

I was drawn to attend NYU (New York University) for my doctoral studies specifically because the school of education at NYU offered the (IES) predoctoral training program, in addition to hosting a core set of faculty with research interests very much aligned with my own. In my first year of graduate school, I quickly became aware that being a part of the IES program allowed me the freedom to study with an interdisciplinary set of scholars who could support multiple components of my training through a diverse set of experiences.

For example, in my work with Elise Cappella, Erin O’Connor, and Sandee McClowry, I was able to learn about the logistics of implementing a cluster-randomized trial across a broad set of schools, and conducting impact analyses to evaluate the efficacy of one social-emotional learning program called INSIGHTS. My experience working with Jim Kemple and Lori Nathanson at the Research Alliance for New York City schools showed me how to use research in a way that was responsive to the needs and goals of education policy makers. Quantitative coursework with Jennifer Hill and Sharon Weinberg helped me to apply rigorous quantitative methods to the data that were collected in schools, and to think concretely about the implications of research design for my future work. Coursework with developmental psychologists conducting policy-relevant research, such as Pamela Morris and Larry Aber, helped me to apply comprehensive theoretical framing when examining research questions of interest, and interpreting results. In addition, I have always been primarily interested in conducting interdisciplinary research that is responsive to policy and practice. My dissertation research grew out of my interest in learning about interdisciplinary methods for causal inference and applying them to research questions I had about how, for whom, and under what circumstances the social-emotional learning program I helped to evaluate effected outcomes for low-income students.

Most importantly, perhaps, having been part of the IES program’s collaborative and interdisciplinary community helped me to identify the type of research I wanted to do after finishing graduate school. Primarily, I knew that I wanted to conduct policy-relevant research, using the most rigorous quantitative methods available, with a team of researchers coming from different backgrounds. This realization led me to work at MDRC, where I have been working with JoAnn Hsueh and other colleagues to apply my skills from the predoctoral training program in new research design work that is responsive to critical policy questions in early education policy and practice right now. I feel prepared for this new work given the opportunities that the IES program afforded me across the last five years. 

Eric Taylor

I would emphasize two benefits. First, the IES program at Stanford helped me create and strengthen professional relationships with other education researchers and practitioners. Those relationships provided important opportunities to learn skills in ways that could not happen in the classroom but also complemented the excellent classroom instruction. The new relationships were diverse: other graduate students in different disciplines, Stanford faculty and faculty at other institutions, and, critically, practitioners and policy makers. For example, supported by my fellowship, I joined faculty at (the University of) Michigan and Columbia (University) working with the DC Public Schools to improve teacher applicant screening and hiring.

Second, those relationships combined with the financial support of the fellowship made it possible to work on new and timely research projects. During my time as an IES predoc, with collaborators at Brown, we started a researcher-practitioner partnership with colleges at the Tennessee Department of Education. The resulting work has taught me much about the day-to-day realities of school policy making and management, and how research can and cannot help. The partnership with Tennessee also grew into a five-year grant from IES, which began last year, to study state policy and teacher development through evaluation.

In short, I am certain my career is much further along today than it would have been without the IES predoc fellowship.

By Katina Stapleton, Education Research Analyst, National Center for Education Research 

 

Back to School by the Numbers

By Dana Tofig, Communications Director, Institute of Education Sciences

Across the country, hallways and classrooms are full of activity as students head back to school for the 2016–17 academic year. Each year, the National Center for Education Statistics (NCES) compiles some back-to-school facts and figures that give a snapshot of our schools and colleges for the coming year. You can see the full report on the NCES website, but here are a few “by-the-number” highlights. You can also click on the hyperlinks throughout the blog to see additional data on these topics.

The staff of NCES and the Institute of Education Sciences hopes our students, teachers, administrators and families have an outstanding school year!

 

50.4 million

The number of students expected to attend public elementary and secondary schools this year—slightly more than the 2015–16 school year. The racial and ethnic profile of these students will continue to shift, with 24.6 million White students, 7.8 million Black students, 13.3 million Hispanic students, 2.7 million Asian/Pacific Islanders students, 0.5 million  American Indian/Alaska Native students, and 1.5 million students who are two or more races. About 5.2 million students are expected to attend private schools.

 

16.1

The expected number of public school students per teacher in fall 2016. This ratio hasn’t changed much since 2000, when it was 16.0. However, the pupil/teacher ratio is lower in private schools—12.1—and has fallen since 2000, when it was 14.5. 

 

$11,600

This is the projected per-student expenditure in public elementary and secondary schools in 2016–17. Adjusting for inflation, per student expenditures are expected to rise about 1.5 percent over last school year.

 

3.5 million

The number of students expected to graduate from high school this academic year—nearly 3.2 million from public school and more than 310,000 from private schools.

 

20.5 million

This is the number of students expected to attend American colleges and universities this fall—an increase of 5.2 million since fall 2000. About 11.7 million of these students will be female, compared with 8.8 million males. About 13.3 million will attend four-year institutions and 7.2 million will attend two-year institutions.

 

14.5% and 16.5%

These percentages represent college students who were Black and Hispanic, respectively, in 2014. From 2000 to 2014, the percent of college students who were Black rose 2.8 percentage points (from 11.7 percent to 14.5 percent) and the percent of college students who were Hispanic rose 6.6 percentage points (from 9.9 percent to 16.5 percent).

 

$16,188 and $41,970

These are the average annual prices for undergraduate tuition, fees, room, and board at public and private non-profit institutions, respectively, for the 2014–15 academic year. The average annual price at private, for-profit institutions was $23,372. 

 

How ERIC is Helping Students

By Erin Pollard, ERIC Project Officer, NCEE

Over the past several years, the staff of ERIC has worked hard to get to know its users—we want to understand who uses ERIC and how they are using it. We found that the most common users are college undergraduates using the ERIC electronic library to write research papers as part of their school work. We also learned that they often weren’t searching for the best resources to meet their needs.

For example, some users told the ERIC help desk they were writing a paper for an introductory “contemporary issues in education” course, but they were requesting documents from over 20 years ago. Also, some of the documents they wanted were not peer-reviewed or particularly relevant to their topic. It became clear that we could do a better job addressing our most frequent users’ needs.

First, we redesigned the ERIC website to be more friendly and intuitive for those who are not familiar with traditional search methods, making it more likely they would get the best results (learn more about searching ERIC). Second, we worked to develop tools to help novice users, including a new series of how-to videos.

The first video in this new series is aimed at students—Using ERIC to Write a Research Paper. The ERIC team worked with undergraduate faculty members to outline a step-by-step approach to describe how a student should begin a search process, refine their results, and use different fields to find the resources they need. The video describes how to use ERIC to find full-text resources and how users can know that they found right resources.

We asked school of education faculty and librarians for their thoughts on this video and received a lot of positive feedback. Some professors said that they would link to this video on their syllabi and post it on their course management systems, while librarians have posted it on their content management system, called LibGuides. Take a look and see what you think.  And if you have feedback on this or any other ERIC videos, or suggestions for new videos and tools, let us know through the ERIC Help Desk.

 

Why We Need Large-Scale Evaluations in Education

By Elizabeth Warner, Team Leader for Teacher Evaluations, NCEE

These days, people want answers to their questions faster than ever, and education research and evaluation is no exception. In an ideal world, evaluating the impact of a program or policy would be done quickly and at a minimal expense. There is increasing interest in quicker-turnaround, low-cost studies – and IES offers grants specifically for these types of evaluations.

But in education research, quicker isn’t always better. It depends, in part, on the nature of the program you want to study and what you want to learn.

Consider the case of complex, multi-faceted education programs.  By their very nature, complex programs may require several years for all of the pieces to be fully implemented. Some programs also may take time to influence behavior and desired outcomes.  A careful and thorough assessment of these programs may require an evaluation that draws on a large amount of data, often from multiple sources over an extended period of time. Though such studies can require substantial resources, they are important for understanding whether and why an investment had an intended effect. This is especially true for Federal programs that involved millions of dollars of taxpayer money.

On August 24, IES plans to release a new report from a large-scale, multi-year study of a complex Federal initiative: the Teacher Incentive Fund (TIF) [1] This report – the third from this evaluation -- will provide estimated program impacts on student achievement after three years of implementation. The Impact Evaluation of the Teacher Incentive Fund is a $13.7 million study that stretches for more than six years and has reported interim findings annually since 2014. (The graphic above is from the a study snapshot of the first TIF evaluation report.) 

A “big study” of TIF was important to do, and here’s why.

Learning from the Teacher Incentive Fund

TIF is a Federal program that provides grants to districts that want to implement performance pay with the goal of improving teacher quality.  TIF grants awarded in 2010 included the requirement that performance pay be based on an educator evaluation system with multiple performance measures consistent with recent research. The grantees were also expected to use the performance measures to guide educator improvement.

The TIF evaluation provides an opportunity not only to learn about impacts on student achievement over time as the grant activities mature but also to get good answers to implementation questions such as:

  • How do districts structure the pay-for-performance bonus component of TIF?
  • Are educators even aware of the TIF pay-for-performance bonuses?
  • Do educators report having opportunities for professional development to learn about the measures and to improve their performance?
  • Do educators change their practice in ways that improve their performance measures? 
  • Are principals able to use their ability to offer pay-for-performance bonuses to hire or retain more effective teachers?  

Analyses to address these questions can suggest avenues for program improvement that a small-scale impact evaluation with limited data collection would miss. 

Studying the Initiative Over Time

With four years of data collection, the TIF evaluation is longer than most evaluations conducted by IES. But the characteristics of the TIF program made that extensive data collection important for a number of reasons.

 

First, TIF’s approach to educator compensation differs enough from the traditional pay structure that it might take time for educators to fully comprehend how it works. They likely need to experience the new performance measures to see how they might score and how that translates in terms of additional money earned. Second, educators might need to receive a performance bonus or see others receive one in order to fully believe it is possible to earn a bonus. (The chart pictured here is from the second report and compares teachers’ understanding with the actual size of the maximum bonuses that they could receive. This chart will be updated in the third report.)

That also might help them better understand what behaviors are needed to earn a bonus.  Finally, time might be needed for educators to respond to all of the intended policy levers of the program, particularly related to recruitment and hiring.

The TIF evaluation is designed to estimate an impact over the full length of the grants as well as provide rich information to improve the program. Sometimes, even after a number of years, some aspects of a program are never fully implemented. Thus, it may take time to see if whether it is even possible to implement a complex policy like TIF with fidelity.  An evaluation that only looks at initial implementation of a complex program may miss important program components that are incorporated or refined with time. Also, it may take several years to determine if the intended educator behaviors and desired outcomes of the policy are realized.

Learning how and why a policy does or does not work is central to program improvement.  For large, complex programs like TIF, this assessment is only possible with data-rich study over an extended period of time.     

 

[1] Under the newly reauthorized Elementary and Secondary Education Act, this program is now called the Teacher and School Leader Incentive Program, but for this blog, we will call it by its original name.