NCEE Blog

National Center for Education Evaluation and Regional Assistance

Recommendations for Teaching Secondary Students to Write Effectively

EDITOR'S NOTE: Dr. Steve Graham was the head of a panel of experts that assisted the What Works Clearinghouse in developing recommendations for its practice guide on effective writing for secondary students. We invited Dr. Graham to author this blog about the guide and a January 18 webinar on its recommendations. 


By Steve Graham, Warner Professor in the Division of Leadership and Innovation, Arizona State University

Effective writing is a vital component of students’ literacy achievement and a life-long skill that plays a key role in postsecondary success. For more than 30 years, I’ve focused my research on how teachers can help students become strong writers, how writing develops, and how writing can be used to support reading and learning. Much progress has been made in the field of writing instruction, and summarizing and sharing these findings will help teachers implement evidence-based practices. Using effective instructional practices will help ensure our students become adept at using writing to support and extend learning, argue effectively and fairly, connect and communicate with others, tell captivating stories, and explore who they are as well as reflect on their experiences. 

Recently, the What Works Clearinghouse (WWC) released a new practice guide to address the challenges of teaching writing to secondary students. Teaching Secondary Students to Write Effectively offers three evidence-based recommendations for helping students in grades 6–12 develop effective writing skills. The first recommendation focuses on teaching students to use writing strategies to plan, think critically, and effectively convey their ideas. The second recommendation suggests integrating reading and writing to emphasize key features of text. Finally, the third recommendation describes how to use a formative assessment cycle to inform writing instruction.

The guide includes practical instructional tips and strategies for each recommendation that teachers can use to help students improve their writing. You’ll find over 30 examples to use in the classroom, including sample writing strategies and prompts and activities that incorporate writing and reading.

I’d like to invite teachers, administrators, and others to join me for a webinar on the recommendations in this practice guide, Wednesday, January 18, at 3 p.m. (ET). During the webinar, we will discuss the guide’s three recommendations and give teachers in all disciplines usable guidance on how to implement them in the classroom. We will also discuss potential challenges educators may face when implementing the recommended practices and provide advice on how to overcome those challenges.

Developing the Practice Guide

The WWC develops practice guides with the support of an expert panel. The panelists combine their expertise with the findings of rigorous research to produce specific recommendations. I was honored to chair this panel, which also included Jill Fitzgerald, from the University of North Carolina at Chapel Hill and MetaMetrics; Linda D. Friedrich, from the National Writing Project; Katie Greene, from Forsyth County (Ga.) Schools; James S. Kim, from Harvard University; and Carol Booth Olson, from the University of California, Irvine. 

For this practice guide, WWC staff conducted a systematic review of the research—a thorough literature search identified more than 3,700 relevant studies. After screening each study, 55 studies were found to use eligible research designs and examine the effec­tiveness of the practices found in this guide’s recommendations. The recommendations are based on the 15 studies that meet the WWC’s rigorous standards. For each of the recommendations, the WWC and the panel rate the strength of the evidence that supports it.  Appendix D in the guide presents a thorough summary of the evidence supporting each recommendation. 

Sharing our Recipe: Online Training in WWC Standards

By Christopher Weiss, Senior Education Research Scientist, WWC

Many individuals and organizations have special ways of doing things, specific procedures that make them unique —Coca-Cola has its formula; sports teams have their playbooks; and grandparents have their secret recipes for biscuits, barbecue, and other family favorites.

It’s the same for the What Works Clearinghouse (WWC). Our “special sauce” is in how we review effectiveness research to help determine what is working in education. But unlike Coke, coaches, and grandma, the WWC doesn’t keep it a secret.

On December 15, the Institute of Education Sciences (IES) launched a set of video training modules – the WWC Group Design Standards Online Training – to share our procedures. These modules are designed to help you learn more about the elements that go into a WWC rating and the features of a research study that WWC examines during evaluation.  The online training will help education decision-makers and researchers better understand key elements of the WWC review process. These modules describe and explain key topics and concepts of the WWC’s Group Design standards and how the WWC uses these standards to identify and evaluate high quality, rigorous research.

The series is designed to address the needs of both consumers and future producers of the WWC’s reviews of educational effectiveness research. Whether you’re a researcher who’s hoping your study will meet the WWC’s standards or someone trying to make an evidence-based decision related to education, this training series will help! And no background in research is needed –we’ve also developed an extensive set of materials to support you as you learn.

Each of the five modules follows a similar structure, including an overview of module objectives, detailed information about the topic, examples, and knowledge checks to reinforce what you’ve learned. (We've embedded the first video in the series at the end of this post, but if you are going to take the training, start it through the WWC website.)  

Each module focuses on a specific aspect of the standards.

  • Group designs – or overall research designs – and the types of research that can be reviewed using the WWC Group Design Standards;
  • Attrition, or loss of participants in a research study, and why this is important;
  • Baseline Equivalence, which assesses how similar two groups are at the beginning of a research study;
  • Confounding Factors, which are study components that make it difficult or impossible to distinguish the effect of an educational intervention from the effect of that component; and
  • Outcome Measures, or what is measured to assess the effectiveness of an intervention.

If you view all five of the training modules, you will earn a certificate of completion. Details about how to view the session and earn this certificate are available on the What Works Clearinghouse website.

The online training takes about seven hours to complete, but the modules are designed so that you can complete them at your own pace. We’ve included a feature that allows you to take a break from the training at any point – then pick it up again where you left off when you’re ready to continue.

These modules cover the same material that WWC reviewers learn through their in-person certification training – and completion of the online training course is one step toward becoming a certified reviewer in WWC Group Design Standards. Certification also requires completing WWC Procedures training and successful completion of a certification exam. We expect to be able to offer online versions of the WWC Procedures training and the certification exam later in 2017.

We hope this online training brings more transparency and understanding to the WWC review process. Then we can work on that secret biscuit recipe. 

 

Bringing Evidence-based Practices to the Field

By Dr. Barbara Foorman, Director Emeritus, Florida Center for Reading Research, Florida State University

The Institute of Education Sciences recently released a What Works Clearinghouse (WWC) educator’s practice guide that has four recommendations to support the development of foundational reading skills that are critically important to every student’s success. The recommendations in Foundational Skills to Support Reading for Understanding in Kindergarten Through 3rd Grade are based on a comprehensive review of 15 years of research on reading, and guidance from a national panel of reading experts, of which I was the chair.

Recently, the Regional Educational Laboratory (REL) Southeast at Florida State University has developed a set of professional learning community (PLC) materials and videos to help teachers and other practitioners implement the guide’s recommendations in classrooms.

Over the past few months, REL Southeast has shared the practice guide and PLC materials with practitioners and policymakers in two states – North Carolina and Mississippi, which both have K-3 reading initiatives and reading coaches who assist with implementation. I’m excited by the feedback we are getting.

During these presentations, we shared the format of the ten 75-min PLC sessions and accompanying videos that demonstrate the recommendations and action steps in actual classrooms. We filmed the videos in partnership with Dr. Lynda Hayes, Director of the PK Yonge Developmental Research School at the University of Florida, and her primary grade teachers.

In North Carolina, we trained K–3 regional literacy consultants, elementary teachers and reading coaches, and higher education faculty on the PLC Facilitator’s Guide in Charlotte and Raleigh. The K-3 regional literacy consultants are organized by the North Carolina Department of Public Instruction.

In Mississippi, we trained the 90 Mississippi Department of Education reading coaches and district-supported special education specialists in Jackson. In turn, the state coaches will train the K–3 teachers who are a part of the reading initiative in the practice guide recommendations and action steps. Additionally, the coaches will work with the primary grade teachers in each of their assigned schools to implement the PLC. Having the state coaches oversee the implementation of the PLC will help ensure commitment and instill depth to the PLC sessions.

Also present at the training in Mississippi were faculty members from the University of Mississippi and Belhaven University. I accepted an invitation from the Mississippi Institutions of Higher Learning Literacy Council to speak to higher education faculty about the guide and PLC materials. The invitation is timely because Mississippi recently completed a study of teacher preparation for early literacy instruction.

I hope you will download the practice guide and PLC materials. If you have any thoughts, comments, or questions, please email Contact.IES@ed.gov. You can learn more about the work of the Regional Educational Laboratories program and REL Southeast on the IES website.  

Dr. Foorman is the Director of REL Southeast, located at Florida State University

The What Works Clearinghouse Goes to College

By Vanessa Anderson, Research Scientist, NCEE

The What Works Clearinghouse (WWC) was founded in 2002 and, in its first decade, focused mainly on reviewing studies of programs, policies, products and practices—or interventions—for improving student outcomes in pre-K, elementary and secondary schools. But in 2012, the WWC broadened its focus and has been using rigorous standards to review studies of interventions designed to increase the success of students in postsecondary education.

This week, the WWC launches a new topic—Supporting Postsecondary Success—and it is a good time to look at the work we’re doing, and will do, in the postsecondary area. 

The WWC postsecondary topic area includes reviews of studies on a wide range of interventions, including learning communities, summer bridge programs, multi-faceted support programs, academic mentoring, and interventions that aim to reduce performance anxiety. As of today, 294 postsecondary studies have been reviewed by the WWC. Those reviews are summarized in six Intervention Reports, 25 Single Study Reviews, and four Quick Reviews. And there’s much more in the works!  For instance, a WWC Educator’s Practice Guide that includes strategies for supporting students in developmental education is planned for publication later this year. (Learn more about Practice Guides)

Identifying Studies for Review

In the postsecondary topic area, there are currently three main ways that studies are identified by the WWC for review.

The first is studies that are reviewed for WWC Intervention Reports. All WWC Intervention Reports use a systematic review process to summarize evidence from all available studies on a given intervention. The WWC conducts a broad search for all publicly available studies of interventions that are related to the topic. This process often identifies hundreds of studies for review. The effectiveness studies are then reviewed against WWC standards. Only the highest quality studies are summarized in an Intervention Report.

We released two new intervention reports this week as part of our new Supporting Postsecondary Success topic. You can view the new Intervention Reports on Summer Bridge programs and first-year experience courses on the WWC website.

The second way that studies are reviewed by the WWC is through Quick Reviews, which are performed on studies that have received a great deal of media attention. In these reports, the WWC provides a brief description of the study, the author-reported results, and a study rating. We like to think of Quick Reviews as a way to help people decide whether to fully believe the results of a study, based on the research design and how the study was conducted. For example, we released a quick review earlier this month that focused on a study of computer usage and student outcomes for a class at the U.S. Military Academy at West Point.

Finally, the WWC reviews postsecondary studies submitted as supporting evidence for discretionary grant competitions funded by the U.S. Department of Education, such as the Strengthening Institutions Program, First in the World and TRIO Student Support Services. These grant competitions require applicants to submit studies as evidence of the effectiveness of the interventions they propose to implement. The WWC reviews these studies and includes the results of those reviews in our database.

If you want to see all the studies on postsecondary interventions that have been reviewed by WWC you can check out—and download—the Reviewed Studies Database. In the “Topic Areas” dropdown menu, just select “Postsecondary,” and then easily customize the search by rating, publication type, and/or reasons for the review (such as a grant competition).  

For more information, visit the WWC postsecondary topic area on the website. To stay up-to-date on WWC news, information, and products, follow us on Facebook, Twitter and sign up for the WWC newsflash!

Five Reasons to Visit the What Works Clearinghouse

By Diana McCallum, Education Research Analyst, What Works Clearinghouse

It’s been more than a decade since the first What Works Clearinghouse reports were released and we have a wealth of information and resources that can help educators and leaders make evidence-based decisions about teaching and learning. Since 2005, the WWC has assessed more than 11,500 education studies using rigorous standards and has published hundreds of resources and guides across many content areas. (View the full version of the graphic to the right.) 

The WWC website has already received more than 1.7 million page views this year, but if you haven’t visited whatworks.ed.gov lately, here are five reasons you might want to click over:

1) We are always adding new and updated reviews. Multiple claims about programs that work can be overwhelming and people often lack time to sift through piles of research. That’s where the WWC comes in. We provide an independent, objective assessment of education research. For example, we have intervention reports that provide summaries of all of the existing research on a given program or practice that educators can use to help inform their choices.  In addition, when a new education study grabs headlines, the WWC develops a quick review that provides our take on the evidence presented to let you know whether the study is credible. In 2015, we added 43 publications to WWC and we’re adding more every month this year.

2) We’ve expanded our reach into the Postsecondary area. In late 2012, the WWC expanded its focus to include reviews of studies within the Postsecondary area to capture the emerging research on studies on a range of topics, from the transition to college to those that focus on postsecondary success.  To date, the WWC has reviewed over 200 studies on postsecondary programs and interventions, and this area continues grow rapidly. In fact, several Office of Postsecondary Education grant competitions add competitive priority preference points for applicants that submit studies that meet WWC standards. (Keep an eye out for a blog post on the postsecondary topic coming soon!)

3) You can find what works using our online tool. Wondering how to get started with so many resources at your fingertips? Find What Works lets you do a quick comparison of interventions for different subjects, grades, and student populations. Want to know more about a specific intervention? We’ve produced more than 400 intervention reports to provide you the evidence about a curriculum, program, software product, or other intervention for your classroom before you choose it.  Recently, we’ve added a feature that allows a user to search for interventions that have worked for different populations of students and in different geographic locations. As we mentioned in a recent blog post, the Find What Works tool is undergoing an even bigger transformation this September, so keep visiting!

4) We identify evidence-based practices to use in the classroom. The WWC has produced 19 practice guides that feature practical recommendations and instructional tips to help educators address common challenges. Practice guides (now available for download as ebooks) provide quick, actionable guidance for educators that are supported by evidence and expert knowledge within key areas.  Some of our guides now feature accompanying videos and brief summaries that demonstrate recommended practices and highlight the meaning behind the levels of evidence. The work of practice guides are also actively disseminated during Regional Educational Laboratory (REL) Bridge events. For instance, REL Southwest held a webinar on Teaching Math to Young Children, which was based on a WWC practice guide. For more information, read a previously published blog post on practice guides.

5) We compile information by topic. Our “Special Features” pages focus on common themes in education, such as tips for college readiness, information for heading back to school, and guidance for what works in early childhood education. These Special Features provide a starting point to access a variety of WWC resources related to a topic.

In the coming months, we’ll post other blogs that will explore different parts of the WWC and tell you about ongoing improvements. So keep visiting the What Works website or signup to receive emails when we release new reports or resources. You can also follow us on Facebook and Twitter.

The What Works Clearinghouse is a part of the National Center for Education Evaluation and Regional Assistance in the Institute of Education Sciences (IES), the independent research, evaluation, and statistics arm of the U.S. Department of Education. You can learn more about IES’ other work on its website or follow IES on Twitter and Facebook

 

Should ESSA Evidence Definitions and What Works Study Ratings be the Same? No, and Here's Why!

By Joy Lesnick, Acting Commissioner, NCEE

The Every Student Succeeds Act (ESSA), the new federal education law, requires education leaders to take research evidence into account when choosing interventions or approaches. ESSA  defines three “tiers” of evidence—strong, moderate, and promising—based on the type and quality of study that was done and its findings.  

Are the ESSA definitions the same as those of Institute of Education Sciences’ What Works Clearinghouse (WWC)?  Not exactly.  ESSA definitions and WWC standards are more like cousins than twins.

Like ESSA, the WWC has three ratings for individual studies – meets standards without reservations, meets standards with reservations, and does not meet standards. The WWC uses a second set of terms to summarize the results of all studies conducted on a particular intervention. The distinction between one study and many studies is important, as I will explain below.

You may be wondering: now that ESSA is the law of the land, should the WWC revise its standards and ratings to reflect the tiers and terminology described in ESSA?  Wouldn’t the benefit of making things nice and tidy between the two sets of definitions outweigh any drawbacks?

The short answer is no.

The most basic reason is that the WWC’s standards come from a decision-making process that is based in science and vetted through scholarly peer review, all protected by the independent, non-partisan status of the Institute of Education Sciences (IES). This fact is central to the credibility of the WWC’s work.  We like to think of the WWC standards as an anchor representing the best knowledge in the field for determining whether a study has been designed and executed well, and how much confidence we should have in its findings.

WWC Standards Reflect the Most Current Scientific Knowledge – and are Always Evolving

WWC standards were developed by a national panel of research experts. After nearly two years of meetings, these experts came to a consensus about what a research study must demonstrate to give us confidence that an intervention caused the observed changes in student outcomes.

Since the first WWC standards were developed over a decade ago, there have been many methodological and conceptual advances in education research. The good news is that the WWC is designed to keep up with these changes in science. As science has evolved, the WWC standards have evolved, too.

One example is the WWC’s standards for reviewing regression discontinuity (RD) design studies.  The first version of RD standards was developed by a panel of experts in 2012.  Since then, the science about RD studies has made so much progress that the WWC recently convened another panel of experts to update the RD standards. The new RD standards are now on the WWC website to solicit scholarly comment.  

When it Comes to Evidence, More is Better

The evidence tiers in ESSA set a minimum bar, based on one study, to encourage states, districts, and schools to incorporate evidence in their decision making. This is a very important step in the right direction.  But a one-study minimum bar is not as comprehensive as the WWC’s approach.

In science, the collective body of knowledge on a topic is always better than the result of a single study or observation. This is why the primary function of the WWC is to conduct systematic reviews of all of the studies on a program, policy, practice, or approach (the results of which are published in Intervention Reports like the one pictured here).

The results of individual studies are important clues toward learning what works. But multiple studies, in different contexts, with different groups of teachers and students, in different states, and with different real-world implementation challenges tell us much more about how well a program, policy, practice or approach works. And that, really, is what we’re trying to find out.

An Improved WWC Search Tool and Ongoing Support for States and Districts

One area where WWC will make changes is in how users find studies that have certain characteristics described in ESSA’s evidence tiers.  For the past 16 months, the WWC team has been hard at work behind the scenes to develop, code, and user-test a dramatically improved Find What Works tool.  We expect to release this tool, along with other changes to the WWC website, in fall 2016. (More on that in another post, but the picture below offers a sneak preview!)

These changes should further increase the utility of the WWC website, which already gets more than 300,000 hits each month and offers products that are downloaded hundreds of thousands of times each year.

We know that providing information on a website about evidence from rigorous research is just a first step.  States and districts may need additional, customized support to incorporate evidence into their decision-making processes in ways that are much deeper than a cursory check-box approach.

To meet that need, other IES programs are ready to help. For example, IES supports 10 Regional Educational Laboratories (RELs) that provide states and districts with technical support for using, interpreting, and applying research. At least two researchers at every REL are certified as WWC reviewers (meaning they have in-depth knowledge of the WWC standards and how the standards are applied), and every REL has existing relationships with states and districts across the nation and outlying regions. Because the RELs are charged with meeting the needs of their regions, every chief state school officer (or designee) sits on a REL Governing Board, which determines the annual priorities of the REL in that area.

As states prioritize their needs and identify ways to incorporate evidence in their decisions according to the new law, the WWC database of reviewed studies will provide the information they need, and the RELs will be ready to help them use that information in meaningful ways.

 

 

 

Practice Guides: How to Use What Works in the Classroom

By Diana McCallum, NCEE

With new education research released every day, it can be difficult to know which teaching methods and classroom practices have been tested and shown to improve student outcomes. You want to know what really works and how to use evidence-based practices in your school or classroom.

What Works Clearinghouse practice guides help bridge the gap between research and practice by examining the findings from existing research studies and combining them with expert advice about applying these findings in the classroom. For each guide, a team of nationally-recognized practitioners and researchers work closely with the WWC to combine evidence from research with guidance from hands-on experience.

Practice guides offer specific recommendations that include a description of the supporting research, steps for carrying out the recommendation, and strategies you can use to overcome potential challenges. Many of the guides also feature supplementary materials, like videos and summaries, to help you quickly find what you need.

One example is our most recent practice guide, Teaching Strategies for Improving Algebra Knowledge in Middle and High School Students. Mastering algebra helps students move from arithmetic operations to understanding abstract concepts, and is for a key to success in future mathematics courses, including geometry and calculus. Teaching Strategies for Improving Algebra Knowledge in Middle and High School Students presents three evidence-based recommendations educators can use to help students develop a deeper understanding of algebra, promote process-oriented thinking, and encourage precise communication. These recommendations help address common challenges in algebra instruction and focus on:

  • Utilizing the structure of algebraic representations to make connections among problems, solution strategies, and representations; 
  • Incorporating solved problems into classroom instruction and activities to help students develop their algebraic reasoning skills; and
  • Comparing and selecting from alternative algebraic strategies to give students flexibility when solving problems. 

You can read the Practice Guide Summary for a quick overview of these recommendations or spend a few minutes watching videos in which Jon Star, of Harvard University’s Graduate School of Education, explain the recommendations.  

The Teaching Strategies for Improving Algebra Knowledge in Middle and High School Students is just one of 19 practice guides available on the What Works Clearinghouse website. Some of the others are:

  • Teaching Math to Young Children: Preschool and kindergarten teachers can get details on how to improve math lessons with this guide, including strategies to create a math-rich environment. You’ll find examples of classroom activities and games that can supplement lesson plans and provide opportunities for children to learn math.

You can find information and links to all 19 practice guides on our website. We also cover a variety of other math and literacy topics, as well as guides focused on dropout prevention, using data to monitor student progress and make decisions, and preparing students for college.

Visit whatworks.ed.gov to find the practice guide that’s right for you or to suggest a topic you’d like us to explore.

Dr. McCallum is an education research analyst on the What Works Clearinghouse team.

About the What Works Clearinghouse (WWC)

For more than a decade, the goal of the WWC has been to provide educators with the information they need to make evidence-based decisions with the aim of improving student outcomes. Established by the U.S. Department of Education’s Institute of Education Sciences, the WWC strives to be a central and trusted source of scientific evidence on education programs, products, practices, and policies. Follow us on Twitter and Facebook.