IES Blog

Institute of Education Sciences

Sustaining School Improvement

By Thomas Wei, Evaluation Team Leader, NCEE

NOTE: In an effort to turn around the nation’s chronically low-performing schools, the Department of Education injected more than $6 billion into the federal School Improvement Grants (SIG) program over the past several years. SIG schools received a lot of money for a short period of time—up to $6 million over three years—to implement a number of prescribed improvement practices.

What is the prognosis for low-performing schools now that many federal School Improvement Grants (SIG) are winding down? This is an important question that the National Center for Education Evaluation and Regional Assistance (NCEE) addressed through its Study of School Turnaround

The second and final report from this study was released on April 14 and describes the experiences of 12 low-performing schools as they implemented SIG from 2010 to 2013 (Read a blog post on the first report). Findings are based on analyses of teacher surveys and numerous interviews with other school stakeholders, such as district administrators, principals, assistant principals, members of the school improvement team, instructional coaches, and parents.

After three years trying a diverse array of improvement activities ranging from replacing teachers to extending learning time to installing behavioral support systems, most of the 12 schools felt they had changed in primarily positive ways (see chart below from report).

The report also found that schools with lower organizational capacity in the first year of SIG appeared to boost their capacity by the final year of SIG. At the same time, schools with higher capacity appeared generally able to maintain that capacity.

Many experts believe that organizational capacity is an important indicator of whether a low-performing school can improve (see chart below showing schools with higher organizational capacity also appeared more likely to sustain improvements). Organizational capacity is indicated by for example, how strong a leader the principal is, how consistent school policies are with school goals, how much school leaders and staff share clear goals, how much collaboration and trust there is among teachers, and how safe and orderly the school climate is.

Despite these promising results, the report found that the overall prospects for sustaining any improvements appeared to be fragile in most of these 12 schools. The report identified four major risk factors, including (1) anticipated turnover or loss of staff; (2) leadership instability; (3) lack of district support, particularly with regard to retaining principals and teachers; and (4) loss of specific interventions such as professional learning or extended day programs. Most of the case study schools had at least one of these major risk factors, and a number of schools had multiple risk factors.

It is important to note that this study cannot draw any causal conclusions and that it is based on surveys and interviews at a small number of schools that do not necessarily reflect the experiences of all low-performing schools. Still, it raises interesting questions for policymakers as they consider how best to deploy limited public resources in support of future school improvement efforts that will hopefully be long-lasting.

NCEE has a larger-scale study of SIG underway that is using rigorous methods to estimate the impact of SIG on student outcomes. The findings from the case studies report released last week may yield important contextual insights for interpreting the overall impact findings. These impact findings are due out later this year, so stay tuned.

IES Funded Researchers Receive Awards at AERA

The annual American Educational Research Association (AERA) conference is a great opportunity for thousands of researchers, practitioners, and policymakers to learn from one another and make connections that will help improve education. It is also a chance to celebrate and honor those who are doing outstanding work in the education research field. This year’s conference, held April 8-12, was no exception.

Among the AERA award winners who were honored this week are five people who have received funding from the Institute of Education Sciences (IES)—Michelene Chi, Douglas and Lynn Fuchs, Andrew Porter, and Daniel Schwartz.

Douglas and Lynn Fuchs received the 2014 “Distinguished Contributions to Research in Education Award,” which AERA describes as the premier acknowledgment of outstanding achievement and success in education research. The winner of this award gives a presentation at the annual conference. The Fuchs’ (pictured right) gave their presentation this year, which was entitled “The Changing Counterfactual in Schools and Classrooms: Implications for Educational Research. They are currently leading an IES research initiative, Improving Reading and Mathematics Outcomes for Students with Learning Disabilities: Next Generation Intensive Interventions.

The 2015 Distinguished Contributions winner, Andrew Porter (pictured left), also gave his address this week, entitled Standards-Based Reforms: Its Implementation and Effects. His most recent IES funding is to stand up a new Research and Development Center, the Center on Standards, Alignment, Instruction and Learning (C-SAIL).

In a recent post on the Inside IES Research blog, Dr. Porter discussed the work of C-SAIL , which seeks to deepen the understanding of the impact that college- and career-readiness standards are having on student outcomes. 

The 2016 winner of the Distinguished Contribution to Research in Education Award is Michelene Chi, who is also an IES-funded principal investigator (PI). She will give her award address at next spring’s AERA conference in San Antonio.

Through an IES-funded grant, Dr. Chi (pictured right) is actively seeking to bring principles of learning from cognitive science into the hands of teachers so that their instruction can transform student learning.

Daniel Schwartz, who has spent his career bringing principles of learning from cognitive science into the classroom, received the 2015 Sylvia Scribner Award, which honors current research that represents a significant advancement in our understanding of learning and instruction. Dr. Schwartz (pictured left) delivered his award address at this week’s AERA meeting. His current IES project is seeking to create a set of principles to select problem sets for students in science, technology, engineering, and math (STEM) domains.

You can learn more about the AERA awards on their website. Congratulations to all the winners! 

Photo Credits: Douglas and Lynn Fuchs, Vanderbilt University; Andrew Porter, University of Pennsylvania; Michelene Chi, Arizona State University; Daniel Schwartz, Stanford University

By Elizabeth Albro, Associate Commissioner, NCER

How to Help Low-performing Schools Improve

By Thomas Wei, Evaluation Team Leader

NOTE: Since 2009, the Department of Education has invested more than $6 billion in School Improvement Grants (SIG)SIG provided funds to the nation’s persistently lowest-achieving schools to implement one of four improvement models. Each model prescribed a set of practices, for example: replacing the principal, replacing at least 50 percent of teachers, increasing learning time, instituting data-driven instruction, and using “value-added” teacher evaluations.

Other than outcomes, how similar are our nation’s low-performing schools? The answers to this question could have important implications for how best to improve these, and other, schools. If schools share similar contexts, it may be more sensible to prescribe similar improvement practices than if they have very different contexts.

This is one of the central questions the National Center for Education Evaluation and Regional Assistance is exploring through its Study of School Turnaround. The first report (released in May 2014) described the experiences of 25 case study schools in 2010-2011, which was their first year implementing federal School Improvement Grants (SIG).

The report found that even though the 25 SIG schools all struggled with a history of low performance, they were actually quite different in their community and fiscal contexts, their reform histories, and the root causes of their performance problems. Some schools were situated in what the study termed “traumatic” contexts, with high crime, incarceration, abuse, and severe urban poverty. Other schools were situated in comparatively “benign” contexts with high poverty but limited crime, homes in good repair, and little family instability. All schools reported facing challenges with funding and resources, but some felt it was a major barrier to improvement while others felt it was merely a nuisance. Some schools felt their problems were driven by student behavior, others by poor instruction or teacher quality, and still others by the school’s external context such as crime or poverty.

Given how diverse low-performing schools appear to be, it is worth wondering whether they need an equally diverse slate of strategies to improve. Indeed, the report found that the 25 case study schools varied in their improvement actions even with the prescriptive nature of the SIG models (see the chart above, showing school improvement actions used by sample schools).

It is important to note that this study cannot draw any causal conclusions and that it is based on a small number of schools that do not necessarily reflect the experiences of all low-performing schools. Still, policymakers may wish to keep this finding in mind as they consider how to structure future school improvement efforts.

The first report also found that all but one of the 25 case study schools felt they made improvements in at least some areas after the first year of implementing SIG. Among the issues studied in the second report, released April 14, 2016, is whether these schools were able to build on their improvements in the second and third year of the grant. Read a blog post on the second report.

UPDATED APRIL 18 to reflect release of second report.

Responding to the Needs of the Field

By Chris Boccanfuso Education Research Analyst, NCEE

One of the most commonly asked questions about the Regional Educational Laboratory, or REL, program is how we choose the applied research and evaluation studies, analytic technical assistance, and dissemination activities we provide for free to stakeholders every year. The answer is simple – we don’t!

Instead, the REL staff at the Institute of Education Sciences (IES) and our contractors who run the nation’s ten regional labs listen to the voices of teachers, administrators, policymakers, and students to identify and address high-leverage problems of practice and build the capacity of stakeholders. In other words, these groups determine where the RELs should use their resources to make the largest, most lasting impact possible on practice, policy and, ultimately, student achievement.

How do the RELs do this? Through a variety of activities we collectively refer to as needs sensing. The following are a few examples of how the RELs engage in needs sensing:

Research Alliances: Research Alliances are a type of researcher–practitioner partnership where a group of education stakeholders convene around a specific topic of concern to work collaboratively to investigate a problem and build capacity to address it. Alliances can be made up of many types of stakeholders, such as teachers, administrators, researchers and members of community organizations. Alliances can vary in size and scope and address a variety of topics. For instance, alliances have formed to address topics as broad as dropout prevention and as specific as Hispanic students’ STEM performance. The vast majority of the RELs’ work is driven by these research alliances.

While the RELs’ 79 research alliances are incredibly diverse, one thing each alliance has in common is that they collectively develop a research agenda. These agendas can change, as the alliance continually weighs the questions and needs of various groups against the types of services and the resources available to address these needs. Not every need has to be addressed through a multi-year research study. Sometimes, it can be addressed through a workshop, a literature review, or a “Bridge Event”, where national experts on certain topic work with practitioners to provide the information that alliance members need, when they need it. Sometimes, a need is state or district-specific, is related to the impact of a specific program, or covers a topic where the existing research literature is thin. In these cases, a research study may be most appropriate.

Governing Boards: Another way that RELs determine their work is through their Governing Boards.  By law, each REL is required to have a Governing Board that consists of the Chief State School Officers (or their designee) for each state, territory, or freely associated state in the region. The Board also includes carefully selected members who equitably represent each state, as well as a broad array of regional interests, such as educating rural and economically disadvantaged populations. (A 2013 REL Northeast and Islands Governing Board meeting is pictured here.)
 
Governing Boards typically include a mix of people with experience in research, policy and teaching practice. Each Governing Board meets two to three times per year to discuss, direct, advise, and approve each REL project that occurs in that region. The intent is to ensure that the work being done by the REL is timely, high-leverage, equitably distributed across the region, and not redundant with existing efforts.                          

“Ask a REL”: A third way in which the RELs engage in needs sensing is through the Ask a REL service. Ask a REL is a publicly available reference desk service that functions much like a technical reference library. It provides requestors with references, referrals, and brief responses in the form of citations on research-based education questions. RELs are able to examine trends in the topics of Ask a REL requests to verify needs determined through other methods, as well as identify new topics that may warrant additional attention.   

RELs use many additional ways to explore the needs of their region, including scans of regional education news sites, reviews of recently published research, and Stakeholder Feedback Surveys that are filled out by alliance members and attendees at REL events.

It’s a thorough and ongoing process that RELs are engaging in to address authentic, high-leverage problems of practice in a variety of ways. In the coming months, we will share stories of the many projects that were informed by this needs sensing process. Stay tuned!

 

The Institute of Education Sciences at AERA

The American Educational Research Association (AERA) will hold its annual meeting April 8 through April 12 in Washington, D.C.—the largest educational research gathering in the nation. This will be a special meeting for AERA, as it is celebrating 100 years of advocating for the development and use of research in education. The program includes hundreds of sessions, including opportunities to learn about cutting edge education research and opportunities to broaden and deepen the field. 

About 30 sessions will feature staff from the Institute of Education Sciences (IES) discussing IES-funded research, evaluation, and statistics, as well as training and funding opportunities.

On Saturday, April 9, at 10:35 a.m., attendees will have a chance to meet the Institute’s leadership and hear about the areas of work that IES will be focusing on in the coming year. Speakers include Ruth Curran Neild, IES’ delegated director, and the leaders of the four centers in IES: Thomas Brock, commissioner of the National Center for Education Research (NCER); Peggy Carr, acting commissioner of the National Center for Educational Statistics (NCES); Joy Lesnick, acting commissioner of the National Center for Education Evaluation and Regional Assistance (NCEE), and Joan McLaughlin, commissioner of the National Center for Special Education Research (NCSER).

On Monday, April 11, at 9:45 a.m., attendees can speak to one of several IES staffers who will be available at the Research Funding Opportunities—Meet Your Program Officers session. Program officers from NCER, NCSER, and NCEE will be on hand to answer questions about programs and grant funding opportunities. Several IES representatives will also be on hand Monday afternoon, at 4:15 p.m. for the Federally Funded Data Resources: Opportunities for Research session to discuss the myriad datasets and resources that are available to researchers.

NCES staff will lead sessions and present on a variety of topics, from The Role of School Finance in the Pursuit of Equity (Saturday, 12:25 p.m.) to Understanding Federal Education Policies and Data about English Learners (Sunday, April 10, 8:15 a.m.) and what we can learn from the results of PIAAC, a survey of adult skills (also Sunday, 8:15 a.m.). Dr. Carr will be a part of several sessions, including one on Sunday morning (10:35 a.m.) about future directions for NCES longitudinal studies and another on Monday morning (10 a.m.) entitled Issues and Challenges in the Fair and Valid Assessment of Diverse Populations in the 21st Century

On Monday, at 11:45 a.m., you can also learn about an IES-supported tool, called RCT-YES, that is designed to reduce barriers to rigorous impact studies by simplifying estimation and reporting of study results (Dr. Lesnick will be among those presenting). And a team from the IES research centers (NCER/NCSER) will present Sunday morning (10:35 a.m.) on communication strategies for disseminating education research (which includes this blog!).

IES staff will also participate in a number of other roundtables and poster sessions. For instance, on Tuesday, April 12, at 8:15 a.m., grab a cup of coffee and attend the structured poster session with the Institute’s 10 Regional Educational Laboratories (RELs). This session will focus on building partnerships to improve data use in education.  REL work will also be featured at several other AERA sessions.  

Did you know that the National Library of Education (NLE) is a component of IES? On Friday and Monday afternoon, attendees will have a unique opportunity to go on a site visit to the library. You’ll learn about the library’s current and historical resources – including its collection of more than 20,000 textbooks dating from the mid-19th century. The Library offers information, statistical, and referral services to the Department of Education and other government agencies and institutions, and to the public.

If you are going to AERA, follow us on Twitter to learn more about our sessions and our work.  And if you are tweeting during one of our sessions, please include @IESResearch in your tweet. 

By Dana Tofig, Communications Director, IES