IES Blog

Institute of Education Sciences

Low-Cost, Short-Duration Evaluations: Helping States and School Districts Make Evidence-based Decisions

Last year, IES launched two new grant programs to help state and district education agencies obtain rigorous and timely evidence they can use to inform decisions about education interventions of high importance to the agency: Low-Cost, Short-Duration Evaluation of Education Interventions (CFDA 84.305L) and Low-Cost, Short-Duration Evaluation of Special Education Interventions (CFDA 84.324L). These programs are intended to support relatively quick (completed within 2 years) and inexpensive ($250,000 or less) evaluations to test the effects of education interventions that schools and other education institutions may want to scale up or redesign to improve student education outcomes.

IES developed the Low-Cost, Short Duration Evaluation grant programs to help states and school districts plan and launch rigorous evaluations when they roll out new interventions. Such interest is likely to intensify under the Every Student Succeeds Act, which requires states to set accountability goals and encourages use of evidence-based programs to close achievement gaps and improve low-performing schools. 

Advances in technology and administrative data systems have also led some policymakers and researchers to suggest that rigorous evaluations ought to be less expensive to perform today than in the past, when evaluators had to do much of their own data collection.

To meet the goals of relevance, rigor, and affordability, the Low-Cost, Short-Duration Evaluation grant programs set out specific requirements and expectations for applicants:

  • To ensure the relevance, the evaluations must be focused on an intervention that the education agency identifies as a priority and will be implemented in Year 1 of the grant, and must be carried out as a partnership between a research institution and a state and local education agency. 
  • To ensure rigor, evaluation designs must either be randomized controlled trials or regression discontinuity designs that have the potential to meet the What Works Clearinghouse evidence standards without reservation. Beginning in FY 2017, single-case designs may also be used in evaluations of special education interventions. 
  • To ensure lower-cost, the evaluations are expected to rely mainly on agency administrative datasets or other secondary sources rather than on new data collection. 

The short-term nature of the grants leads to a focus on interventions that are expected to improve student education outcomes within a summer session, quarter, semester or academic year. The low cost also means that researchers are not expected to conduct in-depth research into program implementation, comparison group practices, and costs and benefits of interventions unless the data exist in the administrative datasets. Other IES funding opportunities – including Education Research Grants, Special Education Research Grants, and Evaluation of State and Local Education Policies and Programs – are available to support longer-term and/or more in-depth evaluations. 

In sum, the Low-Cost, Short-Duration Evaluation grant programs fill a special niche within the IES portfolio by providing results on specific types of interventions that education agencies can use in making important decisions on programs, policies, and spending. They can also lay the groundwork for future grant-funded research to investigate implementation, underlying mechanisms, and longer-term impacts.

The results from last year’s competitions will be released soon and both programs are being re-competed this year with an application deadline of August 4, 2016.  On June 9, 2016, 2:00-3:30 p.m. (EDT), IES will host a webinar on the two grant competitions. Please register in advance. An archive of the webinar will also be available on the IES funding webinar page.

Written by Allen Ruby, Associate Commissioner, National Center for Education Research

NCSER Researchers Receive Awards from the Council for Exceptional Children

In April, the annual Council for Exceptional Children (CEC) held its annual Convention and Expo, during which scholars were recognized for their research contributions to the field. A number of NCSER-funded investigators were among those honored by the CEC.

Michael Wehmeyer (right) received the CEC Special Education Research Award, which recognizes an individual or team whose research has made significant contributions to the education of children and youth with exceptionalities. Dr. Wehmeyer has served as the principal investigator (PI) on two NCSER-funded awards.  He tested the promise of efficacy for the Self-Determined Learning Model of Instruction, an intervention to promote self-determination, access to the general education curriculum, and support for the needs of students with disabilities to achieve academic and transition-related goals.  He and his research team are currently developing an assessment, the Self-Determination Inventory System, to measure self-determination for a range of disability groups as well as youth and young adults without disabilities.

Rob Horner (right) is the 2016 recipient of the Kauffman-Hallahan Distinguished Researcher Award. This honor, awarded by the CEC Division of Research, recognizes individuals or research teams who have made outstanding scientific contributions in basic or applied research in special education over the course of their careers. Dr. Horner has been the recipient of several IES-funded awards. He is currently testing the efficacy of Team-Initiated Problem Solving, a training and coaching intervention for teaching school teams to use behavioral and academic progress-monitoring data to define and solve problems. For a number of years Dr. Horner has also been a lead faculty member of the IES Summer Research Training Institute for Single-Case Intervention Research Design and Analysis, a professional development program that has increased the national capacity of education researchers to conduct methodologically rigorous single-case intervention studies.

Brian Boyd (left) is the winner of the 2016 Distinguished Early Career Research Award, an honor from CEC’s Division of Research that recognizes individuals with outstanding scientific contributions in special education research within the first 10 years after receiving a doctoral degree. Dr. Boyd has played key roles on several NCSER-funded grants, including as co-PI on an award to develop Advancing Social-Communication and Play (ASAP), an intervention aimed at improving the social-communication and play skills of preschoolers with Autism Spectrum Disorders. He is also the PI for a current grant examining the efficacy of ASAP.

All of this year’s CEC Division of Research award winners will give an invited presentation at the 2017 CEC Convention and Expo.  At this year’s convention, held April 13-16, the Division of Research award recipients from 2015 gave presentations. This includes two previous winners of the Distinguished Early Career Research Award – Karrie Shogren and Kent McIntosh – both of whom have been NCSER-funded investigators.

Karrie Shogren (right) had received an award from NCSER in which she analyzed data from the National Longitudinal Transition Study-2 to identify individual and environmental factors that predict self-determination and examine the relationship between self-determination and long-term outcomes of students with disabilities. As part of a current NCSER-funded project, Kent McIntosh (left) is conducting an exploratory study to identify malleable factors that enhance or inhibit the implementation and sustainability of school-wide social-emotional and behavior support practices, focusing on School-wide Positive Behavioral Interventions and Supports.

The Division of Research was not the only CEC group awarding honors to NCSER-funded investigators. Maureen Conroy (right) was awarded the 2016 Outstanding Leadership Award from CEC’s Council for Children with Behavioral Disorders. This award recognizes an individual who has made significant contributions to research, leadership, teacher education, and policy in the field of behavior disorders. Dr. Conroy has received funding to develop a preventative classroom-based intervention that targets problem behaviors by preschool teachers and to evaluate the efficacy of BEST in CLASS, an intervention for young children at high risk for emotional and behavioral disorders.

IES congratulates all the winners!

Written by Amy Sussman, program officer, NCSER

IES Funded Researchers Receive Awards at AERA

The annual American Educational Research Association (AERA) conference is a great opportunity for thousands of researchers, practitioners, and policymakers to learn from one another and make connections that will help improve education. It is also a chance to celebrate and honor those who are doing outstanding work in the education research field. This year’s conference, held April 8-12, was no exception.

Among the AERA award winners who were honored this week are five people who have received funding from the Institute of Education Sciences (IES)—Michelene Chi, Douglas and Lynn Fuchs, Andrew Porter, and Daniel Schwartz.

Douglas and Lynn Fuchs received the 2014 “Distinguished Contributions to Research in Education Award,” which AERA describes as the premier acknowledgment of outstanding achievement and success in education research. The winner of this award gives a presentation at the annual conference. The Fuchs’ (pictured right) gave their presentation this year, which was entitled “The Changing Counterfactual in Schools and Classrooms: Implications for Educational Research. They are currently leading an IES research initiative, Improving Reading and Mathematics Outcomes for Students with Learning Disabilities: Next Generation Intensive Interventions.

The 2015 Distinguished Contributions winner, Andrew Porter (pictured left), also gave his address this week, entitled Standards-Based Reforms: Its Implementation and Effects. His most recent IES funding is to stand up a new Research and Development Center, the Center on Standards, Alignment, Instruction and Learning (C-SAIL).

In a recent post on the Inside IES Research blog, Dr. Porter discussed the work of C-SAIL , which seeks to deepen the understanding of the impact that college- and career-readiness standards are having on student outcomes. 

The 2016 winner of the Distinguished Contribution to Research in Education Award is Michelene Chi, who is also an IES-funded principal investigator (PI). She will give her award address at next spring’s AERA conference in San Antonio.

Through an IES-funded grant, Dr. Chi (pictured right) is actively seeking to bring principles of learning from cognitive science into the hands of teachers so that their instruction can transform student learning.

Daniel Schwartz, who has spent his career bringing principles of learning from cognitive science into the classroom, received the 2015 Sylvia Scribner Award, which honors current research that represents a significant advancement in our understanding of learning and instruction. Dr. Schwartz (pictured left) delivered his award address at this week’s AERA meeting. His current IES project is seeking to create a set of principles to select problem sets for students in science, technology, engineering, and math (STEM) domains.

You can learn more about the AERA awards on their website. Congratulations to all the winners! 

Photo Credits: Douglas and Lynn Fuchs, Vanderbilt University; Andrew Porter, University of Pennsylvania; Michelene Chi, Arizona State University; Daniel Schwartz, Stanford University

By Elizabeth Albro, Associate Commissioner, NCER

The Institute of Education Sciences at AERA

The American Educational Research Association (AERA) will hold its annual meeting April 8 through April 12 in Washington, D.C.—the largest educational research gathering in the nation. This will be a special meeting for AERA, as it is celebrating 100 years of advocating for the development and use of research in education. The program includes hundreds of sessions, including opportunities to learn about cutting edge education research and opportunities to broaden and deepen the field. 

About 30 sessions will feature staff from the Institute of Education Sciences (IES) discussing IES-funded research, evaluation, and statistics, as well as training and funding opportunities.

On Saturday, April 9, at 10:35 a.m., attendees will have a chance to meet the Institute’s leadership and hear about the areas of work that IES will be focusing on in the coming year. Speakers include Ruth Curran Neild, IES’ delegated director, and the leaders of the four centers in IES: Thomas Brock, commissioner of the National Center for Education Research (NCER); Peggy Carr, acting commissioner of the National Center for Educational Statistics (NCES); Joy Lesnick, acting commissioner of the National Center for Education Evaluation and Regional Assistance (NCEE), and Joan McLaughlin, commissioner of the National Center for Special Education Research (NCSER).

On Monday, April 11, at 9:45 a.m., attendees can speak to one of several IES staffers who will be available at the Research Funding Opportunities—Meet Your Program Officers session. Program officers from NCER, NCSER, and NCEE will be on hand to answer questions about programs and grant funding opportunities. Several IES representatives will also be on hand Monday afternoon, at 4:15 p.m. for the Federally Funded Data Resources: Opportunities for Research session to discuss the myriad datasets and resources that are available to researchers.

NCES staff will lead sessions and present on a variety of topics, from The Role of School Finance in the Pursuit of Equity (Saturday, 12:25 p.m.) to Understanding Federal Education Policies and Data about English Learners (Sunday, April 10, 8:15 a.m.) and what we can learn from the results of PIAAC, a survey of adult skills (also Sunday, 8:15 a.m.). Dr. Carr will be a part of several sessions, including one on Sunday morning (10:35 a.m.) about future directions for NCES longitudinal studies and another on Monday morning (10 a.m.) entitled Issues and Challenges in the Fair and Valid Assessment of Diverse Populations in the 21st Century

On Monday, at 11:45 a.m., you can also learn about an IES-supported tool, called RCT-YES, that is designed to reduce barriers to rigorous impact studies by simplifying estimation and reporting of study results (Dr. Lesnick will be among those presenting). And a team from the IES research centers (NCER/NCSER) will present Sunday morning (10:35 a.m.) on communication strategies for disseminating education research (which includes this blog!).

IES staff will also participate in a number of other roundtables and poster sessions. For instance, on Tuesday, April 12, at 8:15 a.m., grab a cup of coffee and attend the structured poster session with the Institute’s 10 Regional Educational Laboratories (RELs). This session will focus on building partnerships to improve data use in education.  REL work will also be featured at several other AERA sessions.  

Did you know that the National Library of Education (NLE) is a component of IES? On Friday and Monday afternoon, attendees will have a unique opportunity to go on a site visit to the library. You’ll learn about the library’s current and historical resources – including its collection of more than 20,000 textbooks dating from the mid-19th century. The Library offers information, statistical, and referral services to the Department of Education and other government agencies and institutions, and to the public.

If you are going to AERA, follow us on Twitter to learn more about our sessions and our work.  And if you are tweeting during one of our sessions, please include @IESResearch in your tweet. 

By Dana Tofig, Communications Director, IES

Building a Better RFA

The Institute of Education Sciences (IES) is committed to continuous improvement and that includes the process by which people apply for and access grants.

Since its authorization in 2002, IES’ research centers—the National Center for Education Research (NCER) and the National Center for Special Education Research (NCSER)—have  been making efforts to improve the Requests for Applications (RFAs) we put out each year. In this spirit, we have conducted surveys of applicants the past few years and used that feedback to improve the current RFAs.

In Fiscal Years (FY) 2014 and 2015, all Principal Investigators (PIs) who submitted an application to the Education Research Grants Program RFA (CFDA # 84.305A) or the Partnerships and Collaborations Focused on Problems of Practice or Policy Program RFA (CFDA # 84.305H) were contacted via e-mail and asked to participate in the web-based survey. In FY 2015, applicants to the Special Education Research Grants Program (CFDA #324A) were included in the survey request.  The response rates were good for all surveys:

Grant Program FY 2014 FY 2015
Education Research Grants Program 62% 66%
Partnerships and Collaborations Program 59% 73%
Special Education Research Grants n/a 55%

 

Survey respondents generally provided positive feedback in both years. Most respondents indicated they felt the RFAs were clear and helpful, though there were some areas that generated some confusion and criticism.  For example, in FY 2014:

  • Applicants to the Education Research Grants program thought it was inconvenient to have to refer to two separate documents, the RFA and the Application Submission Guide, in order to complete their application.
  • Applicants to the Partnerships and Collaborations program reported some confusion about the distinction between partnership activities and research activities.

In response to the FY 2014 RFA survey results, the Institute made a number of changes. For the FY 2015 Education Research Grants and Special Education Research Grants, changes included combining the RFA and the Application Submission Guide into one document to provide all the necessary information in one place. According to responses from the FY 2015 RFA survey, this change was positively received. The majority of the respondents to the Education Research Grants and Special Education Research Grants surveys (n=398; 83%) reported that combining the RFA and Application Submission Guide was much better or somewhat better than having two separate documents. Overall, a majority of respondents (n = 161, 56%) felt the FY 2015 RFA was much better or somewhat better than in previous years, while another 43 percent felt that it was not better or worse.

For the Partnerships and Collaborations RFA, a number of changes were made to the FY 2015 RFA in response to the surveys. For example, the requirements for the research activities were disentangled from the requirements for the partnership in order to reduce redundancy within the application. Most respondents to the FY 2015 RFA survey (n = 53; 73%) felt this change made the RFA much better or somewhat better.

Respondents to the FY 2015 RFA survey also had some criticisms, and the Institute addressed those concerns in the FY 2016 RFAs. Specifically, in the Education Research Grants and Special Education Research Grants RFAs, more detail was added to the requirements for the dissemination plan and for the cost analysis plan.  For the Education Research Grants RFA, the language around research gaps was expanded to clarify that these are not priorities. Changes made in the Special Education Research Grant RFA in response to the feedback from the survey included streamlining application requirements related to student disability, age range or grade level, outcomes, and settings across its 11 research topics.  More details were added about the partnership tracking strategy (an area of confusion for many applicants) in the Partnerships and Collaborations FY 2016 RFA.

IES continues to strive toward improving RFAs and welcomes comments and suggestions for improvement. More information on the RFA results is available here: https://ies.ed.gov/ncer/projects/.

New FY 2017 RFAs are being posted on the IES Funding Opportunities page. If you have comments, please write to us at IESresearch@ed.gov.

By Christina Chhin (NCER), Rebecca McGill-Wilkinson (NCER), Phill Gagné (NCER) and Kristen Rhoads (NCSER)*

* Since this blog post was written, Dr. Rhoads has taken a position with the U.S. Department of Education's Office of Special Education and Rehabilitative Services.