IES Blog

Institute of Education Sciences

How ERIC is Helping Students

By Erin Pollard, ERIC Project Officer, NCEE

Over the past several years, the staff of ERIC has worked hard to get to know its users—we want to understand who uses ERIC and how they are using it. We found that the most common users are college undergraduates using the ERIC electronic library to write research papers as part of their school work. We also learned that they often weren’t searching for the best resources to meet their needs.

For example, some users told the ERIC help desk they were writing a paper for an introductory “contemporary issues in education” course, but they were requesting documents from over 20 years ago. Also, some of the documents they wanted were not peer-reviewed or particularly relevant to their topic. It became clear that we could do a better job addressing our most frequent users’ needs.

First, we redesigned the ERIC website to be more friendly and intuitive for those who are not familiar with traditional search methods, making it more likely they would get the best results (learn more about searching ERIC). Second, we worked to develop tools to help novice users, including a new series of how-to videos.

The first video in this new series is aimed at students—Using ERIC to Write a Research Paper. The ERIC team worked with undergraduate faculty members to outline a step-by-step approach to describe how a student should begin a search process, refine their results, and use different fields to find the resources they need. The video describes how to use ERIC to find full-text resources and how users can know that they found right resources.

We asked school of education faculty and librarians for their thoughts on this video and received a lot of positive feedback. Some professors said that they would link to this video on their syllabi and post it on their course management systems, while librarians have posted it on their content management system, called LibGuides. Take a look and see what you think.  And if you have feedback on this or any other ERIC videos, or suggestions for new videos and tools, let us know through the ERIC Help Desk.

 

Why We Need Large-Scale Evaluations in Education

By Elizabeth Warner, Team Leader for Teacher Evaluations, NCEE

These days, people want answers to their questions faster than ever, and education research and evaluation is no exception. In an ideal world, evaluating the impact of a program or policy would be done quickly and at a minimal expense. There is increasing interest in quicker-turnaround, low-cost studies – and IES offers grants specifically for these types of evaluations.

But in education research, quicker isn’t always better. It depends, in part, on the nature of the program you want to study and what you want to learn.

Consider the case of complex, multi-faceted education programs.  By their very nature, complex programs may require several years for all of the pieces to be fully implemented. Some programs also may take time to influence behavior and desired outcomes.  A careful and thorough assessment of these programs may require an evaluation that draws on a large amount of data, often from multiple sources over an extended period of time. Though such studies can require substantial resources, they are important for understanding whether and why an investment had an intended effect. This is especially true for Federal programs that involved millions of dollars of taxpayer money.

On August 24, IES plans to release a new report from a large-scale, multi-year study of a complex Federal initiative: the Teacher Incentive Fund (TIF) [1] This report – the third from this evaluation -- will provide estimated program impacts on student achievement after three years of implementation. The Impact Evaluation of the Teacher Incentive Fund is a $13.7 million study that stretches for more than six years and has reported interim findings annually since 2014. (The graphic above is from the a study snapshot of the first TIF evaluation report.) 

A “big study” of TIF was important to do, and here’s why.

Learning from the Teacher Incentive Fund

TIF is a Federal program that provides grants to districts that want to implement performance pay with the goal of improving teacher quality.  TIF grants awarded in 2010 included the requirement that performance pay be based on an educator evaluation system with multiple performance measures consistent with recent research. The grantees were also expected to use the performance measures to guide educator improvement.

The TIF evaluation provides an opportunity not only to learn about impacts on student achievement over time as the grant activities mature but also to get good answers to implementation questions such as:

  • How do districts structure the pay-for-performance bonus component of TIF?
  • Are educators even aware of the TIF pay-for-performance bonuses?
  • Do educators report having opportunities for professional development to learn about the measures and to improve their performance?
  • Do educators change their practice in ways that improve their performance measures? 
  • Are principals able to use their ability to offer pay-for-performance bonuses to hire or retain more effective teachers?  

Analyses to address these questions can suggest avenues for program improvement that a small-scale impact evaluation with limited data collection would miss. 

Studying the Initiative Over Time

With four years of data collection, the TIF evaluation is longer than most evaluations conducted by IES. But the characteristics of the TIF program made that extensive data collection important for a number of reasons.

 

First, TIF’s approach to educator compensation differs enough from the traditional pay structure that it might take time for educators to fully comprehend how it works. They likely need to experience the new performance measures to see how they might score and how that translates in terms of additional money earned. Second, educators might need to receive a performance bonus or see others receive one in order to fully believe it is possible to earn a bonus. (The chart pictured here is from the second report and compares teachers’ understanding with the actual size of the maximum bonuses that they could receive. This chart will be updated in the third report.)

That also might help them better understand what behaviors are needed to earn a bonus.  Finally, time might be needed for educators to respond to all of the intended policy levers of the program, particularly related to recruitment and hiring.

The TIF evaluation is designed to estimate an impact over the full length of the grants as well as provide rich information to improve the program. Sometimes, even after a number of years, some aspects of a program are never fully implemented. Thus, it may take time to see if whether it is even possible to implement a complex policy like TIF with fidelity.  An evaluation that only looks at initial implementation of a complex program may miss important program components that are incorporated or refined with time. Also, it may take several years to determine if the intended educator behaviors and desired outcomes of the policy are realized.

Learning how and why a policy does or does not work is central to program improvement.  For large, complex programs like TIF, this assessment is only possible with data-rich study over an extended period of time.     

 

[1] Under the newly reauthorized Elementary and Secondary Education Act, this program is now called the Teacher and School Leader Incentive Program, but for this blog, we will call it by its original name.

A Conversation about Reading for Understanding

There is a lot of discussion these days about the importance of bringing many voices to the table when designing, implementing, and interpreting research studies. The usefulness of educational research is only enhanced when teachers, policmakers, and researchers work together to design studies and understand the findings.

The Educational Testing Service and the Council of Chief State School Officers (CCSSO) sponsored such an opportunity on May 18 and 19 by bringing together over 150 researchers, practitioners, instructional specialists, federal staff, and policy makers to reflect on the efforts of the of the Reading for Understanding Research Initiative (RfU).

Researchers from the six RfU teams shared what they learned about how to work together to design and implement new approaches and improve reading for understanding from PreK through high school. They also discussed what needs to be done to build on this effort.  (Check out this recent blog post to learn how some researchers are building on the work of the RfU Initiative.)

Sessions at the meeting were designed to promote discussion among attendees in order to take advantage of different viewpoints and  better understand the implications of the RfU findings and the relevance for practice and policy. The meeting culminated with two panels tasked with summarizing key points from these discussions. One panel focused on implications for research, and the other on policy implications. The presence of practitioners from the field, researchers, and policymakers led to a well-rounded conversation about how we can build upon the research of RfU and put it into action in the classroom.

The agenda, presentation slides, and webcasts of the closing panels are now available for viewing on the meeting website.  

Written by Karen Douglas, Education Research Analyst, NCER

Exploring the Role of Physical Activity, Inactivity, and Sleep in Academic Outcomes

What role do physical activity, inactivity, and sleep habits play in students’ academic outcomes? Some new IES-funded research grants will help us find out.

Three new IES grants funded under the Education Research Grants program will explore how students’ physical activity, sleep habits, and cognitive tempo are associated with their socio-emotional and academic outcomes.  Here is a brief summary of these studies along with their potential contributions to research, practice, and policy. 

Physical activity – While physical activity is generally accepted as a positive thing, its full effect on children's cognitive and academic outcomes remains uncertain and little is known about whether increased opportunities for physical activity are associated with improved academic achievement. In this new grant, Michael Willoughby, of RTI International,  and colleagues will examine whether, and under what conditions, individual differences in child physical activity in preschool settings are associated with enhanced executive functioning and academic achievement. The findings from this project may help inform school policy decisions about the frequency and amount of student physical activity during the school day.

Sleep - Sleep problems are considerably more common in individuals with Attention-Deficit/Hyperactivity Disorder (ADHD) in comparison to the general population. Approximately 30% of children and 60% of adults with ADHD exhibit significant sleep problems. Little is known, however, about how sleep problems contribute to the educational functioning of adolescents with ADHD or why prevalence rates are higher among adolescents with ADHD. In this new grant, Joshua Langberg, of Virginia Commonwealth University, will lead a team that will conduct a longitudinal study of students with and without ADHD from Grades 8 to 10. They will assess sleep patterns, academic, and social functioning, and factors that may differentially predict the presence of sleep problems. The findings from this project may lead to recommendations for how and when schools can include sleep assessments as part of psychoeducational evaluations and may help inform policy decisions about school contextual factors that can impact sleep, such as the amount of homework assigned. In addition, the findings from this project can be used to inform the development of a school-based intervention that addresses sleep problems.

Sluggish Cognitive Tempo - Sluggish cognitive tempo (SCT) refers to a specific set of attention symptoms, including excessive daydreaming, mental confusion, seeming to be "in a fog," and slowed thinking or behavior. In this new grant, Stephen Becker, of Cincinnati Children’s Hospital Medical Center, and his colleagues will examine the academic and socio-emotional problems experienced by students in grades 2 to 5 with elevated SCT symptoms. They will examine the current patterns of school referrals, educational accommodations, and interventions for children with elevated levels of SCT. The findings from this project can inform the development of interventions to mitigate the long-term consequences of sluggish cognitive tempo on students’ socio-emotional and academic outcomes.

Written by Christina Chhin, Education Research Analyst, National Center for Education Research

Gearing up for the Fall Semester? See How ERIC Can Help!

By Erin Pollard, ERIC Project Officer, NCEE

The ERIC Help Desk frequently receives questions from academic users asking if we have resources to help students and faculty use the Institute of Education Sciences' free online library of education research. In response, ERIC has created several videos that can be embedded in LibGuides (a content management system used by libraries), linked to in syllabi, and posted on course websites. These videos can also be accessed from ERIC’s Multimedia Page.

Our newest videos include:

About ERIC: The About ERIC video is an introduction to our digital library and answers basic questions about who our website is designed for and what types of information you’ll find there. Users can view this video to determine if ERIC is the right resource to use for their research.

Using ERIC to Write a Research Paper: The ERIC Help Desk has received a lot  of requests from students asking how to use ERIC to write a research paper.  This video was specifically created to be a resource that librarians and faculty members could direct students to as a starting point for research projects. For more information on this video, stay tuned for next week’s blog post!

Searching eric.ed.gov: ERIC was redesigned in 2013 using intuitive search technology, as opposed to more traditional searching mechanisms. This video explains how our search engine works and offers tips to help users get the best results.

How ERIC Selects New Sources: Users frequently ask how sources are selected to be indexed in ERIC. This video describes the process laid out in the ERIC Selection Policy and explains how users can recommend new resources for our collection.

IES Public Access Policy: How Grantees and Contractors Meet Requirements by Submitting Work to ERIC: Are you conducting research funded by IES? If so, then you are likely subject to the IES Public Access Policy (learn more about the policy for publications and data).  This video explains how researchers can meet their compliance requirements by submitting their work to ERIC.

Tips for Using the ERIC Online Submission System:  ERIC encourages authors to submit their work through our Online Submission System. Learn what types of materials are eligible to be included and how to prepare publications for submission to ERIC.

Have suggestions for other videos ERIC should make? Email our Help Desk! To learn more about ERIC, sign up for our Newsflash and follow us on Facebook and Twitter.