IES Blog

Institute of Education Sciences

RCT-YES: Supporting a Culture of Research Use in Education

June 14, 2016 IES Blog Editor Research Share on TwitterShare on FacebookShare on Google+

By Ruth Curran Neild, Delegated Director, IES

The mission of the Institute of Education Sciences (IES), at its core, is to create a culture in which independent, rigorous research and statistics are used to improve education. But sometimes research is seen by practitioners and policymakers as something that is done for them or to them, but not by them. And that’s something we’re hoping to change.

IES is always looking for new ways to involve educators in producing and learning about high-quality, useful research. We believe that if state and school district staff see themselves as full participants in scientific investigation, they will be more likely to make research a part of their routine practice. Simply put, we want to make it easier for educators to learn what works in their context and to contribute to the general knowledge of effective practices in education.    

That’s why we’re so pleased to add the RCT-YESTM software to the IES-funded toolkit of free, user-friendly resources for conducting research. Peter Schochet of Mathematica Policy Research, Inc. led the development of the software, as part of a contract with IES held by Decision Information Resources, Inc.

RCT-YES has a straightforward interface that allows the user to specify the analyses for data from a randomized controlled trial (RCT) or a quasi-experiment. Definitions and tips in the software help guide the user and accompanying documentation includes a mini-course on RCTs. When the user enters information about the data set and study design, RCT-YES produces a program to run the specified analyses (in either R or Stata) and provide a set of formatted tables.

The target users are those who have a basic knowledge of statistics and research design but do not have advanced training in conducting or analyzing data from impact studies. But we expect that even experienced researchers will like the simplicity and convenience of RCT-YES and benefit from some of its novel features, such as how it reports results.

When used properly, RCT-YES provides all of the statistics needed by the What Works ClearinghouseTM (WWC) to conduct a study review.  This is an important feature because the WWC often needs to contact authors—even experienced ones—to obtain additional statistics to make a determination of study quality.  RCT-YES could help advance the field by increasing the completeness of study reports.

Another unique feature of the software is that it defaults to practices recommended by IES’ National Center for Education Statistics for the protection of personally identifiable information. For example, the program suppresses reporting on small-size subgroups.

While the user sees only the simplicity of the interface, the underlying estimation methods and code required painstaking and sophisticated work.  RCT-YES relies on design-based estimation methods, and the development, articulation, peer review, and publication of this approach in the context of RCT-YES was the first careful step. Design-based methods make fewer assumptions about the statistical model than methods traditionally used in education (such as hierarchical linear modeling), making this approach especially appropriate for software designed with educators in mind.

The software is available for download from the RCT-YES website, where you can also find support videos, documentation, a user guide, and links to other helpful resources. The videos below, which are also hosted on the RCT-YES website, give a quick overview of the software.

There are many other ways that IES fosters a culture of research use in education. For instance, our 10 Regional Educational Laboratories (RELs) have research alliances that work with states and districts to develop research agendas. The RELs also host events to share best practices for putting research into action, such as the year-long series of webinars and training sessions on building, implementing, and effectively using Early Warning Systems to reduce dropping out.

IES also offers grants to states and districts to do quick evaluations of programs and policies that have been implemented in their schools. The low-cost, short-duration evaluations not only help the grantees discover what is working, but can help others who might use the same program or implement a similar policy. (We’ll announce the first round of grant recipients in the coming weeks).

Visit the IES website to learn more about our work. You can also stay on top of news and information from IES by following us on Facebook and Twitter. 

 

 

 

When It’s Good to Talk in Class

June 9, 2016 Blog Editor NCER, NCSER Share on TwitterShare on FacebookShare on Google+

Most people remember being told not to talk in class or risk a trip to the principal’s office or a note sent home. But researchers in the Reading for Understanding Research Initiative (RfU) want students to talk in class as a way to improve reading comprehension.

Five research teams in the RfU network have designed and tested new interventions intended to provide a strong foundation for reading comprehension in students from pre-kindergarten through high school. And promoting high quality language use and talk among students is a central feature of many of these interventions. The goal is to improve reading outcomes by building students’ understanding of rich syntax and academic language to express and evaluate complex ideas.

RfU researchers have conducted studies in 29 states and interventions developed by the RfU network have been tested for efficacy with over 30,000 students (see the chart to the right for more information on the grantees and the map below to see where they conducted research).

While findings from these studies are still forthcoming, some interventions already show promise toward improving reading for understanding and/or supporting skills. New assessments have been field-tested with over 300,000 students across the country and have documented their capacity to collect valid and useful information for teachers, schools, and researchers.

Support for informative and instructional talk by students was provided in a variety of ways across different academic areas, including social studies, science, and English language arts classes. Some teams developed new classroom activities to structure whole class discussion through student debate on current topics of interest. Using a program like 

Word Generation, students discuss a focal question to stimulate various opinions on current topics, such as ‘Should students be required to wear school uniforms?’ or ‘Are green technologies worth the investment?’  In other interventions, such as PACT, students spend time talking in pairs or small groups to reinforce a new concept or idea.

Teachers are understandably concerned about how to manage a classroom in which students are talking. As part of RfU, curricula and materials were created to help teachers to improve their skills in managing constructive student talk, and several teams also provided extensive professional development for teachers.

Attention to the importance of student talk was also evident in a computer-based assessment called GISA developed by ETS which uses a scenario-based approach. Rather than talking with their peers during the assessments, students interact with avatars on a task that simulates a realistic classroom-based task.

Using student talk to improve reading comprehension is just one of many supports that have been explored by the RfU teams in their extensive body of work over the past six years. The RfU teams provided an update on their research during an event in May. You can watch a webcast of the event until July 31, 2016.

Visit the IES website to see a detailed agenda for the May event and to learn more about the work of the Reading for Understanding Research Initiative. In addition to providing an overview of the work, the abstracts include links to RfU team websites and many of these have examples of their materials. Materials for the Word Generation and PACT interventions are available for free on their websites, and several other RfU grantees will be making their materials freely available in the coming year.

Written by Karen Douglas, project lead, Reading for Understanding Research Initiative, National Center for Education Research

 

 

VIDEO: See How ERIC Selects New Sources

June 1, 2016 Blog Editor ERIC Share on TwitterShare on FacebookShare on Google+

By Erin Pollard, ERIC Project Officer, NCEE

ERIC builds a strong education research collection by continuously seeking out new sources of rigorous content and adding them to the collection. But how does ERIC select publications for the online library?

A new video (embedded below) provides the answer to how ERIC selects new sources, including education-focused journals, grey literature reports, and conference papers. The video was developed to help answer one of the most frequently asked questions by ERIC users and to help publishers and organizations producing materials in the field of education understand what ERIC considers when evaluating potential new sources. Watch this video if you want to learn about the types of resources ERIC will and will not index, the source selection process, and how to recommend a new resource.

Twice a year, in the spring and fall, ERIC reviews journals and producers of conference papers, reports, and books as potential candidates for inclusion in ERIC, using a revised selection policy as a guide when evaluating recommended content. The revised policy was released in January 2016 to clarify the types of materials ERIC is seeking for the collection. ERIC considers resources that are education research focused and include citations, orginal analyses of data, and well-formed arguments. ERIC also considers collection priorities, such as peer- reviewed and full-text materials.

We are continuously working to build a strong education research collection that includes the latest and very best resources in the field. If you are a publisher of high-quality education research, or if you have a favorite journal, or know a source of conference papers or reports not currently in ERIC, please send us your recommendations.

To stay up-to-date on ERIC, follow us on Facebook, YouTube, or Twitter and check out the Notes area of the ERIC website at eric.ed.gov. 

IES at the APS Annual Convention

May 27, 2016 Blog Editor NCER, NCSER Share on TwitterShare on FacebookShare on Google+

Every Memorial Day weekend, thousands of psychological scientists meet to discuss findings from current research at the Association for Psychological Sciences (APS) annual convention. Representatives and grantees from the two research centers at the Institute of Education Sciences (IES) will participate in the 28th annual convention, sharing what we are learning about ways to improve education for all learners.

Erin Higgins, the program officer for Cognition and Student Learning program in the National Center for Education Research (NCER), will discuss current IES funding opportunities on Saturday, May 28, at 1 p.m. (Learn more about current IES funding opportunities.)  Dr. Higgins is also chairing a session on Sunday, May 29th at 1 p.m. focused on the role that graphs, diagrams, and other visual representations play in mathematics. This sessions features  NCER grantees Steven Franconeri, Jennifer Cromley, Martha Alibali, and James McClelland.

We want to extend our congratulations to one of our first IES grantees, Robert Bjork, who is delivering one of three APS James McKeen Cattell Fellow Award Addresses. These awards are given for a lifetime of outstanding contributions to the area of applied psychological research. The research of award recipients addresses a critical problem in society at large.

More information about which IES grantees are participating in the APS convention is available on the NCER website. If you're tweeting about IES funded work at the conference, please tag @IESResearch. 

Addressing Mental Health Needs in Schools, Pre-K to Grade 12

May 26, 2016 Blog Editor NCER, NCSER Share on TwitterShare on FacebookShare on Google+

May is National Mental Health Awareness Month and for educators, mental health is a serious issue. Students who are suffering with mental health issues will have a harder time learning and thriving in school.  

The Institute of Education Sciences (IES) has funded and supported work that seeks to identify how schools can support the 1 in 5 students in the United States who experience a mental health disorder such as disruptive behavior, anxiety, and attention deficit hyperactivity disorder. Below is a snapshot of some of that work.

Preschool

  • Jason Downer (University of Virginia) is developing the Learning to Objectively Observe Kids (LOOK) protocol to help prekindergarten teachers identify and understand children’s engagement in preschool and choose appropriate techniques to supports children’s self-regulation skills.

Elementary School

  • Golda Ginsburg (University of Connecticut) and Kelly Drake (Johns Hopkins University) are developing the CALM (Child Anxiety Learning Modules) protocol for elementary school nurses to work with children who have excessive anxiety.
  • Desiree Murray (University of North Carolina, Chapel Hill) is testing the Incredible Years Dina Dinosaur Treatment Program (IY-child) for helping early elementary school students with social-emotional and behavioral difficulties. IY-child is a small-group, mixed-age pullout program co-led by a clinical therapist and a school-based counselor. Students view brief video vignettes of same-age children in different situations where social-emotional skills and self-regulation are modeled. Students also participate in discussions facilitated by life-sized puppets, and engage in role-play practices and small group activities. Group leaders also provide individual consultation to teachers of participating students.
  • Gregory Fabiano (SUNY-Buffalo) is adapting the Coaching Our Acting Out Children: Heightening Essential Skills (COACHES) program for implementation in schools. This is a clinic-based program to help fathers of children with or at risk for attention-deficit/hyperactivity disorder (ADHD) get more involved and engaged in their child's school performance. 
  • Aaron Thompson (University of Missouri) is testing the Self-Monitoring Training and Regulation Strategy (STARS) intervention to see if it can improve behavior, social emotional learning skills, and academic performance for fifth grade students who engage in disruptive or otherwise challenging classroom behaviors.
  • Karen Bierman (Pennsylvania State University) is testing whether an intensive, individualized social skills training program, the Friendship Connections Program (FCP), can remediate the serious and chronic peer difficulties that 10–15 percent of elementary school students experience. Most of these students have or are at risk for emotional or behavioral disorders and exhibit social skill deficits (e.g., poor communication skills, inability to resolve conflict) that alienate peers. 

Middle School

High School

Policy

  • Sandra Chafouleas (University of Connecticut) is identifying current policies and national practice related to school-based behavioral assessment to determine whether current practice follows recommended best practice, and to develop policy recommendations for behavioral screening in schools. 

Written by Emily Doolittle, Team Lead for Social and Behavioral Research at IES, National Center for Education Research