Inside IES Research

There is a lot of discussion these days about the importance of bringing many voices to the table when designing, implementing, and interpreting research studies. The usefulness of educational research is only enhanced when teachers, policmakers, and researchers work together to design studies and understand the findings.

The Educational Testing Service and the Council of Chief State School Officers (CCSSO) sponsored such an opportunity on May 18 and 19 by bringing together over 150 researchers, practitioners, instructional specialists, federal staff, and policy makers to reflect on the efforts of the of the Reading for Understanding Research Initiative (RfU).

Researchers from the six RfU teams shared what they learned about how to work together to design and implement new approaches and improve reading for understanding from PreK through high school. They also discussed what needs to be done to build on this effort.  (Check out this recent blog post to learn how some researchers are building on the work of the RfU Initiative.)

Sessions at the meeting were designed to promote discussion among attendees in order to take advantage of different viewpoints and  better understand the implications of the RfU findings and the relevance for practice and policy. The meeting culminated with two panels tasked with summarizing key points from these discussions. One panel focused on implications for research, and the other on policy implications. The presence of practitioners from the field, researchers, and policymakers led to a well-rounded conversation about how we can build upon the research of RfU and put it into action in the classroom.

The agenda, presentation slides, and webcasts of the closing panels are now available for viewing on the meeting website.  

Written by Karen Douglas, Education Research Analyst, NCER

After years of intense collaboration and research, the Reading for Understanding (RfU) Research Initiative is coming to an end. But the initiative’s work continues through recently announced IES-funded grants.

Over six years, research teams in the RfU network designed and tested new interventions that aim to improve reading comprehension in students in all grade levels and developed new measures of reading comprehension and component skills that support it. The initiative led to several new and important findings. In the coming years, several teams will build on that work through new research projects funded by IES’ National Center for Education Research (NCER) and National Center for Special Education Research (NCSER).  

During the RfU initiative, the Promoting Adolescents’ Comprehension of Text (PACT) team found positive effects in improving the content-area reading comprehension of middle school students. The PACT intervention uses social studies content to engage students and teach them to build coherent representations of the ideas in texts. Through a new grant from NCER, the PACT team will be testing the effectiveness of the intervention in middle school social studies classrooms in eight states.

Another group of researchers from the PACT team are starting a new project with funding from NCSER to design and test a technology-based intervention aimed at improving how middle school students with reading disabilities make inferences while reading.  

The RfU assessment team is also launching a new NCER-funded project to develop a digital assessment appropriate for adults, in particular those reading between the 3rd- to 8th-grade levels. Building on the Global, Integrated Scenario-Based Assessment (GISA) developed in RfU, the team intends for this new assessment to help determine an adult reader's strengths and weaknesses, inform instruction, and improve programs and institutional accountability. In addition, this team is using assessment items developed with RfU funding to explore the relationship between high school students' background knowledge and their reading comprehension.  

Finally, the Florida State University RfU team is continuing to explore which combination of interventions will improve the early language skills that are foundational to mastery of reading. In this new project, the researchers will examine the relative efficacy and sustained impacts of a language and vocabulary intervention for prekindergarten and kindergarten students, with variations on when and how long the intervention is used.

Written by Elizabeth Albro, Associate Commissioner, NCER

Most people remember being told not to talk in class or risk a trip to the principal’s office or a note sent home. But researchers in the Reading for Understanding Research Initiative (RfU) want students to talk in class as a way to improve reading comprehension.

Five research teams in the RfU network have designed and tested new interventions intended to provide a strong foundation for reading comprehension in students from pre-kindergarten through high school. And promoting high quality language use and talk among students is a central feature of many of these interventions. The goal is to improve reading outcomes by building students’ understanding of rich syntax and academic language to express and evaluate complex ideas.

RfU researchers have conducted studies in 29 states and interventions developed by the RfU network have been tested for efficacy with over 30,000 students (see the chart to the right for more information on the grantees and the map below to see where they conducted research).

While findings from these studies are still forthcoming, some interventions already show promise toward improving reading for understanding and/or supporting skills. New assessments have been field-tested with over 300,000 students across the country and have documented their capacity to collect valid and useful information for teachers, schools, and researchers.

Support for informative and instructional talk by students was provided in a variety of ways across different academic areas, including social studies, science, and English language arts classes. Some teams developed new classroom activities to structure whole class discussion through student debate on current topics of interest. Using a program like 

Word Generation, students discuss a focal question to stimulate various opinions on current topics, such as ‘Should students be required to wear school uniforms?’ or ‘Are green technologies worth the investment?’  In other interventions, such as PACT, students spend time talking in pairs or small groups to reinforce a new concept or idea.

Teachers are understandably concerned about how to manage a classroom in which students are talking. As part of RfU, curricula and materials were created to help teachers to improve their skills in managing constructive student talk, and several teams also provided extensive professional development for teachers.

Attention to the importance of student talk was also evident in a computer-based assessment called GISA developed by ETS which uses a scenario-based approach. Rather than talking with their peers during the assessments, students interact with avatars on a task that simulates a realistic classroom-based task.

Using student talk to improve reading comprehension is just one of many supports that have been explored by the RfU teams in their extensive body of work over the past six years. The RfU teams provided an update on their research during an event in May. You can watch a webcast of the event until July 31, 2016.

Visit the IES website to see a detailed agenda for the May event and to learn more about the work of the Reading for Understanding Research Initiative. In addition to providing an overview of the work, the abstracts include links to RfU team websites and many of these have examples of their materials. Materials for the Word Generation and PACT interventions are available for free on their websites, and several other RfU grantees will be making their materials freely available in the coming year.

Written by Karen Douglas, project lead, Reading for Understanding Research Initiative, National Center for Education Research

 

By Becky McGill-Wilkinson, NCER Program Officer

Gone are the days of library card catalogs and having to consult the 26-volume hardbound encyclopedia gathering dust on your parents’ bookshelf. Students these days have seemingly infinite information at the tips of their fingers. Most households in the U.S. have a computer, and most teachers report at least one computer in their classrooms. Research shows that the majority of high school students use the Internet to complete school assignments, and 71 percent of students use their laptop computers for school. In this changing world, it becomes more and more important to understand how reading and researching on the Internet are different from performing those tasks with books and other paper texts.

Don Leu and his team at the University of Connecticut have been examining this topic for several years. First on their agenda was studying whether reading online is the same as reading on paper. They discovered that students who are poor readers on paper may be good readers online, and students who are good readers on paper are not necessarily good readers online, suggesting that reading online requires some unique skills. Leu and his collaborators argue that reading online requires that students be able to: (1) use search engines; (2) choose appropriate search result; (3) judge whether the source can be trusted to be accurate and unbiased; and (4) consolidate information across multiple websites or online texts.

Of course, it’s not enough to understand the process of reading and researching online. As with any skill, some students are better at it than others, and as computers, tablets, and smart phones become more common, it becomes more and more necessary for students to hone their online reading and research skills if they are to succeed in college and career. Teachers need to be able to teach these skills, and teachers need to be able to identify when their students need extra help or practice. In 2005, Leu received a grant from NCER to study Internet use in adolescents at risk for dropping out of school, and developed an intervention to help teach seventh-grade students specific strategies to locate, evaluate, synthesize, and communicate information on the Internet.

Building on this earlier work, in a 2009 grant from NCER, Leu and his team set out to develop measures of online reading comprehension. The end result of this project is a set of Online Research and Comprehension Assessments (ORCAs) for use with seventh grade students. The team developed both a multiple choice version and a version that allows students to work in a simulated internet environment. In both versions, the student is tasked with answering a research question posed by a simulated peer, and must use a search engine, choose the appropriate search result, determine whether a source is trustworthy, and then tell their simulated peer about what they found. The ORCAs were tested with 2,700 students in two different states, and the researchers surveyed teachers and other practitioners to determine whether the ORCAs were usable.

Leu has been especially interested in thinking about how changing ideas about literacy may impact low-income students differently from middle- and high-income students. In a recently published paper, Leu shows that students who came from families earning approximately $100,000 per year were more than a year ahead of students whose families earn approximately $60,000 per year on online reading abilities as measured by the ORCAs. This study highlights the importance of considering the achievement gaps between high- and low-income students on a variety of domains, including those not typically measured by standardized tests, such as online reading comprehension.

The ORCAs are available online for free, as is a professional development module to help teachers learn to use it. 

Questions? Comments? Please email us at IESResearch@ed.gov.

By Liz Berke, NCSER Intern and Kristen Rhoads, NCSER Program Officer

Welcome to the last installment of our three-part series featuring the Principal Investigators of the inaugural NCSER Early Career Development and Mentoring grants.  To round the series out, we are featuring the work of Dr. Michael Hebert from the University of Nebraska, Lincoln. Dr. Hebert is a former reading specialist in California.

Dr. Hebert is being mentored by Ron Nelson, also from the University of Nebraska, Lincoln.  In his Early Career project, Dr. Hebert is working to improve reading comprehension in the content areas for children with or at risk for learning disabilities.  His intervention Structures, takes place in small groups led by a teacher, and focuses on helping students understand text structure to enhance reading comprehension. 

We had the chance to interview Dr. Hebert and he gave us his insights on the challenges of being a young researcher.

What are some of the biggest challenges that you face as a young researcher? How do you hope this award will help you overcome those challenges?

This is a tough question to answer because I don’t know whether I am aware of all of the challenges I will face.  There are a lot of challenges (big and small) that I will not be aware of until they come up for the first time.  Sometimes these things have to do with the rules and regulations of my university with regard to budgeting or post award support, while others may be challenges to working with student research assistants.  This grant has been great for helping me identify the challenges of funded research projects and learn to work with entities like the Office of Sponsored Programs at my university.  Let’s face it, if I didn’t have these funds now, I wouldn’t learn to navigate these challenges until later in my career.  

Additionally, I find that it is simply a challenge to get research off the ground as an early career researcher.  Establishing relationships with schools, planning studies, establishing systems for collecting and analyzing data, and other tasks take time.  Although some of these things are probably a challenge for all researchers, people who are more established in their careers might already have a lot of strategies and systems in place.  This grant has given me some personnel resources that help with some of the more basic tasks, essentially creating more of the most valuable resource we have… time.

What advice would you give to young researchers? 

First, apply for the IES Early Career Grant award, of course.  It is an excellent way to get started in your research program, while allowing you to develop some additional skills at the same time.  The development aspect of the grant really forces you to focus on some skill areas that may not be your strong suit.  Second, take advantage of every resource you can at your university.  If you can find graduate assistant support or even undergraduate support, hire them even if you aren’t sure what you will do with them yet.   You’ll be surprised how much you can find for them to do.  Even if you have to spend some time training them, the return is worth it. Also, it is really rewarding to share what you know with the next generation of potential researchers.

What is your favorite aspect of working with your mentor?

Working with my mentor has been invaluable.  We work together on aspects of the grant multiple days each week, and sometimes on a daily basis.  We’re almost partners in the research, and he has challenged some of my ideas, while I have been able to challenge some of his, as well.  This type of working relationship has really been a collaboration of sorts, and given me good experience working together with a colleague on projects.   I’ve also had the opportunity to co-mentor one of his doctoral students, which is a nice way to learn.  

What made you decide to apply to for the early career development and mentoring award? 

The Request for Applications for this award came out while I was completing my dissertation, so I actually decided to apply before I started working at my university.  It seemed like the perfect fit, as I was not doing a postdoctoral position, but felt that I needed mentorship in my first position.  I had a research mentor in mind at the university and he agreed to mentor me, so it made the decision to apply very easy.  There were a lot of changes in my life at the time, including moving and starting a new position as an Assistant Professor, so I didn’t have much time to think it over.  That said, I wouldn’t go back and do anything differently.  

Questions? Comments? Please send us an email IESResearch@ed.gov.