Inside IES Research

Notes from NCER & NCSER

Improving the Reading Skills of Middle School Students with and at Risk for Disabilities

Two girls work together to write in notebooks.

NCSER celebrates Middle Level Education Month this year by highlighting some of our current research projects aimed at supporting the literacy skills of middle school students with and at risk for disabilities. Data from the National Assessment of Education Progress (NAEP) indicate that in 2019, 68% of eight graders with disabilities scored “Below NAEP Basic” in reading compared to 22% of students without disabilities, a gap that had grown larger over the previous decade. The urgent need to improve the reading skills of middle school students with disabilities has led to some important NCSER-funded research projects.  

Sharon Vaughn and Leticia Martinez at the University of Texas, Austin and Jeanne Wanzek at Vanderbilt University, along with their colleagues, are testing the efficacy of Promoting Adolescents' Comprehension of Text (PACT), a fully developed reading comprehension intervention for middle school students with evidence of efficacy for students without disabilities. The current study will be the first efficacy trial of PACT that focuses specifically on students with disabilities. PACT is a text- and inquiry-based reading comprehension intervention with instructional supports for teachers and material for students in general education classrooms. In this large-scale, multi-site study, the team is using a randomized controlled trial in approximately 80 eighth-grade social studies classrooms, each with at least two students with disabilities. The team will examine the intervention’s impact on student reading and social studies outcomes, whether the impact differs depending on level of teacher fidelity or by student characteristic, and the intervention’s cost-effectiveness over the typical expenditures.

PACT is also being used as the evidence-based literacy intervention in a project designed to support middle school teachers’ knowledge and practices and improve reading and content area knowledge among students with disabilities. Jade Wexler at University of Maryland, College Park and Elizabeth Swanson at the University of Texas, Austin, along with their colleagues, are developing and testing a model for instructional leaders to provide ongoing support to content-area middle school teachers as they implement PACT. More specifically, the team will be developing an intervention package that includes a multi-stage, adaptive intervention coaching model to systematically tailor support to teachers as they implement Tier 1 literacy practices (in this case, the PACT intervention) in their content areas (English language arts, social studies, and science) to improve reading outcomes for students with disabilities. The package will also include a professional development program to train instructional leaders on how to implement the coaching model with teachers effectively.

Deborah Reed at the University of Iowa and her colleagues are also focusing on integrating literacy instruction with content-area instruction, but this team is focused on the Tier 2 level. They are developing and testing an intervention for middle school students with or at risk for reading disabilities who need support in literacy and text-based content in science and social studies. Pairs of students will alternate reading science and social studies texts, with specific academic vocabulary language, on a digital platform. This platform will provide scaffolded support and allow opportunities for individual work in building related reading and writing skills. Passages on each science or social studies topic repeat 85% or more of the unique words but in different contexts to support students’ ability to recognize and read the academic vocabulary. The overall aim of the intervention is to improve student literacy as well as science and social studies performance.

Marcia Barnes and her team at Vanderbilt University are testing the efficacy of a reading comprehension intervention, Connecting Text by Inference and Technology (Connect-IT), with middle school students with or at risk for reading disabilities. Connect-IT, developed with a prior IES grant, was designed to improve inference-making and reading comprehension in this population. In the current study, the research team will examine the impact of the intervention on students in grades 6-8 who did not pass their state English Language Arts test and who have demonstrated difficulties in reading comprehension. They will compare the efficacy of the intervention as implemented in small groups by a teacher, individual implementation of the intervention through computer software with project interventionist supervision, and the school’s business-as-usual classes. The study aims to determine the effect of each version of the intervention on student inference-making abilities and reading comprehension, as well as whether various student skills (such as vocabulary, word reading, attention, and anxiety) may moderate the impact of the interventions. The interventions’ cost-effectiveness will also be evaluated.

Focused on more intensive intervention for middle school students with or at risk for reading  disabilities, Mary Beth Calhoon at the University of Miami is testing the efficacy of a 2-year implementation of the Adolescent Multi-Component Intensive Training Program (AMP-IT-UP), which was previously tested after 1 year of implementation through an IES-funded grant. The intervention uses direct, systematic, explicit instruction (in phonological decoding with comprehension, spelling, and fluency) and cognitive strategy instruction (including use of cues and anchors), combined with reciprocal peer-mediated instruction. Dr. Calhoon’s research team is conducting a randomized controlled trial with middle school students with or at risk for reading disabilities who are still reading at the third-grade level or below. They will examine the impact after 2 years of intervention as well as 1 year after intervention has ended to determine its effect on student word recognition, spelling, fluency, and comprehension skills.

We look forward to reporting on the results of these studies as these teams complete their work in the years to come.

The blog was authored by Amy Sussman (Amy.Sussman@ed.gov), Sarah Brasiel (Sarah.Brasiel@ed.gov), and Katie Taylor (Katherine.Taylor@ed.gov), program officers for NCSER.

A2i: From Research to Practice at Scale in Education

This blog post is part of an interview series with education researchers who have successfully scaled their interventions.

Assessment-to-Instruction (A2i) is an online Teacher Professional Support System that guides teachers in providing Kindergarten to Grade 3 students individualized literacy instruction and assessments. Students complete the assessments independently online without the teacher taking time away from instruction. A2i generates instantaneous teacher reports with precise recommendations for each student and group recommendations. See a video demo here. Between 2003 and 2017, researchers at Florida State University (FSU) and Arizona State University (ASU), led by Carol Connor, developed and evaluated A2i with the support of a series of awards from IES and the National Institutes of Health. Findings from all publications on the A2i are posted here.

While results across seven controlled studies demonstrated the effectiveness of A2i, feedback from practitioners in the field demonstrated that implementation often required substantial amounts of researcher support and local district adaptation, and that the cost was not sustainable for most school district budgets. In 2014, the development firm Learning Ovations, led by Jay Connor, received an award from the Department of Education (ED) and IES’s Small Business Innovation Research program (ED/IES SBIR) to develop an technologically upgraded and commercially viable version of A2i to be ready to be used at scale in classrooms around the country. In 2018, with the support of a five-year Education Innovation and Research (EIR) expansion grant from ED totaling $14.65 million, A2i is now used in more than 110 schools across the country, with plans for further expansion. 

 

Interview with Carol Connor (CC) and Jay Connor (JC)

From the start of the research in the early 2000s, was it always the goal to develop a reading intervention that would one day be used on a wide scale?
CC: Yes and no. First, we had to answer the question as to whether individualization was effective in achieving student literacy outcomes. Once the research established that, we knew that this work would have wide-scale application.

When did you start thinking about a plan for distribution
CC: Before embarking on the cumulative results studies, in 2008, Jay said that we needed to know who the “customer” was… i.e., how purchasing decisions were made at scale.  His 2008 Phase I ED/IES SBIR was critical in shifting our research focus from individual classrooms to school districts as the key scaling point. 

Did you work with a technology transfer office at the university?
CC: Only to the extent of contractually clarifying intellectual property (IP) ownership and licensing. 

Who provided the support on the business side?
CC: Jay, who has an MBA/JD and has been a senior officer in two Fortune 100 companies was very instrumental in guiding our thinking of this evolution from important research to practical application. 


 Do you have any agreement about the IP with the university? What were the biggest challenges in this area?

JC: Yes, Learning Ovations has a 60-year renewable exclusive licensing agreement with FSU Foundation. FSU couldn’t have been better to work with.  Though there were expected back-and-forth elements of the original negotiations, it was clear that we shared the central vision of transforming literacy outcomes.  They continue to be a meaningful partner.

When and why was Learning Ovations first launched?
JC: In order to pursue SBIR funding we needed to be a for-profit company.  At first, I used my consulting business – Rubicon Partners LLP – as the legal entity for a 2008 Phase I award from ED/IES SBIR.  When we considered applying (and eventually won) a Fast Track Phase I & II award from SBIR in 2014, it was clear that we needed to create a full C – Corp that could expand with the scaling of the business, thus Learning Ovations was formed.

Who has provided you great guidance on the business side over the year? What did they say and do? 
JC: Having run large corporate entities and worked with small business start-ups in conjunction with Arizona State University (Skysong) and the University of California, Irvine (Applied Innovation at The Cove) and having taught entrepreneurship at The Paul Merage School of Business at UC Irvine, I had the experience or network to connect to whatever business guidance we needed.  Further, having attended a number of reading research conferences with Carol, I was quite conversant in the literacy language both from the research side and from the district decision maker’s side.

How do you describe the experience of commercializing the A2i? What were the biggest achievements and challenges in terms of preparing for commercialization?

JC: Having coached scores of entrepreneurs at various stages, I can safely say that there is no harder commercialization than one that must stay faithful to the underlying research.  A key strategy for most new businesses: being able to pivot as you find a better (easier) solution.  It is often circumscribed by the “active ingredients” of the underlying research.  Knowing this, we imbued Learning Ovations with a very strong outcomes mission – all children reading at, or above, grade level by 3rd grade.  This commitment to outcomes certainty is only assured by staying faithful to the research.  Thus, a possible constraint, became our uncontroverted strength.

Do you have advice for university researchers seeking to move their laboratory research in education into wide-spread practice? 
JC:  Start with the end in mind.  As soon as you envision wide-scale usage, learn as much as you can about the present pain and needs of your future users and frame your research questions to speak to this.  Implementation should not be an after-the-fact consideration; build it into how you frame your research questions. On one level you are asking simultaneously “will this work with my treatment group” AND “will this help me understand/deliver to my end-user group.”  I can’t imagine effective research being graphed onto a business after the fact.  One key risk that we see a number of researchers make is thinking in very small fragments whereas application (i.e., the ability to go to scale) is usually much more systemic and holistic.

In one sentence, what would say is most needed for gaining traction in the marketplace?
JC: If not you, as a researcher, someone on your team of advisors needs to know the target marketplace as well as you know the treatment protocols in your RCT.

____________

Carol Connor is a Chancellor’s Professor in the UC Irvine School of Education. Prior she was a professor of Psychology and a Senior Learning Scientist at the Learning Sciences Institute at ASU. Carol’s research focuses on teaching and learning in preschool through fifth grade classrooms – with a particular emphasis on reading comprehension, executive functioning, and behavioral regulation development, especially for low-income children.

Joseph “Jay” Connor, JD/MBA, is the Founder/CEO of Learning Ovations, Inc, the developer of the platform that has enabled the A2i intervention to scale.  Jay has 20+ years of experience in senior business management at the multi-billion dollar corporate level, and has experience in the nonprofit and public policy arenas.

This interview was produced by Edward Metz of the Institute of Education Sciences.

Fieldnotes: Reflections from an Adult Education Instructor on Research and Practice

Approximately 18 percent of US adults are at the lowest levels of literacy and nearly 30 percent are at the lowest levels of numeracy. The adult education system serves adults with low skills, but many education researchers know little about the students or the setting.  Recently, NCER convened a working group of adult education instructors, administrators, and researchers to discuss adult education’s research and dissemination needs.

Mr. Marcus Hall, an adult education instructor at the Community Learning Center and JEVS Human Services in Philadelphia, participated in this working group. He spoke with Meredith Larson, NCER program officer for adult education, about his experiences and interests in research. A copy of the working group meeting summary is available here.

Please describe your adult education classroom.

I once taught a 7-week course with students ranging from 18 to over 60 years old who had low literacy or math scores. I tried to contextualize instruction around their career interests and differentiate it to their learning needs. For example, some students were proficient readers but needed comprehension and math practice while others struggled with one or more of the basic components of reading. Somehow, I needed to help those learning phonics and those struggling with fluency while also challenging those ready for comprehension work. It’s hard to meet student needs in such a short time without teacher aides or adaptive technologies.

Why is research particularly important for adult education?

The challenges we face are monumental. Despite the large number of adults in need, adult education feels under-funded, under-staffed, and under-appreciated. Our students need complex, comprehensive, and well-rounded intervention, but we often have to make the most out of slightly targeted, inexpensive, and difficult-to-implement solutions. We need researchers to provide practical information and recommendations that we can use today to help adults learn and retain information.

Have you used research into your teaching?

Specifically, for reading instruction, I use techniques and activities built on evidence-based reading interventions. I start with tested diagnostic assessments to determine the needs of my students followed by strategies such as Collaborative Oral Reading or Repeated Reading exercises to support my students.

What topic during the meeting stood out to you?

The discussion about the workforce and professional development resonated with me. Many of our educators are part-time, come out of K-12, close to retirement, and may not have specific training for working with adults. They are asked to teach subjects they may not have any certification in, and their programs may not be able to provide the professional development they need. Just as we need supports for our learners, we need research to develop supports for us educators.

What additional research would you like to see?

Many of my students have had traumatic experiences that, when relived in the classroom, can cause them to disengage or struggle. I feel that understanding triggers and signs of discomfort has greatly enhanced my ability to help my students. Many educators want to leverage mental health approaches, like trauma-informed care, but we could use help learning how to integrate these strategies into instruction.

What do you hope researchers and educators keep in mind regarding one another?

It seems that researchers publish and promote their work to other researchers and then move to the next topic. This may be due to time constraints, publishing demands, or institutional requirements. I hope researchers take the time to come into our settings and observe us in action. I want researchers to work with us to help us understand and accept what is and isn’t working.

As for educators, we need to not try things and then stop using them when something unexpected occurs. At times, we revert back to what we know and are most comfortable with in the classroom. We educators can and must think critically about our norms and be ready and willing to enhance our practice with new information. 

Inequity Persists in Gifted Programs

The National Center for Research on Gifted Education (NCRGE) at the University of Connecticut, in Phase I of a rigorous research agenda, examined how academically-gifted students are identified and served in three states in order to provide systematic information for the field. The research team focused especially on the representation of historically underserved groups in gifted education.

NCER recently spoke with the Center’s Principal Investigator, Del Siegle, a nationally-recognized expert on gifted education. 

What is the biggest challenge facing gifted educators today?

Unfortunately, many of our nation’s brightest students from underserved populations (e.g., Black, Hispanic, English Learner, and/or free and reduced-price lunch eligible) are not being identified as gifted and do not receive gifted education services. About 80% of states that completed the most recent National Association for Gifted Children’s State of the States survey indicated that underrepresentation of students from underserved populations was an important or very important issue in their state.

What did you find in your study of identification of underserved students for gifted programs?

During Phase I of our work, we analyzed standardized student achievement test data from three states that mandate gifted identification and programming. We found that schools were less likely to identify students from underserved groups as gifted—even in cases where the underserved child had similar achievement test scores. For example, students with similar test scores who received free and reduced price lunch were less than half as likely to be identified as gifted as students who didn’t receive free or reduced price lunch.

What identification practices are schools using?

Cognitive tests and teacher nominations were the most common identification tools across the three states we studied. The majority (90% to 96%) of the districts in all three states used these practices to select students. Identification for gifted services occurs most often in third grade. Districts seldom reassess identified students once they are identified and only about half reassess non-identified students in elementary schools at regular intervals. Screening all children and using a variety of identification criteria showed promise for reducing under-identification in one of our states.

How are students being serviced in gifted programs?

In the three states we studied, schools primarily focused on critical thinking and creativity followed by communication skills, research skills, and self-directed projects.  Mathematics and reading language arts acceleration was much less of a focus and were ranked among the bottom third of focus areas. Gifted students seldom receive gifted programming in core academic areas. Only 29% of the schools provided a separate gifted curriculum in reading/language arts. Only 24% of the schools had a separate gifted curriculum in mathematics. Gifted students spent 5 hours or more each week in regular education mathematics and reading/language arts classrooms. Of the 74% of schools reporting using pull-out services, only 32% offered separate gifted curriculum in reading/language arts and 28% offered separate gifted curriculum in math. 

What about gifted student growth in mathematics and reading?

In 3rd grade, gifted students are approximately 2 grade levels ahead of students not identified as gifted, but gifted students grow more slowly than non-gifted students between 3rd and 5th grade. Most grouping arrangements for gifted students had no impact on the growth of academic achievement. We believe much of this has to do with the limited advanced mathematics and reading instruction gifted students receive in their classrooms and gifted programs.

What is the next step in your research?

We are examining the effect of attending dedicated gifted classes in core content areas on academic achievement in reading/language arts and mathematics in a large, ethnically, economically, and linguistically diverse urban school district. Our research will compare the reading/language arts and mathematics achievement of gifted students in three different settings: schools offering a full-time gifted-only program with gifted classes in all subject areas, schools offering a part-time gifted-only program with gifted classes in mathematics, and schools offering a part-time gifted-only program with gifted classes in reading/language arts.

Read Across America with IES

Happy Read Across America Day! This year is the 10-year anniversary of this national pep rally for reading, and IES has supported the development of a number of tools to promote reading and literacy.

Did you know that many of the curricula and materials developed by IES researchers are available for free? These materials include reading on topics interesting to students, as well as guidance for teachers on how to engage and motivate students in discussions about what they read. For example, as part of the Reading for Understanding Initiative, IES invested in multiple curricula that are designed to help improve students’ reading comprehension and are available at no charge.

For students in preschool through grade 3, the Let’s Know! curriculum supplement uses easily-accessible books to help teach children about vocabulary, making inferences, and text structures like cause and effect. There’s also a Spanish version of this curriculum (¡Vamos Aprender!). You can gain access to the curriculum through the Language and Reading Research Consortium webpage.

Word Generation is a group of curricula developed for students in grades four through eight with a focus on teaching students to understand multiple perspectives, reason, and learn academic vocabulary, all through high-interest topics in science and social studies.

Example topic questions from units include:

  • When is a crime not a crime?

  • The Legacy of Alexander the Great: Great Leader or Power-Hungry Tyrant? and

  • Thinking About Natural Selection.

You can find more information on WordGen and download materials on their website.

Finally, for high school students, Promoting Adolescents’ Comprehension of Text (PACT) is an intervention aimed at motivating and engaging students to read and understand informational texts in social studies. Students learn vocabulary words and make connections between social studies topics and their own lives. For example, in a unit about the 1920s, students learn about economy and prosperity and complete activities such as listing three items they have purchased and determining whether they are “needs” or “wants,” and how this relates to a consumer economy. Sample materials are available for download on the PACT website.

Have fun celebrating Read Across America Day, and enjoy a book with the students in your lives!

By Becky McGill-Wilkinson, NCER Program Officer