IES Blog

Institute of Education Sciences

Access an NCES Presentation on ECLS Reading Data From the IES Reading Summit

NCES staff presented information on reading data from the Early Childhood Longitudinal Studies (ECLS) Program at the June 2021 Institute of Education Sciences (IES)/Council of the Great City Schools (CGCS) Reading Summit. The ECLS data cover a wide range of reading-related topics, such as children’s reading knowledge and skills, home literacy activities, and teachers’ instructional practices. The presentation included a brief overview of three ECLS program studies and the reading-related data collected by each. In addition, the presentation included a discussion of the resources available to either see what research has been conducted with the data or explore the data independently. As the focus of the presentation was on data available to the public for secondary analysis, its target audience was researchers and others with a data science focus.

Access the Reading Summit presentation—Reading Data Available from the Early Childhood Longitudinal Studies (ECLS)—and handout below to learn more about ECLS reading data.

Be sure to also check out this blog post to learn more about the work highlighted at the IES Reading Summit.

 

By Jill Carlivati McCarroll, NCES

Improving the Reading Skills of Middle School Students with and at Risk for Disabilities

Two girls work together to write in notebooks.

NCSER celebrates Middle Level Education Month this year by highlighting some of our current research projects aimed at supporting the literacy skills of middle school students with and at risk for disabilities. Data from the National Assessment of Education Progress (NAEP) indicate that in 2019, 68% of eight graders with disabilities scored “Below NAEP Basic” in reading compared to 22% of students without disabilities, a gap that had grown larger over the previous decade. The urgent need to improve the reading skills of middle school students with disabilities has led to some important NCSER-funded research projects.  

Sharon Vaughn and Leticia Martinez at the University of Texas, Austin and Jeanne Wanzek at Vanderbilt University, along with their colleagues, are testing the efficacy of Promoting Adolescents' Comprehension of Text (PACT), a fully developed reading comprehension intervention for middle school students with evidence of efficacy for students without disabilities. The current study will be the first efficacy trial of PACT that focuses specifically on students with disabilities. PACT is a text- and inquiry-based reading comprehension intervention with instructional supports for teachers and material for students in general education classrooms. In this large-scale, multi-site study, the team is using a randomized controlled trial in approximately 80 eighth-grade social studies classrooms, each with at least two students with disabilities. The team will examine the intervention’s impact on student reading and social studies outcomes, whether the impact differs depending on level of teacher fidelity or by student characteristic, and the intervention’s cost-effectiveness over the typical expenditures.

PACT is also being used as the evidence-based literacy intervention in a project designed to support middle school teachers’ knowledge and practices and improve reading and content area knowledge among students with disabilities. Jade Wexler at University of Maryland, College Park and Elizabeth Swanson at the University of Texas, Austin, along with their colleagues, are developing and testing a model for instructional leaders to provide ongoing support to content-area middle school teachers as they implement PACT. More specifically, the team will be developing an intervention package that includes a multi-stage, adaptive intervention coaching model to systematically tailor support to teachers as they implement Tier 1 literacy practices (in this case, the PACT intervention) in their content areas (English language arts, social studies, and science) to improve reading outcomes for students with disabilities. The package will also include a professional development program to train instructional leaders on how to implement the coaching model with teachers effectively.

Deborah Reed at the University of Iowa and her colleagues are also focusing on integrating literacy instruction with content-area instruction, but this team is focused on the Tier 2 level. They are developing and testing an intervention for middle school students with or at risk for reading disabilities who need support in literacy and text-based content in science and social studies. Pairs of students will alternate reading science and social studies texts, with specific academic vocabulary language, on a digital platform. This platform will provide scaffolded support and allow opportunities for individual work in building related reading and writing skills. Passages on each science or social studies topic repeat 85% or more of the unique words but in different contexts to support students’ ability to recognize and read the academic vocabulary. The overall aim of the intervention is to improve student literacy as well as science and social studies performance.

Marcia Barnes and her team at Vanderbilt University are testing the efficacy of a reading comprehension intervention, Connecting Text by Inference and Technology (Connect-IT), with middle school students with or at risk for reading disabilities. Connect-IT, developed with a prior IES grant, was designed to improve inference-making and reading comprehension in this population. In the current study, the research team will examine the impact of the intervention on students in grades 6-8 who did not pass their state English Language Arts test and who have demonstrated difficulties in reading comprehension. They will compare the efficacy of the intervention as implemented in small groups by a teacher, individual implementation of the intervention through computer software with project interventionist supervision, and the school’s business-as-usual classes. The study aims to determine the effect of each version of the intervention on student inference-making abilities and reading comprehension, as well as whether various student skills (such as vocabulary, word reading, attention, and anxiety) may moderate the impact of the interventions. The interventions’ cost-effectiveness will also be evaluated.

Focused on more intensive intervention for middle school students with or at risk for reading  disabilities, Mary Beth Calhoon at the University of Miami is testing the efficacy of a 2-year implementation of the Adolescent Multi-Component Intensive Training Program (AMP-IT-UP), which was previously tested after 1 year of implementation through an IES-funded grant. The intervention uses direct, systematic, explicit instruction (in phonological decoding with comprehension, spelling, and fluency) and cognitive strategy instruction (including use of cues and anchors), combined with reciprocal peer-mediated instruction. Dr. Calhoon’s research team is conducting a randomized controlled trial with middle school students with or at risk for reading disabilities who are still reading at the third-grade level or below. They will examine the impact after 2 years of intervention as well as 1 year after intervention has ended to determine its effect on student word recognition, spelling, fluency, and comprehension skills.

We look forward to reporting on the results of these studies as these teams complete their work in the years to come.

The blog was authored by Amy Sussman (Amy.Sussman@ed.gov), Sarah Brasiel (Sarah.Brasiel@ed.gov), and Katie Taylor (Katherine.Taylor@ed.gov), program officers for NCSER.

The Growing Reading Gap: IES Event to Link Knowledge to Action Through Literacy Data

On June 8 and 9, the Institute of Education Sciences (IES) and the Council of the Great City Schools (CGCS) will host a Reading Summit to address one of the most important issues confronting American education today: the declining reading performance of America’s lowest-performing students and the growing gap between low- and high-performing students.

At this 2-day virtual event, participants will explore the results of the National Assessment of Educational Progress (NAEP), as well as other IES data, and learn strategies to help educators and low-performing readers make progress.

Learn more about the summit’s agenda and speakers—including IES Director Mark Schneider, NCES Commissioner James L. Woodworth, and NCES Associate Commissioner Peggy Carr—and register to participate (registration is free).

In the meantime, explore some of the data NCES collects on K–12 literacy and reading achievement, which show that the scores of students in the lowest-performing groups are decreasing over time.

  • The National Assessment of Educational Progress (NAEP) administers reading assessments to 4th-, 8th-, and 12th-grade students. The most recent results from 2019 show that average reading scores for students in the 10th percentile (i.e., the lowest-performing students) decreased between 2017 and 2019 at grade 4 (from 171 to 168) and grade 8 (from 219 to 213) and decreased between 2019 and 2015 at grade 12 (from 233 to 228).
  • The Progress in International Reading Literacy Study (PIRLS) is an international comparative assessment that measures 4th-grade students’ reading knowledge and skills. The most recent findings from 2016 show that the overall U.S. average score (549) was higher than the PIRLS scale centerpoint (500), but at the 25th percentile, U.S. 4th-graders scored lower in 2016 (501) than in 2011 (510).
  • The Program for International Student Assessment (PISA) is a study of 15-year-old students’ performance in several subjects, including reading literacy. The 2018 results show that, although the overall U.S. average reading score (505) was higher than the OECD average score (487), at the 10th percentile, the U.S. average score in 2018 (361) was not measurably different from the score in 2015 and was lower than the score in 2012 (378).

NCES also collects data on young children’s literacy knowledge and activities as well as the literacy competencies of adults. Here are a few data collections and tools for you to explore:

This year, the Condition of Education includes a newly updated indicator on literacy activities that parents reported doing with young children at home. Here are some key findings from this indicator, which features data from the 2019 NHES Early Childhood Program Participation Survey:

In the week before the parents were surveyed,

  • 85 percent of 3- to 5-year-olds were read to by a family member three or more times.
  • 87 percent of 3- to 5-year-olds were told a story by a family member at least once.
  • 96 percent of 3- to 5-year-olds were taught letters, words, or numbers by a family member at least once.

In the month before the parents were surveyed,

  • 37 percent of 3- to 5-year-olds visited a library with a family member at least once.

Be sure to read the full indicator in the 2021 Condition of Education, which was released in May, for more data on young children’s literacy activities, including analyses by race/ethnicity, mother’s educational attainment, and family income.

Don’t forget to follow NCES on Twitter, Facebook, and LinkedIn to stay up-to-date on the latest findings and trends in literacy and reading data and register for the IES Reading Summit to learn more about this topic from experts in the field. 

 

By Megan Barnett, AIR

New International Data Identify “Resilient” Students in Financial Literacy

NCES recently released the results of the Program for International Student Assessment (PISA) 2018 assessment of financial literacy. This assessment measured 15-year-old students’ knowledge and understanding of financial concepts, products, and risks and their ability to apply that knowledge to real-life situations. It found that, on average, U.S. students performed similarly to their peers across the 12 other participating Organization for Economic Cooperation and Development (OECD) countries. 

The assessment also found that 12 percent of U.S. students performed at the highest level of proficiency (level 5). Performance at this level indicates that students can apply their understanding of financial terms and concepts to analyze complex financial products, solve nonroutine financial problems, and describe potential outcomes of financial decisions in the big picture.[1] The U.S. percentage was again similar to the OECD average.

However, this analysis also identified a group of students who might be considered “resilient.” In education research, resilience is defined as the ability to perform well academically despite coming from the disadvantaged backgrounds that have more commonly been associated with lower performance.

High-performing students came from across the spectrum of school poverty levels, as measured by the percentage of students eligible for free or reduced-price lunch (FRPL).[2] In particular, 7 percent of high-performing students in financial literacy came from the highest poverty schools (figure 1).


Figure 1. Percentage distribution of U.S. 15-year-olds in public schools scoring below level 2 and at level 5 of proficiency on the PISA financial literacy scale, by percentage of students eligible for free or reduced-price lunch (FRPL) at their school: 2018

NOTE: Data for percentage of students eligible for FRPL were available for public schools only. An individual student’s level of poverty may vary within schools. Detail may not sum to totals due to rounding.

SOURCE: Organization for Economic Cooperation and Development (OECD), Program for International Student Assessment (PISA), 2018.


It is these 7 percent of students who could be considered “resilient” and may be of interest for further study. For example, research could identify if there are factors that are associated with their high performance when compared to their lower performing peers in similar schools. Research on academically resilient students that used eighth-grade data from TIMSS found, for example, that having high educational aspirations increased the likelihood that students with few home education resources performed at or above the TIMSS Intermediate international benchmark in mathematics.[3] Experiencing less bullying also increased this likelihood.

Examining the “resilient” PISA financial literacy students more closely could also determine the extent to which their individual backgrounds are related to performance. This would be of interest because, even within high-poverty schools, students’ individual circumstances may vary. 

Patterns in Other PISA Subjects

There are similar subsets of “resilient” students in the other PISA 2018 subjects (table 1). Eight percent of high performers in reading were from the highest poverty schools, as were 5 percent of high performers in mathematics and 7 percent of high performers in science.


Table 1. Percentage of U.S. 15-year-olds in public schools scoring at or above level 5 of proficiency, by PISA subject and their schools’ free or reduced-price lunch (FRPL) status: 2018

[Standard errors appear in parentheses]

NOTE: Results are scaled separately; thus, percentages cannot be compared across subjects. Level 5 is the highest level of proficiency in financial literacy; levels 5 and 6 are the highest levels of proficiency in the other PISA subjects. Data for students eligible for FRPL were available for public schools only.

SOURCE: Organization for Economic Cooperation and Development (OECD), Program for International Student Assessment (PISA), 2018.


For more information on the PISA 2018 results in financial literacy and other subjects, visit the NCES International Activities website. To create customized data and charts using PISA and other international assessment data, use the International Data Explorer.

 

By Maria Stephens, AIR


[2] Data for students eligible for FRPL are available for public schools only.

[3] Students at the Intermediate international benchmark can apply basic mathematical knowledge in a variety of situations, and those above this benchmark can do so in increasingly complex situations and, at the highest end, reason with information, draw conclusions, make generalizations, and solve linear equations.

A2i: From Research to Practice at Scale in Education

This blog post is part of an interview series with education researchers who have successfully scaled their interventions.

Assessment-to-Instruction (A2i) is an online Teacher Professional Support System that guides teachers in providing Kindergarten to Grade 3 students individualized literacy instruction and assessments. Students complete the assessments independently online without the teacher taking time away from instruction. A2i generates instantaneous teacher reports with precise recommendations for each student and group recommendations. See a video demo here. Between 2003 and 2017, researchers at Florida State University (FSU) and Arizona State University (ASU), led by Carol Connor, developed and evaluated A2i with the support of a series of awards from IES and the National Institutes of Health. Findings from all publications on the A2i are posted here.

While results across seven controlled studies demonstrated the effectiveness of A2i, feedback from practitioners in the field demonstrated that implementation often required substantial amounts of researcher support and local district adaptation, and that the cost was not sustainable for most school district budgets. In 2014, the development firm Learning Ovations, led by Jay Connor, received an award from the Department of Education (ED) and IES’s Small Business Innovation Research program (ED/IES SBIR) to develop an technologically upgraded and commercially viable version of A2i to be ready to be used at scale in classrooms around the country. In 2018, with the support of a five-year Education Innovation and Research (EIR) expansion grant from ED totaling $14.65 million, A2i is now used in more than 110 schools across the country, with plans for further expansion. 

 

Interview with Carol Connor (CC) and Jay Connor (JC)

From the start of the research in the early 2000s, was it always the goal to develop a reading intervention that would one day be used on a wide scale?
CC: Yes and no. First, we had to answer the question as to whether individualization was effective in achieving student literacy outcomes. Once the research established that, we knew that this work would have wide-scale application.

When did you start thinking about a plan for distribution
CC: Before embarking on the cumulative results studies, in 2008, Jay said that we needed to know who the “customer” was… i.e., how purchasing decisions were made at scale.  His 2008 Phase I ED/IES SBIR was critical in shifting our research focus from individual classrooms to school districts as the key scaling point. 

Did you work with a technology transfer office at the university?
CC: Only to the extent of contractually clarifying intellectual property (IP) ownership and licensing. 

Who provided the support on the business side?
CC: Jay, who has an MBA/JD and has been a senior officer in two Fortune 100 companies was very instrumental in guiding our thinking of this evolution from important research to practical application. 


 Do you have any agreement about the IP with the university? What were the biggest challenges in this area?

JC: Yes, Learning Ovations has a 60-year renewable exclusive licensing agreement with FSU Foundation. FSU couldn’t have been better to work with.  Though there were expected back-and-forth elements of the original negotiations, it was clear that we shared the central vision of transforming literacy outcomes.  They continue to be a meaningful partner.

When and why was Learning Ovations first launched?
JC: In order to pursue SBIR funding we needed to be a for-profit company.  At first, I used my consulting business – Rubicon Partners LLP – as the legal entity for a 2008 Phase I award from ED/IES SBIR.  When we considered applying (and eventually won) a Fast Track Phase I & II award from SBIR in 2014, it was clear that we needed to create a full C – Corp that could expand with the scaling of the business, thus Learning Ovations was formed.

Who has provided you great guidance on the business side over the year? What did they say and do? 
JC: Having run large corporate entities and worked with small business start-ups in conjunction with Arizona State University (Skysong) and the University of California, Irvine (Applied Innovation at The Cove) and having taught entrepreneurship at The Paul Merage School of Business at UC Irvine, I had the experience or network to connect to whatever business guidance we needed.  Further, having attended a number of reading research conferences with Carol, I was quite conversant in the literacy language both from the research side and from the district decision maker’s side.

How do you describe the experience of commercializing the A2i? What were the biggest achievements and challenges in terms of preparing for commercialization?

JC: Having coached scores of entrepreneurs at various stages, I can safely say that there is no harder commercialization than one that must stay faithful to the underlying research.  A key strategy for most new businesses: being able to pivot as you find a better (easier) solution.  It is often circumscribed by the “active ingredients” of the underlying research.  Knowing this, we imbued Learning Ovations with a very strong outcomes mission – all children reading at, or above, grade level by 3rd grade.  This commitment to outcomes certainty is only assured by staying faithful to the research.  Thus, a possible constraint, became our uncontroverted strength.

Do you have advice for university researchers seeking to move their laboratory research in education into wide-spread practice? 
JC:  Start with the end in mind.  As soon as you envision wide-scale usage, learn as much as you can about the present pain and needs of your future users and frame your research questions to speak to this.  Implementation should not be an after-the-fact consideration; build it into how you frame your research questions. On one level you are asking simultaneously “will this work with my treatment group” AND “will this help me understand/deliver to my end-user group.”  I can’t imagine effective research being graphed onto a business after the fact.  One key risk that we see a number of researchers make is thinking in very small fragments whereas application (i.e., the ability to go to scale) is usually much more systemic and holistic.

In one sentence, what would say is most needed for gaining traction in the marketplace?
JC: If not you, as a researcher, someone on your team of advisors needs to know the target marketplace as well as you know the treatment protocols in your RCT.

____________

Carol Connor is a Chancellor’s Professor in the UC Irvine School of Education. Prior she was a professor of Psychology and a Senior Learning Scientist at the Learning Sciences Institute at ASU. Carol’s research focuses on teaching and learning in preschool through fifth grade classrooms – with a particular emphasis on reading comprehension, executive functioning, and behavioral regulation development, especially for low-income children.

Joseph “Jay” Connor, JD/MBA, is the Founder/CEO of Learning Ovations, Inc, the developer of the platform that has enabled the A2i intervention to scale.  Jay has 20+ years of experience in senior business management at the multi-billion dollar corporate level, and has experience in the nonprofit and public policy arenas.

This interview was produced by Edward Metz of the Institute of Education Sciences.