IES Blog

NCES staff presented information on reading data from the Early Childhood Longitudinal Studies (ECLS) Program at the June 2021 Institute of Education Sciences (IES)/Council of the Great City Schools (CGCS) Reading Summit. The ECLS data cover a wide range of reading-related topics, such as children’s reading knowledge and skills, home literacy activities, and teachers’ instructional practices. The presentation included a brief overview of three ECLS program studies and the reading-related data collected by each. In addition, the presentation included a discussion of the resources available to either see what research has been conducted with the data or explore the data independently. As the focus of the presentation was on data available to the public for secondary analysis, its target audience was researchers and others with a data science focus.

Access the Reading Summit presentation—Reading Data Available from the Early Childhood Longitudinal Studies (ECLS)—and handout below to learn more about ECLS reading data.

• Video presentation
• Presentation slides
• Resources handout

Be sure to also check out this blog post to learn more about the work highlighted at the IES Reading Summit.

By Jill Carlivati McCarroll, NCES

As part of our recognition of Native American Heritage Month, we asked Diane Loeb to discuss her IES-funded research on culturally responsive language and literacy enrichment for Native American children.

Development of language and exposure to early literacy is critical to a child’s academic success. Speaking and listening skills are necessary to navigate learning at every level of school. According to NCES, American Indians/Alaska Native populations have the highest percentage of students who receive services under the Individuals with Disabilities Education Act. There continues to be a significant need for Native American speech-language pathologists and audiologists, culturally sensitive assessment tools, and intervention approaches.

In 2006, I had the privilege to work with ten Native American college students who were recruited to the University of Kansas for the speech-language pathology and audiology master’s program. The students were from tribes across the country and varied greatly in their undergraduate preparation and world experiences. One thing that they had in common is that they wanted to make a difference in the lives of others—in particular, those who needed help with their speech, language, hearing skills, and related difficulties. As a result of working with these amazing students, I learned about their families, their customs, and their dreams. I also became painfully aware of the historical trauma Native Americans experience as a result of genocide, colonialism, and racism. In the twentieth century, Native Americans were sent to boarding schools and deprived of their language, culture, and their family.

As the students advanced in their academic studies and clinical work, it became clear to me that there were very few resources for identifying and intervening with language delay and language disorder. Under- and over-identification for special education services were highly possible due to our lack of understanding of Native American history, level of family assimilation, and inter-tribal differences. Although there were a handful of articles related to conducting assessments, very few studies addressed culturally sensitive and responsive intervention, where children’s cultural values and beliefs, experiences, and how they learn guide the assessment and intervention. The lack of culturally responsive tools for Native Americans propelled me to write an IES-funded grant proposal designed to implement culturally authentic intervention designed to be meaningful, sensitive, and respectful of Native American culture.

As a result of the IES grant we received, we developed a culturally based language and vocabulary intervention for Native American kindergarten children at risk for speech and language impairment, as well as a training program for teachers and speech-language pathologists. Language and literacy lessons were based on positive stories about Native Americans in storybooks and storytelling was taught through the venue of shared reading. Native American adults from the Native American school we were working with examined our materials to ensure that our activities were in line with the values and beliefs of the participating children. Pilot testing suggested that students made gains in literacy and language skills following intervention. 

My colleague, Grace McConnell, and I recently published an in-depth analysis of the narratives produced by the children in our initial studies. We found distinct trends in narrative structure and evaluative comments depending on student age and whether there were visual supports. What we found highlights the importance of culturally responsive language and literacy interventions for Native American children. There remains a great need for these interventions. From my work, I have learned several important lessons that may be useful to current and future researchers. The three most salient to me are

• Include members of the tribe with whom you are working as part of the process of developing assessments and interventions for children who are Native American. This helps to ensure that your assessments and interventions are culturally sensitive.
• Develop authentic materials that are culturally relevant, sensitive, and meaningful. We found several books with positive cultural lessons, such as respecting the earth, working together, and harmony with others and nature.
• Remember that tribes can differ substantially from one another and that families may differ regarding cultural values and beliefs within a given tribe. When we designed literacy and language units around Native American storybooks, they often were related to specific tribes (such as Navajo or Apache). This gave us the opportunity to discuss different tribes in various parts of the country and for the children to learn about and compare their own customs and beliefs with another tribe. Students also learned about different family practices within their own tribe by sharing their family experiences with other children.

Following my work with Native American students and children, I pursued grant and research opportunities focused on the development of children born preterm of all races/ethnicities. I am working with neonatologists and nurses on studies to improve the developmental outcomes of children born preterm. Approximately 25% of children born preterm are later diagnosed with language delay or language disorder. I am currently designing NICU interventions to facilitate language, cognitive, motor, and social interaction skills that support academic success. A future goal is to focus my intervention work with Native American infants born preterm and their families. Providing facilitation of language and literacy early in development for these at-risk infants may be key for their later academic success.

Diane Loeb at Diane_Loeb@Baylor.edu is the Martin Family Endowed Chair of Communication Sciences and Disorders and Department Chair at Robbins College of Health and Human Sciences at Baylor University in Waco, Texas. She is a first-generation college graduate. This research was conducted while she was an Associate Professor at the University of Kansas in Lawrence, KS.

This guest blog was produced by Katina Stapleton (Katina.Stapleton@ed.gov), co-Chair of the IES Diversity and Inclusion Council, and Amy Sussman (Amy.Sussman@ed.gov), NCSER Program Officer.

Image: Dr. Hannah Dostal (left), University of Connecticut, and Dr. Kimberly Wolbers (right), University of Tennessee

September is National Literacy Month and Deaf Awareness Month. To celebrate both occasions, we spoke with two IES-funded principal investigators about their intervention aimed at increasing the writing and language skills of students who are deaf or hard of hearing through teacher professional development targeting writing instruction and use of multilingual strategies. Together with their team, Dr. Kimberly Wolbers (University of Tennessee) and Dr. Hannah Dostal (University of Connecticut) developed Strategic and Interactive Writing Instruction (SIWI) and tested SIWI for efficacy. The team is now analyzing effects of SIWI on both student and teacher outcomes in the D/HH space.

What are some challenges facing deaf and hard of hearing (D/HH) students in the area of literacy? How does your project address these student-related challenges?

Children who are D/HH are highly diverse with respect to language modality (spoken, sign) and proficiency. Understanding this diversity is the foundation to their literacy learning and academic engagement. Working between languages and across modalities when engaged in literacy tasks is a unique challenge for D/HH writers. For example, a student may use American Sign Language (ASL), which does not have a written form, while learning and using English text as they read and write. Strategies used during writing instruction that scaffold bilingual and multilingual development have the potential to leverage student knowledge of languages to support literacy development. During SIWI, teachers engage students in explicitly comparing and contrasting ASL and English with the intention of increasing metalinguistic knowledge and translation abilities.

Another unique challenge is that a number of D/HH students lack consistent exposure to accessible language at home and school. They may not hear sufficient amounts of spoken language to acquire its complexities and may not have sufficient exposure to sign language early in their lives to acquire visual language. Such language deprivation directly impacts literacy development. Based on what we are seeing and learning from our school partners, online learning due to COVID-19 exacerbated delays in academic progress when D/HH students experienced greater language isolation during this time.

Teachers implementing SIWI tackle expressive language delays head on. They use a designated space in the classroom, called the Language Zone, to develop, translate, and revise ideas generated in ASL and English.

What are some challenges facing teachers of D/HH in the area of literacy? How does your project address these teacher-related challenges?

It is becoming increasingly more challenging to find qualified teachers of the deaf. Not only are there shortages in the field, but many current teachers also point to limited preparation and a lack of assessment materials, curriculum, and instructional resources specifically designed for D/HH students with distinct languages histories.

The SIWI professional development (PD) program is designed to address these challenges facing teachers. It is a multi-component PD program that is intensive and sustained over a 3-year period and consists of a summer institute, site visits, and individual biweekly online coaching. Teachers not only learn about effective approaches but also how to flexibly enact the approaches with students who have diverse language histories and literacy skills.

What have you found so far?

SIWI has been implemented across settings with D/HH students and studies so far suggest SIWI results in significant language and literacy growth. Results from IES-funded studies using a variety of methods demonstrate SIWI’s positive impact on student outcomes. For example, we found a relationship between SIWI and positive student gains in the effective use of genre-related writing traits and grammar and conventions, including an increase in the length of writing as well as written language clarity and complexity. Recent analyses, currently in press, demonstrate that in one academic year, students participating in SIWI gained an average of 1.2 grade levels as measured by the Woodcock Johnson IV.

The SIWI PD program has also resulted in notable outcomes for SIWI teachers. The randomized control trial demonstrated significant increases in teachers’ knowledge of writing instruction, efficacy in teaching writing, and use of evidence-based practices compared to teachers in the business-as-usual control group (a manuscript is currently in progress). 

What are the next steps for your research?

Analyses of student outcomes in the efficacy trial are currently underway. In addition to analyzing the impact of SIWI on writing and language outcomes, we are also examining the impact on reading comprehension, vocabulary knowledge, handwriting, and motivation to write.

Additionally, we are investigating whether implementation fidelity of SIWI is positively associated with student outcomes. We intend to examine whether teachers with higher implementation fidelity in their second or third year of teaching SIWI demonstrate a significantly greater impact on their students’ writing and language growth.

Dr. Kimberly Wolbers is a Deaf Education Professor and Co-Director of Undergraduate Studies for the Department of Theory & Practice in Teacher Education at the University of Tennessee and Dr. Hannah Dostal is an Associate Professor of Reading Education and an advisory board member of the Aetna Chair of Writing at the University of Connecticut. This interview was produced and edited by Julianne Kasper, Virtual Student Federal Service Intern at IES and graduate student in Education Policy & Leadership at American University.

In recent decades, there has been an emphasis on quantitative, causal research in education policy. These methods are best suited for answering questions about the effects of a policy and whether it achieved its intended outcomes. While the question of, “Did it work?” remains critical, there is a need for research that also asks, “Why did it work? For whom? In what contexts?” To answer these types of questions, researchers must incorporate rigorous qualitative methods into their quantitative studies. Education research organizations like the Association for Education Finance and Policy have explicitly invited proposals using qualitative and mixed methodologies in an effort to elevate research addressing a range of critical education policy questions. Funding organizations, including IES, encourage applicants to incorporate qualitative methods into their research process. In this guest blog, Amy Cummings, Craig De Voto, and Katharine Strunk discuss how they are using qualitative methods in their evaluation of a state education policy.

In our IES-funded study, we use qualitative, survey, and administrative data to understand the implementation and impact of Michigan’s early literacy law—the Read by Grade Three Law. Like policies passed in 19 other states, the Read by Grade Three Law aims to improve K-3 student literacy skills and mandates retention for those who do not meet a predetermined benchmark on the state’s third-grade English language arts assessment. Although the retention component of these policies remain controversial, similar laws are under consideration in several other states, including Alaska, Kentucky, and New Mexico. Below are some of the ways that we have integrated qualitative methods in our evaluation study to better understand the policy process in the development of the Read by Grade Three Law.  

Collecting qualitative sources helped us understand how the policy came to be, thereby assisting in the structure of our data collection for examining the law’s implementation and subsequent effects. In our first working paper stemming from this study, we interviewed 24 state-level stakeholders (policymakers, state department of education officials, early literacy leaders) involved in the development of the law and coded state policy documents related to early literacy to assess the similarity between Michigan’s policy and those of other states. Understanding the various components of the Law and how they ended up in the policy led us to ensure that we asked educators about their perceptions and implementation of these components in surveys that are also part of our evaluation. For example, because our interviews made clear the extent to which the inclusion of the retention component of the Law was controversial during its development, we included questions in the survey to assess educators’ perceptions and intended implementation of this component of the Law. In addition, it confirmed the importance of our plan to use state administrative retention and assessment data to evaluate the effect of retention on student literacy outcomes.

To trace the Read by Grade Three Law’s conception, development, and passage, we analyzed these qualitative data using two theories of the policy process: Multiple Streams Framework (MSF) and policy transfer. MSF says that policy issues emerge on government agendas through three streams: problem, policy, and political. When these streams join, a policy window is opened during which there is a greater opportunity for passing legislation. Meanwhile, policy transfer highlights how policies enacted in one place are often used in the development of policies in another.

We found that events in the problem and political streams created conditions ripe for the passage of an early literacy policy in Michigan:

• A national sentiment around improving early literacy, including a retention-based third-grade literacy policy model that had been deemed successful in Florida
• A pressing problem took shape, as evidenced by the state’s consistently below average fourth-grade reading scores on the National Assessment of Educational Progress
• A court case addressing persistently low-test scores in a Detroit-area district
• Previous attempts by the state to improve early literacy

As a result of these events, policy entrepreneurs—those willing to invest resources to get their preferred policy passed—took advantage of political conditions in the state and worked with policymakers to advance a retention-based third-grade literacy policy model. The figure below illustrates interviewee accounts of the Read by Grade Three Law’s development. Our policy document analysis further reveals that Michigan’s and Florida’s policies are very similar, only diverging on nine of the 50 elements on which we coded.

Although this study focuses on the development and passage of Michigan’s early literacy law, our findings highlight both practical and theoretical elements of the policy process that can be useful to researchers and policymakers. To this end, we show how particular conditions, coupled by policy entrepreneurs, spurred Michigan’s consideration of such a policy. It is conceivable that many state education policies beyond early literacy have taken shape under similar circumstances: a national sentiment combined with influential brokers outside government. In this way, our mixed-methods study provides a practical model of what elements might manifest to enact policy change more broadly.

From a theoretical standpoint, this research also extends our understanding of the policy process by showing that MSF and the theory of policy transfer can work together. We learned that policy entrepreneurs can play a vital role in transferring policy from one place to another by capitalizing on conditions in a target location and coming with a specific policy proposal at the ready.

There is, of course, more to be learned about the intersection between different theories of the policy process, as well as how external organizations as opposed to individuals operate as policy entrepreneurs. As the number of education advocacy organizations continues to grow and these groups become increasingly active in shaping policy, this will be an exciting avenue for researchers to continue to explore.

This study is just one example of how qualitative research can be used in education policy research and shows how engaging in such work can be both practically and theoretically valuable. The most comprehensive evaluations will use different methodologies in concert with one another to understand education policies, because ultimately, how policies are conceptualized and developed has important implications for their effectiveness.

Amy Cummings is an education policy PhD student and graduate research assistant at the Education Policy Innovation Collaborative (EPIC) at Michigan State University (MSU).

Craig De Voto is a visiting research assistant professor in the Learning Sciences Research Institute at the University of Illinois at Chicago and an EPIC affiliated researcher.

Katharine O. Strunk is the faculty director of EPIC, the Clifford E. Erickson Distinguished Chair in Education, and a professor of education policy and by courtesy economics at MSU.

On June 8 and 9, the Institute of Education Sciences (IES) and the Council of the Great City Schools (CGCS) will host a Reading Summit to address one of the most important issues confronting American education today: the declining reading performance of America’s lowest-performing students and the growing gap between low- and high-performing students.

At this 2-day virtual event, participants will explore the results of the National Assessment of Educational Progress (NAEP), as well as other IES data, and learn strategies to help educators and low-performing readers make progress.

Learn more about the summit’s agenda and speakers—including IES Director Mark Schneider, NCES Commissioner James L. Woodworth, and NCES Associate Commissioner Peggy Carr—and register to participate (registration is free).

In the meantime, explore some of the data NCES collects on K–12 literacy and reading achievement, which show that the scores of students in the lowest-performing groups are decreasing over time.

• The National Assessment of Educational Progress (NAEP) administers reading assessments to 4th-, 8th-, and 12th-grade students. The most recent results from 2019 show that average reading scores for students in the 10th percentile (i.e., the lowest-performing students) decreased between 2017 and 2019 at grade 4 (from 171 to 168) and grade 8 (from 219 to 213) and decreased between 2019 and 2015 at grade 12 (from 233 to 228).

• The Progress in International Reading Literacy Study (PIRLS) is an international comparative assessment that measures 4th-grade students’ reading knowledge and skills. The most recent findings from 2016 show that the overall U.S. average score (549) was higher than the PIRLS scale centerpoint (500), but at the 25th percentile, U.S. 4th-graders scored lower in 2016 (501) than in 2011 (510).

• The Program for International Student Assessment (PISA) is a study of 15-year-old students’ performance in several subjects, including reading literacy. The 2018 results show that, although the overall U.S. average reading score (505) was higher than the OECD average score (487), at the 10th percentile, the U.S. average score in 2018 (361) was not measurably different from the score in 2015 and was lower than the score in 2012 (378).

NCES also collects data on young children’s literacy knowledge and activities as well as the literacy competencies of adults. Here are a few data collections and tools for you to explore:

• The National Household Education Surveys (NHES) program collects data on young children’s home literacy activities with family members. See the section below for more on these data. 

• The International Early Learning Study (IELS) is an international play-based assessment of 5-year-olds’ knowledge and skills in four key areas, including emergent literacy. Learn about the 2018 pilot study.

• The Program for the International Assessment of Adult Competencies (PIAAC) assesses the skills of adults across the world in various domains, including literacy.

This year, the Condition of Education includes a newly updated indicator on literacy activities that parents reported doing with young children at home. Here are some key findings from this indicator, which features data from the 2019 NHES Early Childhood Program Participation Survey:

In the week before the parents were surveyed,

• 85 percent of 3- to 5-year-olds were read to by a family member three or more times.

• 87 percent of 3- to 5-year-olds were told a story by a family member at least once.

• 96 percent of 3- to 5-year-olds were taught letters, words, or numbers by a family member at least once.

In the month before the parents were surveyed,

• 37 percent of 3- to 5-year-olds visited a library with a family member at least once.

Be sure to read the full indicator in the 2021 Condition of Education, which was released in May, for more data on young children’s literacy activities, including analyses by race/ethnicity, mother’s educational attainment, and family income.

Don’t forget to follow NCES on Twitter, Facebook, and LinkedIn to stay up-to-date on the latest findings and trends in literacy and reading data and register for the IES Reading Summit to learn more about this topic from experts in the field. 

By Megan Barnett, AIR