Skip Navigation

National Center for Education Evaluation and Regional Assistance


The Impact of a Reading Intervention for Low-Literate Adult ESL Learners

NCEE 2011-4003
December 2010

Executive Summary

According to the 2008 program year statistics from the U.S. Department of Education (ED), 44 percent of the 2.4 million students in the federally funded adult education program in the United States were English as a second language (ESL) students (ED, 2010). Of these, about 185,000 were at the lowest ESL level, beginning literacy. These students, many of whom face the dual challenge of developing basic literacy skills—including decoding, comprehending, and producing print—along with proficiency in English, represent a range of nationalities and cultural backgrounds. Although the majority of students come from Mexico and other Spanish-speaking countries, there are also students from Africa, India, the Philippines, China, Vietnam, and the Caribbean (Wrigley, Richer, Martinson, Kubo, & Strawn, 2003).

Adult basic education (ABE) and ESL programs, authorized by the Workforce Investment Act and also funded with state and local funds, are designed to assist students in their efforts to acquire literacy and language skills by providing instruction through local education agencies, community colleges, and community-based organizations. The content of instruction within ESL classes varies widely. It is often designed to assist students in their efforts to acquire literacy and language skills by providing a combination of oral language, competency-based work skills, and literacy instruction (Condelli, Wrigley, Yoon, Cronen, & Seburn, 2003). There is, however, little rigorous research that identifies effective instruction. A comprehensive review of published research studies on the effects of literacy interventions for ABE and adult ESL learners (Condelli & Wrigley, 2004) found that out of 17 adult education studies that used a rigorous methodology (i.e., quasi-experimental or randomized trials), only 3 included adult ESL learners (Diones, Spiegel, & Flugman, 1999; St. Pierre et al., 1995; St. Pierre et al., 2003). Furthermore, among the 3 studies that included adult ESL learners, only 1 presented outcomes for those learners, and that study experienced substantial methodological problems that limited the validity of the findings (e.g., a 40 percent overall attrition rate and different attrition rates in the intervention vs. control groups; Diones et al., 1999).

To help improve research-based knowledge of effective instruction for low-literate ESL learners, the National Center for Education Evaluation and Regional Assistance of ED's Institute of Education Sciences contracted with the American Institutes of Research (AIR) to conduct a Study of the Impact of a Reading Intervention for Low-Literate Adult ESL Learners. The intervention studied was the basal reader Sam and Pat, Volume I, published by Thomsonvi Heinle (2006). The study team consisted of AIR, Berkeley Policy Associates (BPA), the Lewin Group, Mathematica Policy Research, Inc., Educational Testing Service (ETS), and World Education.

The goal of this study was to test a promising approach to improving the literacy skills of low-literate adult ESL students under real-world conditions. In their review of the research on ESL instruction in related fields, including adult second language acquisition, reading and English as a foreign language instruction, Condelli & Wrigley (2004) concluded that instruction based on a systematic approach to literacy development was a promising intervention for low-literate adult ESL learners that would be valuable to study (Brown et al., 1996; Cheek & Lindsay, 1994: Chen & Graves, 1995; Carrell, 1985; Rich & Shepherd, 1993; Roberts, Cheek & Mumm, 1994). Specifically, the factors identified as defining a systematic approach to literacy development included: (1) a comprehensive instructional scope that includes direct instruction in phonics, fluency, vocabulary development and reading comprehension, (2) a strategic instruction sequence, (3) a consistent instructional format, (4) easy-to-follow lesson plans, and (5) strategies for differentiated instruction.

Sam and Pat was selected as the focus of the study because it offers an approach to literacy development that is systematic, direct, sequential, and multi-sensory. It also includes multiple opportunities for practice with feedback. Consistent with characteristics identified as promising by Condelli & Wrigley (2004), Sam and Pat provides opportunities for cooperative learning, real world tasks, and an explicit focus on reading. In addition, the text was developed for and had been used by the developers with students similar to the study population (literacy level ESL learners).

The impact study used an experimental design to test the effectiveness of Sam and Pat in improving the reading and English language skills of adults enrolled in 66 ESL literacy classes at 10 sites. The study addressed three key research questions:

  1. How effective is instruction based on the Sam and Pat textbook in improving the English reading and language skills of low-literate adult ESL learners compared to instruction normally provided in adult ESL literacy classes?
  2. Is Sam and Pat effective for certain subgroups of students (e.g., native Spanish speakers)?
  3. Is there a relationship between the amount of instruction in reading or English language skills and reading and English language outcomes?

This report describes the implementation of Sam and Pat at the study sites, compares the instruction and student attendance in Sam and Pat classes with that in the standard adult ESL classes, and examines the impact of Sam and Pat on reading and English language outcomes. In addition, the report examines the relationship between instruction, attendance, and student outcomes.

The study produced the following key results:

  • More reading instruction was observed in Sam and Pat classes, while more English language instruction was observed in control classes. The Sam and Pat classrooms spent more time on reading development instruction (66 percent of observed intervals in Sam and Pat classrooms compared to 19 percent in control classrooms), and the difference was statistically significant. Conversely, the control classrooms spent more time on English language acquisition instruction (68 percent of observed intervals in control classrooms compared to 27 percent in Sam and Pat classrooms), and this difference was also statistically significant.

  • Although students made gains in reading and English language skills, no differences in reading and English language outcomes were found between students in the Sam and Pat group and students in the control group. On average, students participating in the study made statistically significant gains in reading and English language skills over the course of the term (effect sizes of 0.23 to 0.40). However, there were no statistically significant impacts of Sam and Pat on the reading and English language outcomes measured for the overall sample.

  • There were no impacts of Sam and Pat on reading and English language outcomes for five of six subgroups examined. For students with relatively lower levels of literacy at the start of the study, there was some suggestive evidence of a positive impact on reading outcomes.2 Among students with lower levels of literacy at the beginning of the term, Sam and Pat group students scored higher on the Woodcock Johnson word attack (decoding) assessment than control group students (effect size = 0.16). Because this difference was not significant after adjusting for multiple comparisons, however, it is possible that the effect is due to chance alone.

Top

2 Lower literacy was defined as scoring at a Grade 2 equivalent or below on the Woodcock Johnson Letter-Word Identification and Word Attack subtests (raw scores of 31 and 9, respectively).