Evaluation of the Impact of Literacy Instruction on Adult ESL (English as a Second Language) Learners

Contractors: AIR, The Lewin Group, Educational Testing Service (ETS), Mathematica, and Berkeley Policy Associates (BPA)

Background/Research Questions:

The Adult Education and Family Literacy Act (AEFLA) authorizes grants to states to fund local programs of adult education and literacy services. Funded at $554 million in FY 2008, the programs provide instruction in reading, numeracy, GED preparation, and English literacy. The AEFLA also authorizes National Leadership Activities. Such activities, which are designed to enhance the quality of adult education and literacy programs nationwide, include the identification of the most successful methods and techniques for addressing the education needs of adults.

Many learners in federally-funded adult education programs face the dual challenge of developing proficiency in the English language as well as basic literacy skills. However, little research exists to help guide instruction for these learners. The current evaluation examines the effectiveness of Thomson-Heinle's Sam & Pat in improving the English reading and speaking skills of adults in ESL programs who have low levels of literacy in their native language. The study will address the following key research questions:

• How effective is instruction based on Sam and Pat in improving the English reading and speaking skills of low-literacy adult ESL learners?
• Is Sam and Pat more effective for certain groups of students (e.g. native Spanish speakers)?
• Do teachers who deliver Sam and Pat instruction engage in more literacy instruction than teachers who deliver their program's typical adult ESL curriculum (i.e., is there a "service contrast"?), and are impacts on learners' English reading and speaking skills greater where the service contrast is greater?

Design:

Ten sites across four states were recruited for the study. Across these sites, approximately 30 teachers were randomly assigned to implement Sam & Pat within their classrooms or to deliver their site's usual ESL curriculum. Approximately 1,800 adult ESL learners (180 learners per site) from two cohorts will participate in the evaluation. Learners will be randomly assigned to a classroom in which Sam and Pat is implemented or a classroom in which the site's usual ESL curriculum is delivered. The result of the random assignment process will be a total of approximately 90 learners from each site receiving Sam and Pat instruction and approximately 90 learners from each site receiving their site's typical instruction. The study's data collection includes teacher background surveys, learner background interviews, classroom observations, classroom attendance logs, and pre- and post-tests of learners' English reading and speaking abilities.

Duration: 5 1/2 years (September, 2004 – March, 2010)

Current Status: The first of two cohorts of learners have been randomly assigned and received instruction based upon Sam and Pat or their site's typical adult ESL curriculum in fall 2008. The second cohort was randomly assigned and is receiving instruction in the spring 2009 semester.

Key Findings

Younger students benefited more from the interventions than older students.

• For 3rd grade students the four interventions combined had impacts on phonemic decoding, word reading accuracy and fluency, and reading comprehension. The reading gap with the average population was narrowed for third graders participating in the interventions.
• For 5th grade students, the four interventions combined improved phonemic decoding on one measure, but led to a reduction in oral reading fluency. The three word-level interventions had similar impacts to the four interventions combined, but did not show an impact on reading comprehension.