An Evaluation of the Impact of Supplemental Literacy Interventions in Freshman Academies

Contractors: MDRC, AIR, and Survey Research Management

Background/Research Questions:

Since 2000, the Department, under the Smaller Learning Communities (SLC) Program (funded at $80,107,636 in FY 2008), authorized by the Elementary and Secondary Education Act, has awarded discretionary grants to school districts to support the implementation of SLCs and activities to improve student academic achievement in large public high schools. To support both a demonstration and a rigorous evaluation of supplemental literacy interventions for striving ninth grade readers, the Office of Elementary and Secondary Education awarded SLC grants of $1.25 million per school in FY 2005 to 10 LEAs and their 34 high schools.

According to the National Assessment of Educational Progress, over 70 percent of students nationally arrive in high school with reading skills that are below proficient. A range of reading programs intended to substantially boost adolescent literacy skills are increasingly in demand, particularly in high schools that are implementing Smaller Learning Communities (SLCs) for ninth graders. This study is testing two supplemental year-long literacy programs selected by a national panel of experts from among those submitted by 17 curriculum developers in a competition conducted under this evaluation. The evaluation focuses on the following questions:

• What are the short-term impacts of these interventions on ninth-grade students reading skills and behaviors?
• What are the longer-term impacts on academic outcomes in the 10th and 11th grades, such as achievement on high-stakes standards based assessments, performance in academic courses, and progress toward graduation?

Design:

NCEE awarded a contract to evaluate the impact of two supplemental literacy programs for striving ninth grade readers in 10 districts and 34 high schools that received awards through the U.S. Department of Education's SLC grant program in 2005. Within each district, participating high schools were randomly assigned so that half implementing each of the two interventions — Reading Apprenticeship [WestEd] and Xtreme Reading [University of Kansas] — within their freshman academies. At each high school, two cohorts of eligible students, assessed as being two years or more behind grade level in reading comprehension in the spring of their 8th grade year, were randomly assigned to the supplemental literacy program offered at their school in 9th grade or to an elective. Approximately five thousand students (about 2,500 9th graders in 2005–06 and an additional 2,500 9th graders in 2006–07) are involved in the study.

Duration: 5 years 2 months (September 30, 2004 – November 30, 2009)

Current Status: The first report was released in January 2008, and the second report in November 2008 (http://ies.ed.gov/ncee/pubs/20094036/). The final report is expected in late 2009.

Key Findings:

• Across the 34 study high schools, on average, the two supplemental literacy programs improved reading comprehension test scores for both the first cohort and second cohort of 9th graders participating in the study by about 2 months of instruction. Still, about three-fourths of students enrolled in the programs were still reading at two or more years below grade level at the end of ninth grade.
• When impacts for the programs were analyzed separately, there were no statistically significant impacts on comprehension, except for Reading Apprenticeship which had a statistically significant impact on reading comprehension equivalent to 2 months of instruction among 2nd cohort students.