The ERO Evaluation
The supplemental literacy programs were implemented in 34 high schools from 10 school districts across the country. The districts were selected through a special grant competition organized by the U.S. Department of Education’s Office of Vocational and Adult Education (OVAE). Experienced, full-time English/language arts or social studies teachers were selfselected and approved by ED, the districts, and the schools to teach the programs for a period of two years.
The ERO evaluation utilizes a two-level random assignment research design. First, within each district, eligible high schools were randomly assigned prior to the first year of program implementation to use one of the two supplemental literacy programs: 17 of the high schools were assigned to use RAAL, and 17 schools were selected to use Xtreme Reading. Each school implemented the same program in two school years: 2005-2006 and 2006-2007. In the second stage of the study design, eligible students within each of the participating high schools and in each year of the study were randomly assigned either to enroll in the ERO class (the "ERO group") or to take one of their school’s regularly offered elective classes (the "non-ERO group").
During the second year of the study, the participating high schools identified 2,679 ninth-grade students with baseline test scores indicating that they were reading two to five years below grade level (an average of 79 students per school). Approximately 57 percent of these students were randomly assigned to enroll in the ERO class, and the remaining students make up the study’s control group and were enrolled in or continued in a regularly scheduled elective class.
Evaluation data were collected with the Group Reading Assessment and Diagnostic Examination (GRADE) reading comprehension and vocabulary tests and a survey.5 Both instruments were administered to students at two points in time: a baseline assessment and survey in the spring of eighth grade and a follow-up assessment and survey at the end of ninth grade.6 Follow-up test scores are available for 2,171 (81 percent) of the students in the study sample. To learn about the fidelity of program implementation, the study also includes observations of the supplemental literacy classes during the first and second semester of the school year.
5 American Guidance Service, Group Reading Assessment and Diagnostic Evaluation: Teacher’s Scoring and Interpretive Manual, Level H; and Technical Manual (Circle Pines, MN: American Guidance Service, 2001a, 2001b).
6 In four of the 34 participating schools, baseline testing occurred in the fall of ninth grade rather than the spring of eighth grade.