First-Year Implementation

During the first year of the project, the developers for each of the ERO programs provided three types of training and technical assistance to one teacher from each of the 34 participating schools who volunteered to teach the ERO classes: a five-day summer training institute in August 2005, booster training sessions during the 2005-2006 school year, and a minimum of two 1-day coaching visits during the 2005-2006 school year.

Each ERO teacher was responsible for teaching four sections of the ERO class. Each section accommodated between 10 and 15 students. Classes were designed to meet for a minimum of 225 minutes per week and were scheduled as a 45-minute class every day or as a 75- to 90-minute class that met every other day. The ERO classes began an average of six weeks after the start of the 2005-2006 school year, with the earliest programs starting three weeks into the school year and the latest programs starting 10 weeks into the school year. The late start was due to the fact that the process for identifying eligible students for the program could not begin until the start of the school year and required extensive effort on the part of school staff and the study team to help complete the baseline data collection process and gain consent from students and their parents.

The study team assessed the overall fidelity with which the ERO programs were implemented in each school during the first year of the project. In the context of this study, "fidelity" refers to the degree to which the observed operation of the ERO program in a given high school was aligned with the intended learning environments and instructional practices that were specified by the model developers. Measures of implementation fidelity were developed from 140 to 180 minutes of observation of each ERO classroom conducted in the second semester of the school year. Composite fidelity scores were calculated from numeric ratings (ranging from one to three) of classroom activities related to two overarching program dimensions: classroom learning environment and comprehension instruction. The implementation fidelity for each dimension was classified as well aligned, moderately aligned, or poorly aligned, based on the composite scores. Following is a summary of key findings.

• The implementation of the ERO programs in 24 of the 34 schools was classified as well aligned or moderately aligned with their program models on both the classroom learning environment and the comprehension instruction dimensions. This included 11 of the schools using Reading Apprenticeship Academic Literacy and 13 of the schools using Xtreme Reading.

The implementation of the ERO programs in 16 of the 34 schools was classified as well aligned on both program dimensions. Because the classroom learning environments and comprehension instruction activities were designed to be interdependent and mutually reinforcing, the implementation of ERO program in a given school was classified as well aligned only if both of these dimensions were rated as well aligned. According to the protocols used for the classroom observations, teacher behaviors and classroom activities in these schools were rated consistently as being well developed and reflective of the behaviors and activities specified by the developers.

The implementation of the ERO programs in eight of the 34 schools was classified as moderately aligned with the program model on at least one of the two key program dimensions and moderately or well aligned on the other dimension. In six of these schools, the classroom learning environment was classified as well aligned with the program model while the comprehension instruction was classified as moderately aligned. In the remaining two schools, both the classroom learning environment and the comprehension instruction were rated as being moderately aligned with their program models.

• The implementation of the ERO programs in 10 of the 34 schools was classified as poorly aligned with the program models on at least one of the two overarching program dimensions. This includes six of the schools using Reading Apprenticeship Academic Literacy and four of the schools using Xtreme Reading.

Overall implementation fidelity was judged to be poorly aligned with the program model if the composite rating for either the classroom learning environment dimension or the comprehension instruction dimension was rated as inadequate. Poorly aligned implementation for a given dimension meant that the classroom observers found that at least half of the classroom characteristics were not aligned with the behaviors and activities specified by the developers and described in the protocols. The ERO programs in these schools were the least representative of the activities and practices intended by the respective program developers and were found to have encountered serious implementation problems on at least one of the two key program dimensions during the first year of the study.

Top