Each ERO teacher (one per school) was responsible for teaching four sections of the ERO class. Each section accommodated between 10 and 15 students. Classes were designed to meet for a minimum of 225 minutes per week and were scheduled as a 45-minute class every day or as a 75- to 90-minute class that met every other day.
During the second year of the project, the developers for each of the ERO programs provided three types of training and technical assistance to both new and returning ERO teachers: a three-day summer training institute in July or August 2006, booster training sessions during the 2006-2007 school year, and three 2-day coaching visits during the 2006-2007 school year. Prior to the summer institute, teachers new to the ERO programs also attended additional training sessions at which they were taught the central strategies of the program being implemented in their school.
The study team assessed the overall fidelity with which the ERO programs were implemented in each school during the second year of the project. In the context of this study, “fidelity” refers to the degree to which the observed operation of the ERO program in a given high school was aligned with the intended learning environment and instructional practices that were specified by the model’s developers. The analysis of implementation fidelity in the second year of the study is based on two field research visits to each of the 34 high schools — one during the first semester and one during the second semester of the 2006-2007 school year. The classroom observation protocols used in the site visits provided a structured process for observers to rate the characteristics of the ERO classroom learning environments and the use of ERO instructional strategies by teachers. The instrument included ratings for six characteristics (referred to as "constructs" from here forward) that are common to both programs, as well as ratings for seven program-specific constructs. For each construct, a category rating of 1 ("poorly aligned"), 2 ("moderately aligned"), or 3 ("well aligned") was given.
The analysis of the classroom observation ratings sought to capture implementation fidelity on two key overarching dimensions of both programs: the classroom learning environment and the teacher’s use of instructional strategies focused on reading comprehension. A composite measure of implementation fidelity was calculated for each of these two dimensions by averaging across the relevant characteristics in the observation protocol. A composite rating of 2.0 or higher indicates that the school's ERO program was well aligned with the developers' implementation specifications; a rating of 1.5 to 1.9 means that the program was moderately aligned; and a rating of 1.0 to 1.4 means that it was poorly aligned. Following is a summary of key findings.
The overall implementation of the ERO program in a given school was classified as well aligned if both the classroom environment and the comprehension instruction dimension were rated as being well aligned. According to the protocols used for the classroom observations, teacher behaviors and classroom activities in these schools were consistently rated as being well developed and reflective of the behaviors and activities specified by the developers. At the fall site visit, the implementation of the ERO programs in 20 of the 34 schools was classified as well aligned on both program dimensions, and, at the spring site visit, 26 schools had attained this benchmark. Because implementation fidelity in the majority of the study schools was deemed to be well aligned to the models, the study team also examined the number of schools whose implementation of the programs was "very well aligned" to developers' specifications (defined here as a composite score of 2.5 or higher on both program dimensions). At the spring site visit, implementation in 13 schools could be classified as such.
Conversely, a school’s overall implementation fidelity was judged to be poorly aligned with the program model if the composite rating for either the classroom learning environment dimension or the comprehension instruction dimension was rated as poorly aligned. The ERO programs in these schools were not representative of the activities and practices intended by the respective program developers and were found to have encountered serious implementation problems on at least one of the two key program dimensions during the second year of the study.7 At the fall site visit, implementation of the ERO programs in three of the 34 schools was classified as poorly aligned with the program models on at least one of the two program dimensions. At the spring site visit, implementation at one school was considered to be poorly aligned with the program models.8
At the spring site visit in the second year of the study, the ERO programs in 33 of the 34 schools reached an overall level of implementation fidelity that was at least moderately aligned to the program models (of these, 26 were considered to be well aligned). This is an improvement over the first year of the study, when 24 of the 34 schools had reached a moderate level of alignment at the spring site visit (of these, 16 schools were deemed to be well aligned). Also, during the spring site visit of the second year, only one school's implementation of the program was poorly aligned to the developers' specifications. This is lower than what was found during the first-year spring site visit, when 10 schools were ranked as poorly aligned on at least one of the two key program dimensions.