Skip Navigation
The Enhanced Reading Opportunities Study: Findings from the Second Year of Implementation

NCES 2009-4036
November 2008

Second-Year Implementation

Each ERO teacher (one per school) was responsible for teaching four sections of the ERO class. Each section accommodated between 10 and 15 students. Classes were designed to meet for a minimum of 225 minutes per week and were scheduled as a 45-minute class every day or as a 75- to 90-minute class that met every other day.

  • Of the 34 teachers who participated in the second year of the study, 25 had taught the entire first year of the study, and two had taught a portion of the first year (having replaced a teacher midyear). Seven teachers were new to the ERO programs at the start of the second year.

During the second year of the project, the developers for each of the ERO programs provided three types of training and technical assistance to both new and returning ERO teachers: a three-day summer training institute in July or August 2006, booster training sessions during the 2006-2007 school year, and three 2-day coaching visits during the 2006-2007 school year. Prior to the summer institute, teachers new to the ERO programs also attended additional training sessions at which they were taught the central strategies of the program being implemented in their school.

The study team assessed the overall fidelity with which the ERO programs were implemented in each school during the second year of the project. In the context of this study, “fidelity” refers to the degree to which the observed operation of the ERO program in a given high school was aligned with the intended learning environment and instructional practices that were specified by the model’s developers. The analysis of implementation fidelity in the second year of the study is based on two field research visits to each of the 34 high schools — one during the first semester and one during the second semester of the 2006-2007 school year. The classroom observation protocols used in the site visits provided a structured process for observers to rate the characteristics of the ERO classroom learning environments and the use of ERO instructional strategies by teachers. The instrument included ratings for six characteristics (referred to as "constructs" from here forward) that are common to both programs, as well as ratings for seven program-specific constructs. For each construct, a category rating of 1 ("poorly aligned"), 2 ("moderately aligned"), or 3 ("well aligned") was given.

The analysis of the classroom observation ratings sought to capture implementation fidelity on two key overarching dimensions of both programs: the classroom learning environment and the teacher’s use of instructional strategies focused on reading comprehension. A composite measure of implementation fidelity was calculated for each of these two dimensions by averaging across the relevant characteristics in the observation protocol. A composite rating of 2.0 or higher indicates that the school's ERO program was well aligned with the developers' implementation specifications; a rating of 1.5 to 1.9 means that the program was moderately aligned; and a rating of 1.0 to 1.4 means that it was poorly aligned. Following is a summary of key findings.

  • At the spring site visit, implementation fidelity in 26 of the 34 schools was classified as well aligned on both program dimensions. In seven schools, implementation was classified as moderately aligned with the program model on at least one of the two key program dimensions and as moderately or well aligned on the other dimension. In one school, implementation was deemed to be poorly aligned with the program models.

The overall implementation of the ERO program in a given school was classified as well aligned if both the classroom environment and the comprehension instruction dimension were rated as being well aligned. According to the protocols used for the classroom observations, teacher behaviors and classroom activities in these schools were consistently rated as being well developed and reflective of the behaviors and activities specified by the developers. At the fall site visit, the implementation of the ERO programs in 20 of the 34 schools was classified as well aligned on both program dimensions, and, at the spring site visit, 26 schools had attained this benchmark. Because implementation fidelity in the majority of the study schools was deemed to be well aligned to the models, the study team also examined the number of schools whose implementation of the programs was "very well aligned" to developers' specifications (defined here as a composite score of 2.5 or higher on both program dimensions). At the spring site visit, implementation in 13 schools could be classified as such.

Conversely, a school’s overall implementation fidelity was judged to be poorly aligned with the program model if the composite rating for either the classroom learning environment dimension or the comprehension instruction dimension was rated as poorly aligned. The ERO programs in these schools were not representative of the activities and practices intended by the respective program developers and were found to have encountered serious implementation problems on at least one of the two key program dimensions during the second year of the study.7 At the fall site visit, implementation of the ERO programs in three of the 34 schools was classified as poorly aligned with the program models on at least one of the two program dimensions. At the spring site visit, implementation at one school was considered to be poorly aligned with the program models.8

  • The number of schools considered to be well aligned with the program developers' specifications for implementation fidelity was greater in the second year of the study than in the first year (26 schools in the second year, compared with 16 schools in the first year).

At the spring site visit in the second year of the study, the ERO programs in 33 of the 34 schools reached an overall level of implementation fidelity that was at least moderately aligned to the program models (of these, 26 were considered to be well aligned). This is an improvement over the first year of the study, when 24 of the 34 schools had reached a moderate level of alignment at the spring site visit (of these, 16 schools were deemed to be well aligned). Also, during the spring site visit of the second year, only one school's implementation of the program was poorly aligned to the developers' specifications. This is lower than what was found during the first-year spring site visit, when 10 schools were ranked as poorly aligned on at least one of the two key program dimensions.

Top


7 In particular, poorly aligned implementation for a given dimension means that the classroom observers found that at least half of the classroom characteristics were not aligned with the behaviors and activities specified by the developers and described in the protocols.
8 In the second year of the study, implementation-fidelity ratings were similar for the 25 schools where the ERO teacher taught two full years of the program and for the nine schools where the ERO teacher had replaced another teacher at some point during the study (an average rating of 2.5 for returning teachers and 2.4 for replacement teachers, out of a maximum of score 3).