Effectiveness of a Supplemental Early Reading Intervention Scaled Up in Multiple Schools

Findings

Reading Comprehension outcomes—Statistically significant positive effect found for the domain

Outcome measure	Comparison	Period	Sample	Intervention mean	Comparison mean	Significant?	Improvement     index	Evidence tier
Woodcock Johnson - Passage Comprehension	Responsive reading instruction—Denton et al. (2010) vs. Business as usual	25 Weeks	Full sample; 422 students	447.26	447.34	Yes	20

Word reading  outcomes—Statistically significant positive effect found for the domain

Outcome measure	Comparison	Period	Sample	Intervention mean	Comparison mean	Significant?	Improvement     index	Evidence tier
Test of Word Reading Efficiency-Phonemic Decoding Efficiency	Responsive reading instruction—Denton et al. (2010) vs. Business as usual	25 Weeks	Full sample; 422 students	8.39	8.14	Yes	16

Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.

Study Details

Setting

The intervention was expected to take place for 40 minutes per day, five days per week, for 25 weeks, in small groups of approximately three to four students. However, researchers attempted to play a minimal role, so actual implementation varied across schools and teachers.

Study sample

STUDENTS: 507 students were randomized to study conditions, 222 to RRI and 285 to TSP. After attrition, 182 RRI students and 240 TSP students had complete data and were included in the analysis. To participate in the study, students had to meet all the following criteria at the beginning of the first grade: 1. Failed the letter sounds, blending phonemes, and word reading screening instruments from the Texas Primary Reading Inventory. 2. Scored 8 or less (in the first year of the study) or 3 or less (in the second year) on the Woodcock-Johnson III Letter-Word Identification subtest. 3. Had an oral reading fluency rate below 8 correct words per minute (in the first year of the study) or 5 correct words per minute (in the second year) on an unspecified first-grade oral reading fluency passage. 4. Did not have a severe disability. 5. Received primary reading instruction in English. TEACHERS: The principal at each school selected teachers to provide the intervention. Thirty-one were full-time reading teachers, four were classroom teachers, two were librarians, one was a special educator, one was a physical education coach, and one was an educational assistant. All but one held elementary education teaching credentials, and several were certified in other areas as well. All but one of the teachers implementing the intervention had an elementary teaching credential, and they had a mean of 17 years of teaching experience. SCHOOLS: The study took place in 31 elementary schools from 16 school districts in a southwestern state (likely Texas) over a two-year period with two cohorts of first-grade students. On average, 53 percent of students in the schools were economically disadvantaged. Forty-one percent of students in the final sample were Caucasian or Asian, 15 percent were African American, and 43 percent were Hispanic. Six percent received ESL and three percent received special education services.

Intervention Group

"Number of lessons, frequency, duration: Responsive Reading Instruction (RRI) is a stand-alone intervention comprising five lesson components designed to be implemented in a 40-minute lesson. The intervention was expected to take place for 40 minutes per day, five days per week, for 25 weeks, in small groups of approximately three to four students. However, researchers attempted to play a minimal role, so actual implementation varied across schools and teachers. Formative assessment, materials, and script: RRI teachers choose from a selection of activities for each of the lesson components based on diagnostic assessments of students' strength and needs. Teachers had access to a manual for the intervention that gave step-by-step instructions to implement each activity, as well as a CD with video clips for many activities. The teacher's manual also included instructions for using student assessment data to plan lessons. The authors do not describe this formative assessment process in detail. Components of Intervention: As implemented in the study, the intervention had four components: Word Work, Print Concepts/Fluency and Assessment, Supported Reading, and Supported Writing. 1. Word Work (10 minutes): Teachers provide explicit instruction and practice in phonemic awareness, letter-sound correspondences, sight word recognition, phonemic decoding, and spelling. 2. Print Concepts/Fluency and Assessment (10 minutes): Students received explicit instruction in basic print concepts during the first lesson. In later lessons, teacher modeling, repeated oral reading with feedback, and partner reading help to develop fluent reading. Teachers assess each student at least once per week, and were given a choice of diagnostic curriculum-based assessment tools to use as appropriate. 3. Supported Reading (10 minutes): Teachers provide feedback as students read increasingly difficult text. Teachers instruct students to use letter-sound relationships as the primary strategy for identifying unknown words and were discouraged from using pictures and context to identify words. Daily text reading included comprehension instruction involving a brief introduction, purpose-setting, and guiding question supplied by the teacher before reading. 4. Supported Writing (10 minutes): One student in a group wrote a complete sentence in response to the teacher's question. All students recorded the same sentence, applying their knowledge of orthographic patterns and phonemic analysis to spell words. Students learned to edit their writing in later lessons. There is no home component to the intervention. Intervention Fidelity: Researchers assessed fidelity of implementation. There were a total of three fidelity observations per teacher per year, using a fidelity measure closely aligned with the intervention. The mean overall fidelity rating for all teachers across all observations was 80.14 percent (SD = 8.43 percent). Mean ratings for adherence to RRI procedures, quality of implementation, and keeping students on task averaged over 2 points (out of three). Students in cohort 2 received an average of 62.16 hours of instruction, compared to a target of approximately 75 hours (40 minutes daily for 25 weeks, excluding holidays)."

Comparison Group

Comparison students received "typical school practice," (TSP) which varied substantially across schools. 43% of comparison students received some alternative reading intervention, which was generally less structured and sometimes shorter than RRI. Instruction ranged from silent book reading to one-on-one skills-based instruction. Most students received small group skills-based instruction. Some of these alternative interventions were led by teaching assistants, but a complete description of staff implementing the comparison condition is not included.

Support for implementation

"Program developers and one experienced RRI teacher provided new teachers approximately 18 hours of professional development over three days at the beginning of the school year. The professional development focused on the research basis for the program, RRI instructional procedures, and the use of assessments results in lesson planning. Professional development featured videotape modeling and mock teaching sessions. In year two, continuing teachers received about six hours of professional development at the beginning of the school year. In each of the two years, teachers participated in 12 hours of follow-up professional development over two days. In addition to this professional development, which reflects the kind of support a school might receive from the program developer during the first years of implementing a new program, two experienced RRI teachers provided coaching to the intervention teachers. As part of a different study, intervention teachers were randomized to receive traditional on-site coaching, technology-based coaching, or on-demand coaching."