Co-Principal Investigator: Smolkowski, Keith

Purpose: This research will investigate the efficacy of Reading Mastery on first graders' reading achievement, reading disability (RD) identification, and risk of RD. Reading Mastery has been used in schools since the 1970s and the 2008 Signature Edition is still widely used today. Despite the many publications that reference this comprehensive reading intervention, its efficacy has not yet been rigorously tested. This study seeks to fill this gap in research by examining the intervention's efficacy, including whether student response depends on student entry-level reading, oral language skills, and cognitive profiles and teacher experiences with explicit instruction. Secondary aims include an investigation of the role played by effective instructional practices, fidelity, and dosage as well as a complete cost analysis and cost-effectiveness evaluation.

Project Activities: The efficacy of Reading Mastery on student reading outcomes will be evaluated using a randomized controlled trial. In addition, the research team will evaluate differential response to intervention to determine which students benefit most and in which settings, as well as the cost and cost-effectiveness of Reading Mastery. 

Products: The primary product of this project will be evidence of the efficacy of Reading Mastery and the types of students or settings associated with the greatest impacts. The project will also result in an understanding of the costs and cost-effectiveness of Reading Mastery, knowledge of other aspects of implementation, and how the practices relate to student outcomes. Dissemination activities will include peer-reviewed publications, presentations, a publicly available final dataset, and additional products that reach stakeholders such as practitioners and policymakers.

Structured Abstract

Setting: The study will take place in elementary schools in a large, diverse school district in Maryland.

Sample: The project will enroll up to 120 small-group teachers (SGTs, who are classroom teachers or teachers who provide supplemental instruction) and 600 first grade students identified with or at risk for RD and in need of secondary reading support, including English learners and students with other identified disabilities who meet risk criteria for RD. The study will exclude students who require additional, specialized services during small-group instruction.

Intervention: Reading Mastery is a comprehensive reading program used in thousands of schools across the United States. It emphasizes instruction from decoding to comprehension and critical thinking skills and helps students acquire the background knowledge necessary to succeed at these reading task. Program materials include fully scripted lessons to guide SGTs through carefully constructed instructional steps: (a) modeling new content; (b) providing guided practice; (c) offering independent, individualized practice; and (d) applying new knowledge to higher-order skills. Reading Mastery will be used daily in small groups for supplemental reading instruction in this project.

Research Design and Methods: This study will test the efficacy of Reading Mastery using a randomized controlled trial. After randomly assigning SGTs to teach either Reading Mastery or business-as-usual instruction, researchers will randomly assign students to SGTs. SGTs in both conditions will teach small groups of approximately five students for the same amount of time, with each SGT teaching in only one condition. Researchers will collect data on reading, cognitive abilities, and demographics from students. They will also gather data about the reading programs including dosage, costs, practices, strategies, and focus of instruction (such as decoding, fluency, and comprehension) from teachers in both the small-group and whole classroom contexts. Investigators will follow participating students from grade 1 to grade 2 to examine both immediate and follow-up intervention effects. Researchers will also collect data on numerous aspects of instruction and program costs in both conditions.

Control Condition: The control condition will consist of business-as-usual instruction using the instructional programs chosen by the district throughout the study.

Key Measures: The research plan includes both proximal and distal measures of student reading, such as decoding, sight word reading, sentence reading, reading fluency, and word and passage comprehension using the Formative Assessment System for Teachers and Woodcock Reading Mastery Test. In addition, CBMreading, a measure of oral reading fluency, will be used for reading progress monitoring. Instructional quality and implementation fidelity will be measured by data collected with the Survey of Quality of and Adherence to Explicit Instruction, Classroom Observations of Student-Teacher Interactions, and small-group teaching logs. The project will also capture the schools' instructional decisions about assignment to tier of service or special education in grades 1 and 2. As potential moderators, researchers will collect data on student cognitive functioning using the Woodcock-Johnson IV Tests of Cognitive Abilities, teacher report of student classroom behavior using the Elementary Social Behavior Assessment, and school data on student demographic characteristics (gender, ethnicity, first language, EL status, English proficiency, and free or reduced-price lunch status.)

Data Analytic Strategy: To address the nested structure of students clustered within small groups, researchers will use mixed-model analysis of covariance for testing the overall student outcomes and a nested random coefficients analysis with repeated measurements to model growth. Researchers will use growth modeling for progress monitoring data to determine whether there is differential impact (moderation) based on student or teacher characteristics. They will also use multilevel logistic regression to examine whether RD determination differs as a function of receiving Reading Mastery.

Cost Analysis: The researchers will identify, measure, and value the resources needed to implement Reading Mastery, including new expenditures and reallocations of existing resources, using the ingredients method. They will then use these data to estimate cost-effectiveness ratios. They will also report local costs for local decision makers.