|
|
Adolescent Literacy
Research
One hundred one studies reviewed by the WWC investigated the effects of READ 180 on adolescent learners. Seven studies (Haslam, White, & Klinge, 2006; Interactive Inc., 2002; Lang, Torgesen, Petscher, Vogel, Chanter, & Lefsky, 2008; Scholastic Research, 2008; White, Haslam, & Hewes, 2006; White, Williams, & Haslem, 2006; Woods, 2007), one of which is a randomized controlled trial and six of which are quasi-experimental designs, meet WWC evidence standards with reservations.5 The remaining 94 studies do not meet either WWC evidence standards or eligibility screens.
Meets evidence standards with reservations
• Haslam, White, and Klinge (2006) conducted a quasi-experiment that examined the effects of READ 180 on struggling readers in grades 7 and 8 in the Austin Independent School District of Texas. Intervention students were matched with comparison-group students using a one-to-one propensity score matching method. The WWC based its effectiveness ratings on findings from comparisons of the 307 students that received READ 180 and the 307 comparison-group students that received the standard district curriculum. The study reported student outcomes after one year of program implementation.
• Interactive Inc. (2002) conducted a quasi-experiment that examined the effects of READ 180 on students in grades 6, 7, and 8 in three urban school districts in Texas and Ohio.6 The WWC based its effectiveness ratings on findings from comparisons of the 52 control students and 119 students enrolled in READ 180 in Columbus, the 142 control students and 101 students enrolled in READ 180 in Dallas, and the 36 control students and 59 students enrolled in READ 180 in Houston.7 The study reported student outcomes after one year of program implementation.
• Lang et al. (2008) conducted a randomized controlled trial of 1,265 struggling readers in seven high schools in Florida. Ninth-grade students who scored in the high-risk or moderate-risk categories on the Florida Comprehensive Assessment Test (FCAT) were randomly assigned to one of four groups (Appendix A1.3 provides more details about these groups). The WWC based its effectiveness ratings on findings from comparisons of the 100 high-risk students who received READ 180 and 90 high-risk students who were in the school-designed “business as usual” control group, as well as comparisons between 207 moderate-risk students who received READ 180 and 202 moderate-risk students who were in the control group. The inability to determine whether differential attrition occurred and the use of multiple imputation of missing data led to the study’s rating of “meets standards with reservations.” The study reported student outcomes after one year of program implementation.
• Scholastic Research (2008) conducted a quasi-experiment that examined the effects of READ 180 on students in grades 6, 7, and 9 in California. READ 180 students who scored at the below-basic or basic performance level on the prior-year state reading test were matched to comparison-group students on the basis of reading scores and demographic characteristics. The WWC based its effectiveness ratings on findings from comparisons of the 285 students that received READ 180 and the 285 comparison-group students that received either the Holt Literature and Language Arts curriculum (grades 6 and 7) or the Prentice Hall Literature curriculum (grade 9). The study reported student outcomes after one year of program implementation.
• White, Haslam, and Hewes (2006) conducted a quasi-experiment that examined the effects of READ 180 on three cohorts of students in 12 schools in Arizona. READ 180 students in grade 9 who were reading one or more grade levels below their assigned grade level were matched to comparison-group students with similar reading levels and demographic characteristics. The comparison-group students received the standard district curriculum. The WWC based its effectiveness ratings on findings from three cohorts of students. Cohort 1 was formed in the 2003–04 school year and consisted of 826 students in grade 9 who received READ 180 and 826 comparison-group students in grade 9 who did not receive READ 180. Cohort 2 was formed in the 2004–05 school year and consisted of 815 students in grade 9 who received READ 180 and 815 comparison-group students in grade 9 who did not receive READ 180. Cohort 3 was formed in the 2005-06 school year and consisted of 1,029 students in grade 9 who received READ 180 and 1,029 comparison students who did not receive READ 180. The study reported student outcomes after the first year of program implementation.
• White, Williams, and Haslem (2005) conducted a quasi-experiment that examined the effects of READ 180 on students in grades 4–8 in 16 schools in New York City. READ 180 students’ test outcomes were compared to outcomes of their peers attending the same schools who did not receive READ 180 but instead received the standard reading instruction in the 2001–02 school year. Comparisons were made between students with the same proficiency levels (1, 2, and 3)8 within each grade.9 The WWC based its effectiveness ratings on findings from comparisons between 362 students in the treatment group and 2,528 students in the control group, across grades 4, 6, and 8.10 The study reported student outcomes after one year of program implementation.
• Woods (2007) conducted a quasi-experiment that examined the effects of READ 180 on three annual cohorts of students in an urban middle school in Virginia. Based on pretest scores and teacher recommendations, the school’s guidance counselor assigned students in grades 6–8 who needed additional literacy support to either the READ 180 treatment group or the comparison group that would participate in the school’s traditional reading-remediation program. The WWC based its effectiveness ratings on findings from comparisons of 58 middle-school students who received READ 180 during the 2003–04 school year and 58 matched students who received the school’s traditional reading-remediation program.11 The study reported student outcomes after the first year of program implementation.
Extent of evidence
The WWC categorizes the extent of evidence in each domain as small or medium to large (see the WWC Procedures and Standards Handbook, Appendix G). The extent of evidence takes into account the number of studies and the total sample size across the studies that meet WWC evidence standards with or without reservations.12
The WWC considers the extent of evidence for READ 180 for adolescent learners to be medium to large for comprehension and medium to large for general literacy achievement. No studies that meet WWC evidence standards with or without reservations examined the effectiveness of READ 180 for adolescent learners in the alphabetics or reading fluency domains.
5 During the period covered by the studies in this report, two versions of READ 180 (version 1.6 and the Enterprise Edition) were available. The WWC was unable to obtain information about which version of READ 180 was used in the studies included in this report.
6 The study was originally designed as a randomized controlled trial, but none of the districts ultimately followed through with the research design.
7 The intervention and comparison groups in the grade 7 sample in Houston were not shown to be equivalent at baseline, so they were excluded from the review.
8 There were only two treatment students in proficiency level 4 across grades 4–8; therefore, proficiency level 4 was excluded from the review.
9 The sample sizes, means, and standard deviations that were used to calculate the intervention group values by grade and proficiency level were not reported in White, Williams, and Haslem (2005), but were provided to the WWC by the author.
10 The intervention and comparison groups at grade 5 (proficiency levels 1, 2, and 3) were not shown to be equivalent at baseline and were excluded from the review. In addition, there were no treatment students in the grade 7 analysis sample, so grade 7 students were excluded from the review.
11 The 2004–05 and 2005–06 student cohorts, though included in the study, do not meet WWC evidence standards because the measures of effect cannot be attributed solely to the intervention—there was only one READ 180 teacher in the treatment condition in both cohorts. This information was not reported in Woods (2007), but was provided to the WWC by the author.
12 The extent of evidence categorization was developed to tell readers how much evidence was used to determine the intervention rating, focusing on the number and size of studies. Additional factors associated with a related concept—external validity, such as the students’ demographics and the types of settings in which studies took place—are not taken into account for the categorization. Information about how the extent of evidence rating was determined for READ 180 is in Appendix A6.