Springfield-Chicopee School Districts Striving Readers (SR) Program. Final Report Years 1-5: Evaluation of Implementation and Impact [Xtreme Reading vs. business as usual]
Sprague, Kimberley; Zaller, Colleen; Kite, Anita; Hussar, Karen (2012). Education Alliance at Brown University. Retrieved from: https://eric.ed.gov/?id=ED600926
-
examining448Students, grade9
Practice Guide
Review Details
Reviewed: November 2021
- Practice Guide (findings for Xtreme Reading)
- Randomized Controlled Trial
- Meets WWC standards without reservations because it is a randomized controlled trial with low attrition.
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Findings
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|
|---|---|---|---|---|---|---|---|---|---|
|
Stanford Diagnostic Reading Test, Edition 4 (SDRT-4) |
Xtreme Reading vs. Business as usual |
0 Months |
Full sample- Xtreme Reading intervention vs. business as usual comparison;
|
21.95 |
21.75 |
No |
-- | ||
| Show Supplemental Findings | |||||||||
|
Stanford Diagnostic Reading Test, Edition 4 (SDRT-4) |
Xtreme Reading vs. READ 180® |
0 Months |
Full sample- Xtreme Reading intervention v. READ 180 comparison;
|
21.95 |
34.20 |
No |
-- | ||
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Sample Characteristics
Characteristics of study sample as reported by study author.
-
4% English language learners -
Female: 55%
Male: 45% -
Urban
-
- B
- A
- C
- D
- E
- F
- G
- I
- H
- J
- K
- L
- P
- M
- N
- O
- Q
- R
- S
- V
- U
- T
- W
- X
- Z
- Y
- a
- h
- i
- b
- d
- e
- f
- c
- g
- j
- k
- l
- m
- n
- o
- p
- q
- r
- s
- t
- u
- v
- x
- w
- y
Massachusetts
-
Race Other or unknown 75% White 25%
Study Details
Setting
The study was conducted in two school districts, Chicopee and Springfield, in western Massachusetts. A total of five schools implemented the intervention, with three schools implementing in the Springfield School District and two schools implementing in the Chicopee School District.
Study sample
In each of the 5 study years, students in five study schools were screened prior to random assignment. Students at least two—but less than four—grade levels behind in reading performance were selected to participate. Students were excluded from the sample if (a) they had an Individualized Education Plan (IEP) that specified reading supports not compatible with Xtreme Reading, (b) they lacked sufficient English language proficiency, (c) their parents opted out of the study, (d) they were enrolled in an off-campus evening school, (e) they were deemed not to be a “struggling reader” based on grade history and MCAS scores, or (f) they could not be located in school enrollment records. In the contrast of interest in this review (namely Xtreme Reading intervention group vs. business-as-usual comparison group), there were 679 9th grade students. Among the intervention group, 22% of students were White, 57% were female, 24% were special education students, and 4% were English learners. A majority of students (76%) were eligible for free or reduced-price lunch. Among the comparison group, 21% of students were White, 53% were female, 19% were special education students, and 4% were English learners. A majority of students (74%) were eligible for free or reduced-price lunch.
Intervention Group
The study examined the effectiveness of a reading intervention for students struggling with reading. Xtreme Reading is an intervention for struggling readers that uses a meta-cognitive approach and focuses on explicit strategy instruction. The intervention was implemented across 5 years as an "add-on" or supplemental intervention. The approach to instruction involves intensive lessons in which students have numerous opportunities to practice targeted learning strategies. Xtreme Reading relies on whole-group and teacher-directed instruction, teacher modeling, guided activities, and paired and independent work. Class size was set at 15 students. Dosage was set to 45 minutes per day, sometimes within a 90-minute block of English language arts (ELA) courses. Some barriers prevented students from receiving the full 45 minutes of the Xtreme Reading instruction in some cases, including (a) a lack of buy-in or satisfaction from teachers with the program, (b) issues with classroom management and student engagement with the material, and (c) attendance issues and other school requirements, like testing.
Comparison Group
Students in the primary comparison condition received the standard ELA course (as did students in the intervention condition), as well as supplemental services ordinarily available to all students. In practice, comparison group students had minimal access to supplemental services.
Support for implementation
For Xtreme Reading, developers train teachers on "learning strategies" for students. The professional development model includes initial training, ongoing in-class mentoring by providers, and workshops on specific routines. Teachers received 3 days of summer training in year 1, which was shortened to 2 days in year 2. Administrators held a 1-day summer meeting to support teachers in year 1 only. Developers provided in-class mentoring monthly during the school year (8 times in year 1 and 9 times in year 2). Teachers also attended 4 full days of additional workshops in year 1 and 5 days in year 2. Teachers received a reading library, lists of supplements, a curriculum including teacher/student notebooks, lesson plans for the curriculum, assessments, and measures of fidelity.
Xtreme Reading Intervention Report - Adolescent Literacy
Review Details
Reviewed: February 2021
- Randomized Controlled Trial
- Meets WWC standards without reservations because it is a randomized controlled trial with low attrition.
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Please see the WWC summary of evidence for Xtreme Reading.
Findings
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Stanford Diagnostic Reading Test, fourth edition (SDRT-4) |
Xtreme Reading vs. Business as usual |
0 Months |
Full sample- Xtreme Reading intervention vs. business as usual comparison;
|
21.95 |
21.75 |
No |
-- | |
|
Stanford Diagnostic Reading Test, fourth edition (SDRT-4) |
Xtreme Reading vs. READ 180® |
0 Months |
Full sample- Xtreme Reading intervention vs. READ 180 comparison;
|
21.95 |
24.14 |
No |
-- |
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Sample Characteristics
Characteristics of study sample as reported by study author.
-
4% English language learners -
Female: 57%
Male: 43% -
Urban
-
- B
- A
- C
- D
- E
- F
- G
- I
- H
- J
- K
- L
- P
- M
- N
- O
- Q
- R
- S
- V
- U
- T
- W
- X
- Z
- Y
- a
- h
- i
- b
- d
- e
- f
- c
- g
- j
- k
- l
- m
- n
- o
- p
- q
- r
- s
- t
- u
- v
- x
- w
- y
Massachusetts
-
Race Other or unknown 74% White 26%
Study Details
Setting
The study was conducted in five schools within two school districts, Chicopee and Springfield, in western Massachusetts.
Study sample
Across the 5 study years, the analytic samples for the three study conditions —Xtreme Reading, READ 180®, and business-as-usual condition—included 223, 231 and 225 ninth-grade students, respectively, with reading proficiencies 2 to 4 years below the ninth-grade level. Of the 679 students in the analytic sample, 73% were eligible for free or reduced-price lunch, 26% were White, and 20% were in special education. A small percentage (4%) of the students were English learners, and more than half (57%) were female.
Intervention Group
Xtreme Reading is a supplemental literacy curriculum designed to improve the literacy skills of struggling students in grades 6 to 12. The curriculum is primarily designed to help students improve their vocabulary, decoding, fluency, and reading comprehension skills. The Xtreme Reading program was offered to students as a supplement to their standard English language arts course. Class size was capped at 15 students, and the class was taught by the teacher randomly assigned to administer the Xtreme Reading program in each study school in each of the 5 study years. The class took place for 45 minutes per day, sometimes within a 90-minute block of English language arts courses.
Comparison Group
Students in the business-as-usual comparison condition received the standard English language arts instruction provided in the regular school curriculum and continued their participation in any regularly scheduled elective class, such as career and technical education, art, physical education, health, or foreign language. They did not receive supplemental English language arts instruction. Students in the READ 180® condition received the READ 180® curriculum as a supplement to the standard English language arts course. The class was taught by the teacher randomly assigned to administer the Read 180® curriculum in each study school in each of the 5 study years. The class took place for 90 minutes per day, paced to complete the Read 180® curriculum over 125 to 145 school days.
Support for implementation
The developers of the Xtreme Reading program trained teachers on learning strategies for students. The professional development model included initial training, ongoing in-class mentoring by developers, and workshops on specific routines. The professional development varied across study years. Teachers received 3 days of summer training in the first study year, which was shortened to 2 days in the second study year. Administrators held a 1-day summer meeting to support teachers in the first study year only. Developers also conducted monthly in-class mentoring with teachers during both study years. Teachers also attended 4 to 5 full days of additional workshops, depending on the study year.
IES Performance Measure
Review Details
Reviewed: February 2018
- IES Performance Measure (findings for READ 180®)
- Randomized Controlled Trial
- Meets WWC standards without reservations because it is a randomized controlled trial with low attrition.
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Findings
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Stanford Diagnostic Reading Test, fourth edition (SDRT-4) |
READ 180® vs. Business as usual |
0 Days |
Read 180 vs. Control (Stayers);
|
24.14 |
21.75 |
Yes |
|
|
|
Stanford Diagnostic Reading Test, fourth edition (SDRT-4) |
READ 180® vs. Xtreme Reading |
0 Days |
Read 180 vs. Xtreme Reading (Stayers);
|
24.14 |
21.95 |
No |
-- | |
|
Stanford Diagnostic Reading Test, fourth edition (SDRT-4) |
READ 180® vs. Business as usual |
0 Days |
Xtreme Reading vs. Control (Stayers);
|
21.95 |
21.75 |
No |
-- | |
|
Stanford Diagnostic Reading Test, fourth edition (SDRT-4) |
READ 180® vs. READ 180® |
0 Days |
Xtreme Reading vs. Read 180 (Stayers);
|
21.95 |
24.14 |
No |
-- |
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Sample Characteristics
Characteristics of study sample as reported by study author.
-
4% English language learners -
- B
- A
- C
- D
- E
- F
- G
- I
- H
- J
- K
- L
- P
- M
- N
- O
- Q
- R
- S
- V
- U
- T
- W
- X
- Z
- Y
- a
- h
- i
- b
- d
- e
- f
- c
- g
- j
- k
- l
- m
- n
- o
- p
- q
- r
- s
- t
- u
- v
- x
- w
- y
Massachusetts
Study Details
Setting
The study was conducted in five high schools in two districts in Massachusetts.
Study sample
For the analytic ITT sample (N = 679), 71% were minority, 56% were female, 21% were receiving special education and related services, 4% were EL, 69% received free or reduced price lunch, and average attendance was 92%. Comparison was 71% minority, 53% female, 19% were receiving special education and related services, 4% were EL, 74% received free or reduced price lunch, and average attendance was 91%. Read 180 was 74% minority, 61% female, 18% were receiving special education and related services, 3% were EL, 69% received free or reduced price lunch, and average attendance was 90%. Xtreme Reading was 78% minority, 57% female, 24% were receiving special education and related services, 4% were EL, 76% received free or reduced price lunch, and average attendance was 91%.
Intervention Group
Read 180 is a multi-part intervention for struggling readers that includes extensive use of instructional software, small-group instruction, and modeled and independent reading. The program uses "anchored instruction," a pedagogical technique that relies on "authentic situations as anchors" for problem solving. The program also uses computer-assisted instructional software that tracks individual student progress and adjusts reading instruction accordingly. The software has "an animated tutor who guides the student and provides feedback via a digitized human voice." The model is based on a 90-minute block that blends whole-class instruction and small-group student work, beginning with 20 minutes of whole-class instruction and concluding with 10 minute whole-class wrap up. For the intervening 60 minutes, students rotate among various stations. Measures of implementation fidelity varied over the five years of the study, depending in part on whether teachers followed the pacing calendar or devoted the full 90 minutes to READ 180 instruction. "Over time, data from multiple sources suggest READ 180 classes in one of the vocational-technical schools did not occur as planned, and were blended with regular ELA content." It appears that as of Years 4 and 5, all participating schools scheduled students to receive 90 minutes of Read 180 instruction Xtreme Reading targets students reading at least two years below grade level but who read at or above the fourth grade level. Intensive strategy instruction focuses on accurate word recognition and increased fluency and comprehension. The approach to instruction involves intensive lessons in which students have numerous opportunities to practice targeted learning strategies. Developers train teachers on “Learning Strategies” for students. The professional development model includes initial training, ongoing in-class mentoring by providers, and workshops on specific routines (See support for implementation below.) Class size was set at 15, and dosage was set to 45 minutes per day, sometimes within a 90-minute block of ELA courses. Some barriers prevented students from receiving the full 45 minutes of Xtreme Reading instruction in some cases: mediating factors for whether students received the correct dosage of the intervention included "Teacher buy-in and satisfaction with the program; (2) teacher ability to manage student behavior and elicit student engagement with material;; and (3) prevalence of reported barriers such as ELA and/or district or school assessment requirements as well as low rates of student attendance."
Comparison Group
Students in the comparison condition received supplemental services available to students in need of additional reading support and standard English language art courses for all students.
Support for implementation
Read 180: Teachers attended an initial 2 day summer training and administrators attended a 1 day training. Mentoring was provided by developers in-class monthly during the school year (only 6 months of the 8 in Year 1). Online RED course of 7 online sessions and in-person seminar (8 3 hours sessions) were held during the year. Materials included a paper library, CDs, audio-books, computers, audio equipment, interactive software (including online books, CAI-software, online lessons, and videos), and data management software and assessments. Xtreme Reading: Teachers received three days of summer training in Year 1, which was shortened to 2 days in Year 2. Administrators held a one day summer meeting to support teachers in Year 1 only. Developers provided in-class mentoring monthly during the school year (8 times in Year 1 and 9 times in Year 2). Teachers also attended 4 full days of additional workshops in Year 1 and 5 days in Year 2. Teachers received a reading library, lists of supplements, a curriculum included teacher/student notebooks, lesson plans for curriculum, assessments and measures of fidelity.
READ 180® Intervention Report - Adolescent Literacy
Review Details
Reviewed: November 2016
- Randomized Controlled Trial
- Meets WWC standards with reservations because it is a randomized controlled trial with high attrition, but the analytic intervention and comparison groups satisfy the baseline equivalence requirement.
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Please see the WWC summary of evidence for READ 180®.
Findings
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|
|---|---|---|---|---|---|---|---|---|---|
|
Stanford Diagnostic Reading Test, fourth edition (SDRT-4) |
READ 180® vs. Business as usual |
1 Year |
Cohorts 1-5;
|
24.14 |
21.75 |
Yes |
|
|
|
| Show Supplemental Findings | |||||||||
|
Stanford Diagnostic Reading Test, fourth edition (SDRT-4) |
READ 180® vs. Business as usual |
1 Year |
Cohorts 1-4;
|
665.41 |
660.12 |
Yes |
|
||
|
Stanford Diagnostic Reading Test, fourth edition (SDRT-4) |
READ 180® vs. Business as usual |
1 Year |
Cohorts 1-3;
|
665.27 |
659.99 |
Yes |
|
||
|
Stanford Diagnostic Reading Test, fourth edition (SDRT-4) |
READ 180® vs. Business as usual |
1 Year |
Cohorts 1-2;
|
664.78 |
661.94 |
No |
-- | ||
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Sample Characteristics
Characteristics of study sample as reported by study author.
-
3% English language learners -
74% Minority -
26% Non-minority -
Female: 61%
Male: 39% -
- B
- A
- C
- D
- E
- F
- G
- I
- H
- J
- K
- L
- P
- M
- N
- O
- Q
- R
- S
- V
- U
- T
- W
- X
- Z
- Y
- a
- h
- i
- b
- d
- e
- f
- c
- g
- j
- k
- l
- m
- n
- o
- p
- q
- r
- s
- t
- u
- v
- x
- w
- y
Massachusetts
Study Details
Setting
The study was conducted in two school districts, Chicopee and Springfield, in western Massachusetts.
Study sample
In each of the 5 study years, students in five study schools were screened prior to random assignment. Students at least two—but less than four—grade levels behind in reading performance were selected to participate. Students were excluded from the sample if (a) they had an IEP that specified reading supports not compatible with READ 180®, (b) they lacked sufficient English language proficiency, (c) their parents opted out of the study, (d) they were enrolled in an off-campus evening school, (e) they were deemed not to be a “struggling reader” based on grade history and MCAS scores, or (f) they could not be located in school enrollment records. Over the five annual cohorts, a total of 548 ninth-grade students with five teachers per year (one in each of five schools) were randomly assigned to the READ 180® group. The READ 180® analysis sample included 231 students taught by five teachers in five schools. This analysis sample was comprised of 74% racial and/or ethnic minorities, 61% female students, 18% special education students, and 3% English learners. A majority of students (69%) were eligible for free or reduced-price lunch. A total of 566 students with five teachers per year (one in each of five schools) were randomly assigned to the comparison group. The analysis sample for the comparison group includes 225 students taught by five teachers in five schools. This analysis sample was comprised of 71% racial and/or ethnic minorities, 53% female students, 19% special education students, and 4% English learners. A majority of students (74%) were eligible for free or reduced-price lunch. Results for additional samples were reported in Year 2, Year 3, and Year 4 reports. In the Year 2 report, which includes impact estimates for a sample combining Cohorts 1–2, there were 128 students in the intervention group and 113 students in the comparison group. The Year 3 report presents findings for Cohorts 1–3, which included 175 students in the intervention group and 159 in the comparison. The Year 4 report presents findings on Cohorts 1–4, which included 186 students in the intervention group and 178 in the comparison. These supplemental findings do not factor into the intervention’s rating of effectiveness.
Intervention Group
The READ 180® intervention was delivered as a 90-minute daily supplement to the standard ninth-grade ELA course. A typical daily session included 20 minutes of whole-class instruction, 60 minutes of small-group breakouts involving direct instruction, independent work using program software, and modeled or independent reading. In addition, the intervention included recommended instructional strategies and instructional materials, including videos and interactive work texts. The READ 180® curriculum was paced to be completed over 125–145 school days; the average number of sessions attended by each student was not reported.
Comparison Group
Students in the comparison condition received the standard ELA course (as did students in the intervention condition), as well as supplemental services ordinarily available to all students. In practice, comparison group students had minimal access to supplemental services. None of the comparison group teachers reported having any past experience with the READ 180® program, and they did not receive formal professional development in literacy instruction beyond what was customarily provided to all teachers. Use of multimedia appears to have been much more limited in the comparison group than in the intervention group.
Support for implementation
Teachers implementing the intervention were required to participate in professional development activities. Those implementing READ 180® for the first time were required to complete 52 hours of professional development over the course of the year in online training (seven sessions), group seminars (up to 30 hours), and individual face-to-face sessions (up to 16 hours). Less professional development was required of more experienced users: teachers with 3 years of prior READ 180® experience had to complete only 8 hours, and those implementing their fifth year had no such requirement.
Additional Sources
In the case of multiple manuscripts that report on one study, the WWC selects one manuscript as the primary citation and lists other manuscripts that describe the study as additional sources.
-
Sprague, K., Zaller, C., Kite, A., & Hussar, K. (2010). Springfield-Chicopee School Districts Striving Readers (SR) program Year 3 report: Evaluation of implementation and impact. Providence, RI: The Education Alliance at Brown University.
-
Sprague, K., Zaller, C., Kite, A., & Hussar, K. (2009). Springfield-Chicopee School Districts Striving Readers (SR) program Year 2 report: Evaluation of implementation and impact. Providence, RI: The Education Alliance at Brown University.
-
Sprague, K., Zaller, C., Kite, A., & Hussar, K. (2011). Springfield-Chicopee School Districts Striving Readers (SR) program Year 4 report: Evaluation of implementation and impact. Providence, RI: The Education Alliance at Brown University.
IES Performance Measure
Review Details
Reviewed: March 2016
- IES Performance Measure
- Randomized Controlled Trial
- Meets WWC standards without reservations
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Findings
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Sample Characteristics
Study sample characteristics were not reported.An indicator of the effect of the intervention, the improvement index can be interpreted as the expected change in percentile rank for an average comparison group student if that student had received the intervention.
For more, please see the WWC Glossary entry for improvement index.
An outcome is the knowledge, skills, and attitudes that are attained as a result of an activity. An outcome measures is an instrument, device, or method that provides data on the outcome.
A finding that is included in the effectiveness rating. Excluded findings may include subgroups and subscales.
The sample on which the analysis was conducted.
The group to which the intervention group is compared, which may include a different intervention, business as usual, or no services.
The timing of the post-intervention outcome measure.
The number of students included in the analysis.
The mean score of students in the intervention group.
The mean score of students in the comparison group.
The WWC considers a finding to be statistically significant if the likelihood that the finding is due to chance alone, rather than a real difference, is less than five percent.
The WWC reviews studies for WWC products, Department of Education grant competitions, and IES performance measures.
The name and version of the document used to guide the review of the study.
The version of the WWC design standards used to guide the review of the study.
The result of the WWC assessment of the study. The rating is based on the strength of evidence of the effectiveness of the intervention. Studies are given a rating of Meets WWC Design Standards without Reservations, Meets WWC Design Standards with Reservations, or >Does Not Meet WWC Design Standards.
A related publication that was reviewed alongside the main study of interest.
Study findings for this report.
Based on the direction, magnitude, and statistical significance of the findings within a domain, the WWC characterizes the findings from a study as one of the following: statistically significant positive effects, substantively important positive effects, indeterminate effects, substantively important negative effects, and statistically significant negative effects. For more, please see the WWC Handbook.
The WWC may review studies for multiple purposes, including different reports and re-reviews using updated standards. Each WWC review of this study is listed in the dropdown. Details on any review may be accessed by making a selection from the drop down list.
Tier 1 Strong indicates strong evidence of effectiveness,
Tier 2 Moderate indicates moderate evidence of effectiveness, and
Tier 3 Promising indicates promising evidence of effectiveness,
as defined in the
non-regulatory guidance for ESSA
and the regulations for ED discretionary grants (EDGAR Part 77).