Efficacy of a High School Extensive Reading Intervention for English Learners with Reading Difficulties [Reading Intervention for Adolescents vs. business as usual]
Vaughn, Sharon; Martinez, Leticia R.; Williams, Kelly J.; Miciak, Jeremy; Fall, Anna-Maria; Roberts, Greg (2018). Retrieved from: https://eric.ed.gov/?id=ED586763
-
examining318Students, grades9-10
Practice Guide
Review Details
Reviewed: September 2021
- Practice Guide (findings for Reading Intervention for Adolescents (RIA))
- Randomized Controlled Trial
- Meets WWC standards without reservations because it is a randomized controlled trial with low attrition.
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Findings
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Test of Silent Reading Efficiency and Comprehension (TOSREC) |
Reading Intervention for Adolescents (RIA) vs. Business as usual |
0 Days |
Full sample Spring, 9th grade;
|
80.90 |
79.51 |
No |
-- |
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Gates-MacGinitie Reading Tests (GMRT-4) reading comprehension subtest |
Reading Intervention for Adolescents (RIA) vs. Business as usual |
0 Days |
Full sample Spring, 9th grade;
|
81.89 |
83.01 |
No |
-- |
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Gates-MacGinitie Reading Tests (GMRT-4) vocabulary subtest |
Reading Intervention for Adolescents (RIA) vs. Business as usual |
0 Days |
Full sample Spring, 9th grade;
|
79.81 |
79.89 |
No |
-- |
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Test of Word Reading Efficiency (TOWRE)- Phonemic Decoding Efficiency subtest |
Reading Intervention for Adolescents (RIA) vs. Business as usual |
0 Days |
Full sample Spring, 9th grade;
|
96.31 |
96.91 |
No |
-- | |
|
Test of Word Reading Efficiency (TOWRE)- Sight Word Efficiency subtest |
Reading Intervention for Adolescents (RIA) vs. Business as usual |
0 Days |
Full sample Spring, 9th grade;
|
91.72 |
93.53 |
No |
-- |
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Sample Characteristics
Characteristics of study sample as reported by study author.
-
63% English language learners -
Female: 38%
Male: 53% -
Urban
-
- B
- A
- C
- D
- E
- F
- G
- I
- H
- J
- K
- L
- P
- M
- N
- O
- Q
- R
- S
- V
- U
- T
- W
- X
- Z
- Y
- a
- h
- i
- b
- d
- e
- f
- c
- g
- j
- k
- l
- m
- n
- o
- p
- q
- r
- s
- t
- u
- v
- x
- w
- y
South, West
-
Race Black 1% Native American 0% Other or unknown 99% White 0% -
Ethnicity Hispanic 89% Not Hispanic or Latino 11%
Study Details
Setting
This study was conducted in three high schools from a diverse, urban school district in the southwestern United States. In all schools, the majority of students were Hispanic (54.9%-90.5%) and economically disadvantaged (75%-90%) based on free and/or reduced-price lunch status. Students at the start of the study who identified as English learners (ELs) ranged from 13.4%-49.4%. One of the three schools was on "needs improvement" status, while the other two met the state standards of accountability measures.
Study sample
Students were primarily Hispanic (89%, with race unknown for over 98% of students) and male (53%, with 9% missing gender data), and most spoke Spanish at home (89%). All students had been classified as English learners at some point in the five years prior to random assignment, and 63% were English learners at the start of the intervention. Seventy-five percent of these students were economically disadvantaged (presumably based on free and/or reduced-price lunch status), and 12% received special education services.
Intervention Group
The study examined the effectiveness of a reading intervention for students struggling with reading. The intervention examined by the study, the Reading Intervention for Adolescents (RIA) program, was administered over two full school years, Fall 2015 to Spring 2017. Participating students that had been randomly assigned to either of the two intervention groups—RIA or RIA with a modified version of the Check & Connect dropout prevention intervention—received RIA instruction in groups of 10-15 students. Students received approximately 3.75-4.25 hours of the intervention each week (approximately 50 minutes daily). The intervention included two phases. The first semester of the first school year was Phase I, which was based on REWARDS Secondary, an explicit instruction program that focused on reading fluency and vocabulary activities. This portion of the program covered topics such as identification of prefixes, suffixes, and vowels; reading parts of and complete words, and reading words both in isolation and in context. The second semester of the first year and each semester of the second year comprised Phase II, which included 14 instructional units corresponding to content areas in science and social studies. Phase II continued to use the collaborative learning approach and emphasized using the skills previously taught but in the respective content-area contexts. Intervention students continued to receive their core classes in English, math, science, and social studies during both phases but attended RIA instruction in place of an elective class. The intervention was delivered by five reading interventionists who were state certified in reading or English/language arts and had at least five years of teaching experience.
Comparison Group
Students in the comparison condition had “business-as-usual” classes with each comparison student enrolling in an elective class, as per usual, instead of enrolling in the RIA intervention course. For their elective courses, some comparison students received an extension of the school-provided English I or II course providing continued instruction of the in-class lesson, while other students enrolled in other elective courses (e.g. Cosmetology, Principles of Information Technology, Concepts of Engineering and Technology, etc.).
Support for implementation
The researchers hired and trained five reading interventionists to implement RIA. In both years, interventionists received 40 hours of training, which focused on elements of effective instruction and implementation. If needed, an additional 8-16 hours of training were provided at the end of Phase I. Two researchers provided limited coaching through in-person and audio observations. Researchers also held biweekly phone conferences with interventionists to discuss student progress and, if necessary, adjust instruction.
Single Study Review
Review Details
Reviewed: June 2021
- Single Study Review (findings for Reading Intervention for Adolescents (RIA))
- Randomized Controlled Trial
- Meets WWC standards with reservations because it is a randomized controlled trial with high attrition, but the analytic intervention and comparison groups satisfy the baseline equivalence requirement.
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Findings
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|
|---|---|---|---|---|---|---|---|---|---|
|
Test of Word Reading Efficiency (TOWRE-2): Phonemic Decoding Efficiency subtest |
Reading Intervention for Adolescents (RIA) vs. Business as usual |
0 Days |
Full sample, Spring 10th Grade;
|
98.74 |
98.28 |
No |
-- | ||
| Show Supplemental Findings | |||||||||
|
Test of Word Reading Efficiency (TOWRE-2): Phonemic Decoding Efficiency subtest |
Reading Intervention for Adolescents (RIA) vs. Business as usual |
0 Days |
Full sample, Fall 10th Grade;
|
96.22 |
97.03 |
No |
-- | ||
|
Test of Word Reading Efficiency (TOWRE-2): Phonemic Decoding Efficiency subtest |
Reading Intervention for Adolescents (RIA) vs. Business as usual |
0 Days |
Full sample, Spring 9th Grade;
|
95.35 |
96.91 |
No |
-- | ||
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|
|---|---|---|---|---|---|---|---|---|---|
|
Test of Silent Reading Efficiency and Comprehension (TOSREC) |
Reading Intervention for Adolescents (RIA) vs. Business as usual |
0 Days |
Full sample, Spring 10th Grade;
|
79.08 |
77.78 |
No |
-- | ||
| Show Supplemental Findings | |||||||||
|
Test of Silent Reading Efficiency and Comprehension (TOSREC) |
Reading Intervention for Adolescents (RIA) vs. Business as usual |
0 Days |
Full sample, Spring 9th Grade;
|
80.51 |
79.51 |
No |
-- | ||
|
Gates-MacGinitie Reading Tests (GMRT): Vocabulary subtest |
Reading Intervention for Adolescents (RIA) vs. Business as usual |
0 Days |
Full sample, Spring 9th Grade;
|
79.89 |
79.89 |
No |
-- | ||
|
Test of Silent Reading Efficiency and Comprehension (TOSREC) |
Reading Intervention for Adolescents (RIA) vs. Business as usual |
0 Days |
Full sample, Fall 10th Grade;
|
74.62 |
74.77 |
No |
-- | ||
|
Gates-MacGinitie Reading Tests (GMRT): Reading comprehension subtest |
Reading Intervention for Adolescents (RIA) vs. Business as usual |
0 Days |
Full sample, Spring 9th Grade;
|
82.42 |
83.01 |
No |
-- | ||
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Sample Characteristics
Characteristics of study sample as reported by study author.
-
63% English language learners -
Female: 38%
Male: 53% -
Urban
-
- B
- A
- C
- D
- E
- F
- G
- I
- H
- J
- K
- L
- P
- M
- N
- O
- Q
- R
- S
- V
- U
- T
- W
- X
- Z
- Y
- a
- h
- i
- b
- d
- e
- f
- c
- g
- j
- k
- l
- m
- n
- o
- p
- q
- r
- s
- t
- u
- v
- x
- w
- y
South, West
-
Race Black 1% Native American 0% Other or unknown 99% White 0% -
Ethnicity Hispanic 89% Not Hispanic or Latino 1%
Study Details
Setting
The study was conducted in three large high schools in an urban school district in the southwestern United states.
Study sample
The study included 260 ninth- and tenth-grade students. Sixty-three percent were current English learners (ELs) and 37 percent were former ELs. Fifty-three percent were male, 38 percent were female, and the rest did not report their gender. Eighty-nine percent were Hispanic/Latino; 1 percent were Black, White, or Native American; and 9 percent did not report race or ethnicity. Three-quarters were eligible for free or reduced-price lunch. Twelve percent were receiving special education services.
Intervention Group
Reading Intervention for Adolescents is a two-year intervention designed to develop reading skills in struggling readers. The intervention consists of a set of practices implemented in two phases: the first phase focuses on word study, fluency, vocabulary and comprehension at the sentence and paragraph levels while the second phase emphasizes vocabulary and comprehension instruction within content area texts. During both phases, teachers use an approach called Collaborative Strategic Reading in which students work collaboratively in small groups to learn and practice comprehension strategies. In this study, the first phase of the intervention was implemented during the first semester of the first year and the second phase occurred during the second semester of that year until the end of the second year. The researchers adapted the intervention for English learners by explicitly teaching academic vocabulary words and providing students with opportunities to use oral and written academic language in the context of content areas. Reading Intervention for Adolescents was provided in groups of 10 to 14 students who participated in the intervention instead of taking an elective class. Students received Reading Intervention for Adolescents instruction for about 4 hours each week over the two school years.
Comparison Group
Students in the comparison condition enrolled in an elective class instead of participating in the intervention. A wide range of elective courses were available to students, including extensions of English I or II courses, Introduction to Cosmetology, Principles of Information Technology, Concepts of Engineering and Technology, Art, Welding, and Junior Reserve Officer Training Corps.
Support for implementation
Five reading instructors hired by the research team delivered the intervention. Researchers provided 40 hours of training before the intervention began and an additional 8 to 16 hours after the end of the first phase of implementation. Two members of the research team provided in-person and remote coaching throughout the year.
Department-funded evaluation
Review Details
Reviewed: March 2020
- Department-funded evaluation (findings for Reading Intervention for Adolescents (RIA))
- Randomized Controlled Trial
- Meets WWC standards without reservations because it is a randomized controlled trial with low attrition.
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Findings
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Test of Silent Reading Efficiency and Comprehension |
Reading Intervention for Adolescents (RIA) vs. Business as usual |
0 Days |
Full sample Spring, 9th grade;
|
80.51 |
79.51 |
No |
-- | |
|
Gates-MacGinitie Reading Tests (4th ed.; GMRT-4) Vocabulary Subtest |
Reading Intervention for Adolescents (RIA) vs. Business as usual |
0 Days |
Full sample Spring, 9th grade;
|
79.89 |
79.89 |
No |
-- | |
|
Gates-MacGinitie Reading Tests (4th ed.; GMRT-4) Comprehension Subtest |
Reading Intervention for Adolescents (RIA) vs. Business as usual |
0 Days |
Full sample Spring, 9th grade;
|
82.42 |
83.01 |
No |
-- | |
|
Test of Word Reading Efficiency (TOWRE): Phonemic Decoding Efficiency subtest |
Reading Intervention for Adolescents (RIA) vs. Business as usual |
0 Days |
Full sample Spring, 9th grade;
|
95.35 |
96.91 |
No |
-- | |
|
Test of Word Reading Efficiency (TOWRE): Sight Word Efficiency subtest |
Reading Intervention for Adolescents (RIA) vs. Business as usual |
0 Days |
Full sample Spring, 9th grade;
|
90.34 |
93.53 |
No |
-- |
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Sample Characteristics
Characteristics of study sample as reported by study author.
-
63% English language learners -
Female: 38%
Male: 53% -
Urban
-
- B
- A
- C
- D
- E
- F
- G
- I
- H
- J
- K
- L
- P
- M
- N
- O
- Q
- R
- S
- V
- U
- T
- W
- X
- Z
- Y
- a
- h
- i
- b
- d
- e
- f
- c
- g
- j
- k
- l
- m
- n
- o
- p
- q
- r
- s
- t
- u
- v
- x
- w
- y
South
-
Race Black 1% Native American 0% Other or unknown 99% White 0% -
Ethnicity Hispanic 89% Not Hispanic or Latino 11%
Study Details
Setting
This intervention was delivered in an urban school district in the southwestern region of the United States (p. 4).
Study sample
Table 1 (p. 5) reports demographics for the sample of 358 students who enrolled in the study schools in grade 9 (this is slightly larger than the analytic sample). Students were primarily Hispanic (89 percent), and a majority of these students were male (53 percent, with 9 percent missing gender data), and most spoke Spanish at home (89 percent). All students had been classified as English learners at some point in the five years prior to random assignment, and 63 percent were English learners at the time of the intervention. Three quarters of these students were economically disadvantaged, and 12 percent received special education services.
Intervention Group
The Reading Intervention for Adolescents (RIA) program took place over two school years, and was delivered to groups of 10 to 15 students (p. 5). Students received approximately 3.75 to 4.25 hours of the intervention each week. The intervention included two phases. The first semester of the first school year was Phase I, which was based on REWARDS Secondary, an explicit instruction program designed that focused on reading fluency and vocabulary activities. This portion of the program covered topics such as identification of prefixes, suffixes, and vowels; reading parts of and complete words, and reading words both in isolation and in context. The second semester of the first year and each semester of the second year were Phase II, which included 14 instructional units corresponding to content areas in science and social studies. Phase II continued to use the collaborative learning approach and emphasized using the skills previously taught, but in the respective content-area contexts. (pp. 5-6) Approximately half of the students in the intervention group were also enrolled in a dropout prevention intervention, which was a modified version of Check & Connect. (p. 4)
Comparison Group
Students in the comparison condition either received a school-provided extension to their English I or English II class, or they enrolled in an elective. Electives included Introduction to Cosmetology, Principles of Information Technology, Concepts of Engineering and Technology, etc (p. 7). Similar to the intervention group, approximately half of the students in the comparison group were also enrolled in a dropout prevention intervention, which was a modified version of Check & Connect. (p. 4)
Support for implementation
The researchers hired and trained five reading interventionists (p. 6). In both years, interventionists received 40 hours of training, focused on elements of effective instruction and implementation. If needed, an additional 8-16 hour training session was provided at the end of Phase I. Two researchers provided limited coaching through in-person and audio observations. Researchers also held biweekly phone conferences with interventionists to discuss student progress and, if necessary, adjust instruction. (p. 6)
An indicator of the effect of the intervention, the improvement index can be interpreted as the expected change in percentile rank for an average comparison group student if that student had received the intervention.
For more, please see the WWC Glossary entry for improvement index.
An outcome is the knowledge, skills, and attitudes that are attained as a result of an activity. An outcome measures is an instrument, device, or method that provides data on the outcome.
A finding that is included in the effectiveness rating. Excluded findings may include subgroups and subscales.
The sample on which the analysis was conducted.
The group to which the intervention group is compared, which may include a different intervention, business as usual, or no services.
The timing of the post-intervention outcome measure.
The number of students included in the analysis.
The mean score of students in the intervention group.
The mean score of students in the comparison group.
The WWC considers a finding to be statistically significant if the likelihood that the finding is due to chance alone, rather than a real difference, is less than five percent.
The WWC reviews studies for WWC products, Department of Education grant competitions, and IES performance measures.
The name and version of the document used to guide the review of the study.
The version of the WWC design standards used to guide the review of the study.
The result of the WWC assessment of the study. The rating is based on the strength of evidence of the effectiveness of the intervention. Studies are given a rating of Meets WWC Design Standards without Reservations, Meets WWC Design Standards with Reservations, or >Does Not Meet WWC Design Standards.
A related publication that was reviewed alongside the main study of interest.
Study findings for this report.
Based on the direction, magnitude, and statistical significance of the findings within a domain, the WWC characterizes the findings from a study as one of the following: statistically significant positive effects, substantively important positive effects, indeterminate effects, substantively important negative effects, and statistically significant negative effects. For more, please see the WWC Handbook.
The WWC may review studies for multiple purposes, including different reports and re-reviews using updated standards. Each WWC review of this study is listed in the dropdown. Details on any review may be accessed by making a selection from the drop down list.
Tier 1 Strong indicates strong evidence of effectiveness,
Tier 2 Moderate indicates moderate evidence of effectiveness, and
Tier 3 Promising indicates promising evidence of effectiveness,
as defined in the
non-regulatory guidance for ESSA
and the regulations for ED discretionary grants (EDGAR Part 77).