Replicating the Impact of a Supplemental Beginning Reading Intervention: The Role of Instructional Context
Coyne, Michael D.; Little, Mary; Rawlinson, D'Ann; Simmons, Deborah; Kwok, Oi-man; Kim, Minjun; Simmons, Leslie; Hagan-Burke, Shanna; Civetelli, Christina (2013). Journal of Research on Educational Effectiveness, v6 n1 p1-23 2013. Retrieved from: https://eric.ed.gov/?id=EJ994735
-
examining162Students, gradeK
Publication
Review Details
Reviewed: January 2024
- Publication (findings for Early Reading Intervention (ERI))
- Randomized Controlled Trial
- Meets WWC standards with reservations because it is a cluster randomized controlled trial with a risk of bias from individuals who entered clusters after random assignment, but the analytic intervention and comparison groups satisfy the baseline equivalence requirement.
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Findings
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
|---|---|---|---|---|---|---|---|
|
Comprehensive Test of Phonological Processing: Sound Matching Subtest |
Early Reading Intervention (ERI) vs. Business as usual |
2 Weeks |
Full sample;
|
9.61 |
9.37 |
No |
-- |
|
Woodcock Reading Mastery Test-Revised (WRMT-R): Word Attack Subtest |
Early Reading Intervention (ERI) vs. Business as usual |
2 Weeks |
Full sample;
|
108.09 |
108.60 |
No |
-- |
|
Woodcock Reading Mastery Tests–Revised/Normative Update Supplementary Letter Checklist- Letter Sound |
Early Reading Intervention (ERI) vs. Business as usual |
2 Weeks |
Full sample;
|
26.15 |
25.76 |
No |
-- |
|
Comprehensive Test of Phonological Processing: Blending Words Subtest |
Early Reading Intervention (ERI) vs. Business as usual |
2 Weeks |
Full sample;
|
10.87 |
11.27 |
No |
-- |
|
Dynamic Indicators of Basic Early Literacy Skills (DIBELS): Phoneme Segmentation Fluency Subtest |
Early Reading Intervention (ERI) vs. Business as usual |
2 Weeks |
Full sample;
|
43.86 |
44.51 |
No |
-- |
|
Woodcock Reading Mastery Tests–Revised/Normative Update Supplementary Letter Checklist- Letter Name |
Early Reading Intervention (ERI) vs. Business as usual |
2 Weeks |
Full sample;
|
25.87 |
26.15 |
No |
-- |
|
Dynamic Indicators of Basic Early Literacy Skills (DIBELS): Nonsense Word Fluency Subtest |
Early Reading Intervention (ERI) vs. Business as usual |
2 Weeks |
Full sample;
|
30.08 |
32.71 |
No |
-- |
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
|---|---|---|---|---|---|---|---|
|
Woodcock Reading Mastery Test-Revised (WRMT-R): Word Identification Subtest |
Early Reading Intervention (ERI) vs. Business as usual |
2 Weeks |
Full sample;
|
107.13 |
109.19 |
No |
-- |
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Sample Characteristics
Characteristics of study sample as reported by study author.
-
15% English language learners -
Female: 46%
Male: 54% -
Rural, Suburban
-
- B
- A
- C
- D
- E
- F
- G
- I
- H
- J
- K
- L
- P
- M
- N
- O
- Q
- R
- S
- V
- U
- T
- W
- X
- Z
- Y
- a
- h
- i
- b
- d
- e
- f
- c
- g
- j
- k
- l
- m
- n
- o
- p
- q
- r
- s
- t
- u
- v
- x
- w
- y
Florida
-
Race Black 9% Other or unknown 30% White 61% -
Ethnicity Hispanic 24% Not Hispanic or Latino 76% -
Eligible for Free and Reduced Price Lunch Other or unknown 100%
Study Details
Setting
The study was conducted in a large school district in Florida. The sample consisted of 48 kindergarten classrooms within 10 schools. Eight of the 10 schools were Title I schools.
Study sample
The analytic sample was 54 percent male. The sample was 61 percent White, 9 percent African American, 6 percent other race, and race was not specified for 24 percent of students. Twenty-four percent of students were Latino. Fifteen percent of students were English learners.
Intervention Group
The ERI curriculum includes 126 daily 30-minute lessons and was delivered as a supplemental intervention to the main reading curriculum. Each lesson includes seven activities that last 3-5 minutes, including phonological awareness, alphabetic understanding, and integration of writing and spelling. The program has four major components: learning letters and sounds; segmenting, blending, and integrating; reading words; and reading sentences and storybooks. The lessons are scripted and were implemented in small groups of students by kindergarten classroom teachers.
Comparison Group
Students in the comparison group received supplementary reading instruction in daily 30-minute small-group lessons using typical reading interventions focusing on early literacy skills.
Support for implementation
Treatment teachers received a two-day professional development training. The first day was before the start of the intervention to introduce the teachers to the curriculum. The second day was held half-way through the school year and focused on lessons and materials for the remaining lessons.
Department-funded evaluation
Review Details
Reviewed: May 2023
- Department-funded evaluation (findings for Early Reading Intervention)
- Randomized Controlled Trial
- The study does not meet WWC group design standards because the equivalence of the analytic intervention and comparison groups is necessary but the requirement was not satisfied.
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Findings
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Sample Characteristics
Study sample characteristics were not reported.Additional Sources
In the case of multiple manuscripts that report on one study, the WWC selects one manuscript as the primary citation and lists other manuscripts that describe the study as additional sources.
-
Simmons, Deborah C.; Coyne, Michael D.; Hagan-Burke, Shanna; Kwok, Oi-man; Simmons, Leslie; Johnson, Caitlin; Zou, Yuanyuan; Taylor, Aaron B.; McAlenney, Athena Lentini; Ruby, Maureen; Crevecoeur, Yvel C. (2011). Effects of Supplemental Reading Interventions in Authentic Contexts: A Comparison of Kindergarteners' Response. Exceptional Children, v77 n2 p207-228.
Practice Guide
Review Details
Reviewed: February 2023
- Practice Guide (findings for Kindergarten beginning reading intervention—Coyne et al. (2013))
- Randomized Controlled Trial
- Meets WWC standards with reservations because it is a randomized controlled trial with low attrition, but the randomization was compromised.
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Findings
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
|---|---|---|---|---|---|---|---|
|
Woodcock Reading Mastery Test- Alphabet knowledge letter checklist-sounds |
Kindergarten beginning reading intervention—Coyne et al. (2013) vs. Business as usual |
126 Days |
Full sample;
|
25.56 |
25.76 |
No |
-- |
|
Woodcock Reading Mastery Test- letter name checklist |
Kindergarten beginning reading intervention—Coyne et al. (2013) vs. Business as usual |
126 Days |
Full sample;
|
25.82 |
26.15 |
No |
-- |
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
|---|---|---|---|---|---|---|---|
|
Woodcock Reading Mastery Test - Revised: Word Attack |
Kindergarten beginning reading intervention—Coyne et al. (2013) vs. Business as usual |
126 Days |
Full sample;
|
108.57 |
108.60 |
No |
-- |
|
Woodcock Reading Mastery Test - Revised: Word Identification |
Kindergarten beginning reading intervention—Coyne et al. (2013) vs. Business as usual |
126 Days |
Full sample;
|
107.30 |
109.19 |
No |
-- |
|
DIBELS Nonsense Word Fluency |
Kindergarten beginning reading intervention—Coyne et al. (2013) vs. Business as usual |
126 Days |
Full sample;
|
29.80 |
32.71 |
No |
-- |
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Sample Characteristics
Characteristics of study sample as reported by study author.
-
- B
- A
- C
- D
- E
- F
- G
- I
- H
- J
- K
- L
- P
- M
- N
- O
- Q
- R
- S
- V
- U
- T
- W
- X
- Z
- Y
- a
- h
- i
- b
- d
- e
- f
- c
- g
- j
- k
- l
- m
- n
- o
- p
- q
- r
- s
- t
- u
- v
- x
- w
- y
Florida
Study Details
Setting
The study took place in 48 classrooms in central Florida.
Intervention Group
Teachers implemented the Early Reading Intervention program with small groups of students. The intervention had four units: (1) learning letters and sounds; (2) segmenting, blending, and integrating sounds; (3) reading words; and (4) reading sentences and storybooks. The first half of each session focused on phonological awareness and the alphabet, while the second half focused on writing and spelling using the sounds previously taught. The intervention involved 30-minute sessions daily for 126 days.
Comparison Group
Teachers taught their regular lessons.
An indicator of the effect of the intervention, the improvement index can be interpreted as the expected change in percentile rank for an average comparison group student if that student had received the intervention.
For more, please see the WWC Glossary entry for improvement index.
An outcome is the knowledge, skills, and attitudes that are attained as a result of an activity. An outcome measures is an instrument, device, or method that provides data on the outcome.
A finding that is included in the effectiveness rating. Excluded findings may include subgroups and subscales.
The sample on which the analysis was conducted.
The group to which the intervention group is compared, which may include a different intervention, business as usual, or no services.
The timing of the post-intervention outcome measure.
The number of students included in the analysis.
The mean score of students in the intervention group.
The mean score of students in the comparison group.
The WWC considers a finding to be statistically significant if the likelihood that the finding is due to chance alone, rather than a real difference, is less than five percent.
The WWC reviews studies for WWC products, Department of Education grant competitions, and IES performance measures.
The name and version of the document used to guide the review of the study.
The version of the WWC design standards used to guide the review of the study.
The result of the WWC assessment of the study. The rating is based on the strength of evidence of the effectiveness of the intervention. Studies are given a rating of Meets WWC Design Standards without Reservations, Meets WWC Design Standards with Reservations, or >Does Not Meet WWC Design Standards.
A related publication that was reviewed alongside the main study of interest.
Study findings for this report.
Based on the direction, magnitude, statistical significance, and sample size of the findings within a domain, the WWC assigns effectiveness ratings as one of the following: Tier 1 (strong evidence), Tier 2 (moderate evidence), Tier 3 (promising evidence), uncertain effects, and negative effects. For more detail, please see the WWC Handbook.
The WWC may review studies for multiple purposes, including different reports and re-reviews using updated standards. Each WWC review of this study is listed in the dropdown. Details on any review may be accessed by making a selection from the drop down list.
Tier 1 Strong indicates strong evidence of effectiveness,
Tier 2 Moderate indicates moderate evidence of effectiveness, and
Tier 3 Promising indicates promising evidence of effectiveness,
as defined in the
non-regulatory guidance for ESSA
and the regulations for ED discretionary grants (EDGAR Part 77).