The effects of Read Naturally on fluency and reading comprehension: A supplemental service intervention (four-school study) [Read Naturally vs. business as usual]
Heistad, D. (2008). [Unpublished manuscript]. https://www.readnaturally.com/userfiles/ckfiles/files/heistad-study_4schools.pdf.
-
examining156Students, grades3-5
Practice Guide
Review Details
Reviewed: September 2021
- Practice Guide (findings for Read Naturally)
- Quasi-Experimental Design
- Meets WWC standards with reservations because it uses a cluster quasi-experimental design that provides evidence of effects on individuals by satisfying the baseline equivalence requirement for the individuals in the analytic intervention and comparison groups.
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Findings
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Northwest Achievement Levels Tests (NALT) |
Read Naturally vs. Business as usual |
0 Days |
Full sample;
|
195.40 |
192.90 |
No |
-- |
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Sample Characteristics
Characteristics of study sample as reported by study author.
-
35% English language learners -
Female: 44%
Male: 56% -
Urban
-
- B
- A
- C
- D
- E
- F
- G
- I
- H
- J
- K
- L
- P
- M
- N
- O
- Q
- R
- S
- V
- U
- T
- W
- X
- Z
- Y
- a
- h
- i
- b
- d
- e
- f
- c
- g
- j
- k
- l
- m
- n
- o
- p
- q
- r
- s
- t
- u
- v
- x
- w
- y
Minnesota
-
Race Black 35% Native American 4% Other or unknown 39% White 22% -
Ethnicity Hispanic 39% Not Hispanic or Latino 61%
Study Details
Setting
This study was conducted in public elementary schools in Minneapolis, MN. Four schools implemented the intervention.
Study sample
The sample was 56% male, 44% female, 35% English learners, 65% eligible for free or reduced-price lunch, 35% Black, 4% Native American, 22% White, and 39% Hispanic. Twelve percent of the sample had an Individualized Education Plan. Students in the district as a whole were 52% male, 25% English learners, 61% eligible for free or reduced-priced lunch, 44% Black, and 14% Hispanic.
Intervention Group
The study examined the effectiveness of a reading intervention for students struggling with reading. The intervention group received the Read Naturally intervention. Read Naturally was delivered to students selected for the intervention group for the entire 2003-2004 school year, but was implemented differently in each of the four schools with intervention group students. Two schools implemented the Master's Edition of the intervention and the other two schools implemented the Software Edition of the intervention. Additionally, the intervention was implemented as a pull-out supplement during the school day in two schools and as an after-school program in the other two schools. (The study does not indicate whether these are the same sets of schools that implemented the different editions of Read Naturally.) Key components of the program included reading aloud with a fluent model, repeated reading of passages at individual student reading levels, literal and inferential comprehension questions, and continuous progress self-monitoring using charts and graphs. A trained instructor monitored the process and provided corrective feedback and guided practice.
Comparison Group
Students in the comparison group received their district's business-as-usual reading curriculum.
Support for implementation
One supervising teacher in each of the four schools implementing the intervention was trained in the appropriate procedures by a Read Naturally certified instructor. This training included an initial assessment of student level of instruction using curriculum-based measurement procedures, placement procedures, use of comprehension assessments and strategies, student goal setting, and progress monitoring procedures.
Read Naturally Intervention Report - Adolescent Literacy
Review Details
Reviewed: March 2013
- Quasi-Experimental Design
- Meets WWC standards with reservations
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Please see the WWC summary of evidence for Read Naturally.
Findings
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Northwest Achievement Levels Test (NALT): Reading |
Read Naturally vs. Business as usual |
Posttest, Spring 2004 |
Grades 3-5;
|
195.40 |
192.90 |
No |
-- |
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Sample Characteristics
Characteristics of study sample as reported by study author.
-
35% English language learners -
Female: 44%
Male: 56% -
Urban
-
- B
- A
- C
- D
- E
- F
- G
- I
- H
- J
- K
- L
- P
- M
- N
- O
- Q
- R
- S
- V
- U
- T
- W
- X
- Z
- Y
- a
- h
- i
- b
- d
- e
- f
- c
- g
- j
- k
- l
- m
- n
- o
- p
- q
- r
- s
- t
- u
- v
- x
- w
- y
Minnesota
-
Race Black 35% -
Ethnicity Hispanic 39% Not Hispanic or Latino 61%
Study Details
Setting
The study took place in the Minneapolis Public School district, in schools that were not on the No Child Left Behind list of schools failing to make adequate yearly progress in 2003.
Study sample
Read Naturally® was implemented with students in grades 3–5 in four elementary schools in the Minneapolis Public School district. Comparison group students were drawn from the same grade in the same school district. The author does not specify the number of schools attended by comparison group students. Students were selected for the Read Naturally® intervention based on parent and teacher recommendations and, according to the author, were generally not considered to be “on course” for proficiency on the state assessments administered in the spring of grades 3–5. The analysis sample included 156 students in grades 3–5 (78 in Read Naturally® and 78 in the comparison group); 46 in grade 3, 66 in grade 4, and 44 in grade 5. The demographic characteristics of the Read Naturally® students were: 56% male, 12% classified as special education, 35% classified as English language learners (ELL), and 65% receiving free or reduced-price lunch. With respect to race and ethnicity, 39% of the intervention group students were Hispanic, 35% were African American, 22% were White, and 4% were Native American. No similar demographic information for the comparison sample was presented in the study.
Intervention Group
Two schools used the Read Naturally® Masters version that employed audio cassettes and hard-copy reading materials, while two schools used the Read Naturally® Software Edition. Two schools implemented Read Naturally® as a pull-out intervention during the school day, while two schools used it as part of an after-school program. No further information was provided in the study regarding the intervention condition.
Comparison Group
The study author created a matched comparison group from within the Minneapolis Public School district using students that were not receiving the Read Naturally® program. Students were first matched by a pretest score on the Northwest Achievement Levels Test (NALT)– Reading measure, followed by the following demographic factors: grade, ELL status, special education status, free or reduced-price lunch status, race/ethnicity, home language, and gender. Read Naturally® students were only matched to other students attending schools classified with the same AYP status as their own school.
Outcome descriptions
The study included one eligible outcome measure, the reading portion of the NALT, a statebased adaptive assessment. The NALT is administered in the spring, with prior year’s NALT scores used as a pretest measure in the study. For a more detailed description of this outcome measure, see Appendix B. The reading portion of the Minnesota Comprehensive Assessment (MCA) was also administered to the subsample of 88 students in grades 3 and 5 at posttest in the spring of 2004. However, the results for the subsample of students using this outcome measure are not included in this review because baseline equivalence for the analysis sample was not established.
Support for implementation
One teacher in each intervention group school was trained in Read Naturally® procedures by a Read Naturally® instructor. Training included: initial assessment of student level of instruction using curriculum-based measurement procedures, placement procedures, use of comprehension assessments and strategies, student goal setting, and progress monitoring procedures.
Additional Sources
In the case of multiple manuscripts that report on one study, the WWC selects one manuscript as the primary citation and lists other manuscripts that describe the study as additional sources.
-
Heistad, D. (2008c). The effects of Read Naturally on grade 3 reading. (Unpublished manuscript).
-
Read Naturally. (n.d.). Case 3: Third-grade students, Minneapolis, MN. Read Naturally, Inc. Retrieved November 5, 2009, from http://www.readnaturally.com/approach/case3.htm.
-
Heistad, D. (2008b). The effects of Read Naturally on fluency and reading comprehension: A supplemental service intervention (two-school study). (Unpublished manuscript).
-
Read Naturally. (n.d.). Case 7: Two-school study, Minneapolis, MN. St. Paul, MN: Author. Retrieved from http://www.readnaturally.com
-
Read Naturally. (n.d.). Case 4: Four-school study, Minneapolis, MN. St. Paul, MN: Author. Retrieved from http://www.readnaturally.com
An indicator of the effect of the intervention, the improvement index can be interpreted as the expected change in percentile rank for an average comparison group student if that student had received the intervention.
For more, please see the WWC Glossary entry for improvement index.
An outcome is the knowledge, skills, and attitudes that are attained as a result of an activity. An outcome measures is an instrument, device, or method that provides data on the outcome.
A finding that is included in the effectiveness rating. Excluded findings may include subgroups and subscales.
The sample on which the analysis was conducted.
The group to which the intervention group is compared, which may include a different intervention, business as usual, or no services.
The timing of the post-intervention outcome measure.
The number of students included in the analysis.
The mean score of students in the intervention group.
The mean score of students in the comparison group.
The WWC considers a finding to be statistically significant if the likelihood that the finding is due to chance alone, rather than a real difference, is less than five percent.
The WWC reviews studies for WWC products, Department of Education grant competitions, and IES performance measures.
The name and version of the document used to guide the review of the study.
The version of the WWC design standards used to guide the review of the study.
The result of the WWC assessment of the study. The rating is based on the strength of evidence of the effectiveness of the intervention. Studies are given a rating of Meets WWC Design Standards without Reservations, Meets WWC Design Standards with Reservations, or >Does Not Meet WWC Design Standards.
A related publication that was reviewed alongside the main study of interest.
Study findings for this report.
Based on the direction, magnitude, and statistical significance of the findings within a domain, the WWC characterizes the findings from a study as one of the following: statistically significant positive effects, substantively important positive effects, indeterminate effects, substantively important negative effects, and statistically significant negative effects. For more, please see the WWC Handbook.
The WWC may review studies for multiple purposes, including different reports and re-reviews using updated standards. Each WWC review of this study is listed in the dropdown. Details on any review may be accessed by making a selection from the drop down list.
Tier 1 Strong indicates strong evidence of effectiveness,
Tier 2 Moderate indicates moderate evidence of effectiveness, and
Tier 3 Promising indicates promising evidence of effectiveness,
as defined in the
non-regulatory guidance for ESSA
and the regulations for ED discretionary grants (EDGAR Part 77).