Empirical evaluation of Read Naturally effects: A randomized control trial (RCT) (Unpublished journal article).
Christ, T. J., & Davie, J. (2009). University of Minnesota, Minneapolis.
-
examining106Students, grade3
Practice Guide
Review Details
Reviewed: February 2023
- Practice Guide (findings for Read Naturally)
- Randomized Controlled Trial
- Meets WWC standards without reservations because it is a randomized controlled trial with low attrition.
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Findings
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Gray Oral Reading Test-Accuracy |
Read Naturally vs. Business as usual |
11 Weeks |
Full sample;
|
8.50 |
7.20 |
Yes |
|
|
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Gray Oral Reading Test-Fluency |
Read Naturally vs. Business as usual |
11 Weeks |
Full sample;
|
8.50 |
7.50 |
No |
-- | |
|
DIBELS CBM-R passages - words correctly per minute |
Read Naturally vs. Business as usual |
11 Weeks |
Full sample;
|
76.00 |
70.00 |
No |
-- |
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Gray Oral Reading Test-Comprehension |
Read Naturally vs. Business as usual |
11 Weeks |
Full sample;
|
10.00 |
10.00 |
No |
-- | |
|
WRMT-R - Passage Comprehension subtest |
Read Naturally vs. Business as usual |
11 Weeks |
Full sample;
|
96.00 |
97.00 |
No |
-- |
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
TOWRE - Test of Word Reading Efficiency |
Read Naturally vs. Unknown |
11 Weeks |
Full sample;
|
94.90 |
93.50 |
No |
-- | |
|
Woodcock Reading Mastery Test - Revised: Word Identification |
Read Naturally vs. Business as usual |
11 Weeks |
Full sample;
|
99.00 |
98.00 |
No |
-- |
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Sample Characteristics
Characteristics of study sample as reported by study author.
-
23% English language learners -
- B
- A
- C
- D
- E
- F
- G
- I
- H
- J
- K
- L
- P
- M
- N
- O
- Q
- R
- S
- V
- U
- T
- W
- X
- Z
- Y
- a
- h
- i
- b
- d
- e
- f
- c
- g
- j
- k
- l
- m
- n
- o
- p
- q
- r
- s
- t
- u
- v
- x
- w
- y
Midwest
Study Details
Setting
The study was conducted in six schools within four school districts in the Midwest. No schools had previously used Read Naturally. Instruction took place in either a computer lab or unoccupied classrooms.
Study sample
Demographics for this sample were as follows: 10% special education, 23% ELL, 60% on FRPL; 42% white, 28% AA, 23% Hispanic, 6% Asian, 1% Native American. Four teachers were used to implement the intervention; all were recommended by the districts in the student; three had recently retired.
Intervention Group
The intervention involved 10 weeks of instruction beginning in January 2009. Baseline measures were collected ~2 weeks prior to the beginning of the intervention, and outcomes were collected ~1 week after the conclusion of the intervention. Instruction in Read Naturally was intended to be daily for 30 minutes a session. The time of day designated for Read Naturally instruction varied, but was designed to not conflict with existing reading instruction. Read Naturally instruction groups consisted of no more than 6 students, with one teacher supervising. Analysis of student intervention usage indicated an average of 20 minutes per session using the Read Naturally software, as opposed to the designed 30 minute session.
Comparison Group
Control group students continued to receive their classroom's normal reading instruction, with no supplemental fluency instruction. During the class time designated for Read Naturally instruction, control group students engaged in non-reading related activities.
Support for implementation
Each teacher attended a six-hour Read Naturally training session, including lecture and software practice. Intervention integrity checklists, produced by the developer for both students and teachers, were used to assess and evaluate the component of the intervention and their implementation. Bi-monthly classroom observations were also used to assess implementation fidelity.
Read Naturally Intervention Report - Beginning Reading
Review Details
Reviewed: July 2013
- Randomized Controlled Trial
- Meets WWC standards without reservations
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Please see the WWC summary of evidence for Read Naturally.
Findings
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Test of Word Reading Efficiency (TOWRE) |
Read Naturally vs. Business as usual |
Full |
Grade 3;
|
94.90 |
93.50 |
No |
-- | |
|
Woodcock Reading Mastery Tests- Revised (WRMT-R): Word Identification subtest |
Read Naturally vs. Business as usual |
Full |
Grade 3;
|
99.00 |
98.00 |
No |
-- |
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Gray Oral Reading Tests, Fourth Edition (GORT-4): Comprehension subtest |
Read Naturally vs. Business as usual |
Posttest |
Grade 3;
|
10.00 |
10.00 |
No |
-- | |
|
Woodcock Reading Mastery Test-Revised (WRMT-R): Passage Comprehension subtest |
Read Naturally vs. Business as usual |
Posttest |
Grade 3;
|
96.00 |
97.00 |
No |
-- |
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Gray Oral Reading Test Fourth Edition (GORT-4): Reading Accuracy subtest |
Read Naturally vs. Business as usual |
Posttest |
Grade 3;
|
8.50 |
7.20 |
Yes |
|
|
|
Gray Oral Reading Tests, Fourth Edition (GORT-4): Fluency subtest |
Read Naturally vs. Business as usual |
Posttest |
Grade 3;
|
8.50 |
7.50 |
Yes |
|
|
|
Dynamic Indicators of Basic Early Literacy Skills (DIBELS): Curriculum-Based Measurement of Reading (CBM-R) passages |
Read Naturally vs. Business as usual |
Posttest |
Grade 3;
|
76.00 |
70.00 |
No |
-- |
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Sample Characteristics
Characteristics of study sample as reported by study author.
-
23% English language learners -
Female: 45%
Male: 55% -
- B
- A
- C
- D
- E
- F
- G
- I
- H
- J
- K
- L
- P
- M
- N
- O
- Q
- R
- S
- V
- U
- T
- W
- X
- Z
- Y
- a
- h
- i
- b
- d
- e
- f
- c
- g
- j
- k
- l
- m
- n
- o
- p
- q
- r
- s
- t
- u
- v
- x
- w
- y
Midwest
-
Race Asian 6% Black 28% White 42% -
Ethnicity Hispanic 23% Not Hispanic or Latino 77%
Study Details
Setting
The study was conducted in six schools in four Midwestern school districts. None of the participating schools had previously used Read Naturally®.
Study sample
Third-grade students in the participating schools were eligible for the study if they were at or below the 40th percentile on a measure of oral reading fluency (DIBELS or AIMSweb) in the fall of third grade, and at or below the 40th percentile on reading comprehension as measured by the Measures of Academic Progress assessment at the end of second grade. After applying these criteria and obtaining consent from the parents of eligible students, 109 students were randomized within their classrooms to either the Read Naturally® group or the comparison group. Demographics for the randomized sample were as follows: 10% received special education, 23% were English language learners, and 60% received free or reduced-price lunch. The racial demographics were: 42% White, 28% African American, 23% Hispanic, 6% Asian, and 1% Native American. The analysis sample included 106 students (53 in the Read Naturally® group and 53 in the comparison group).
Intervention Group
Read Naturally® Software Edition was the version used and involved 10 weeks of instruction beginning in January 2009. Instruction in Read Naturally® was intended to be daily for 30 minutes a session. The time of day designated for Read Naturally® instruction varied across teachers, but was selected so that it would not conflict with existing reading instruction. Instruction groupings for the intervention consisted of no more than six students, with one teacher supervising. Analysis of student intervention usage indicated an average of 20 minutes per session using the Read Naturally® software, as opposed to the targeted 30 minutes per session.
Comparison Group
Comparison group students continued to receive their classroom’s normal reading instruction, with no supplemental fluency instruction. During the class time designated for Read Naturally® instruction, comparison group students engaged in non-reading related activities.
Outcome descriptions
In the alphabetics domain, the authors used the WRMT-R Word Identification subtest and the TOWRE. In the reading fluency domain, three outcome measures were included: the GORT-4 Fluency subtest, the GORT-4 Accuracy subtest, and a CBM-R based on three passages from the DIBELS assessment, selected by the authors. In the comprehension domain, the authors used the GORT-4 Comprehension subtest and the WRMT-R Passage Comprehension subtest. Baseline measures were collected approximately two weeks prior to the beginning of the intervention, and outcomes were collected approximately one week after the conclusion of the intervention. For a more detailed description of these outcome measures, see Appendix B.
Support for implementation
Each teacher attended a 6-hour Read Naturally® training session, which included lecture sessions and software practice. Intervention integrity checklists, produced by the developer for both students and teachers, were used to assess and evaluate the implementation of the intervention. Bi-monthly classroom observations were also used to assess implementation fidelity.
Additional Sources
In the case of multiple manuscripts that report on one study, the WWC selects one manuscript as the primary citation and lists other manuscripts that describe the study as additional sources.
-
Read Naturally, Inc. (n.d.). Case 2: University of Minnesota study, Minneapolis, Minn. Retrieved from http://www.readnaturally.com
-
Read Naturally, Inc. (n.d.). University study of Read Naturally gets top rating from National Center on Response-to-Intervention. Retrieved from http://www.readnaturally.com
Read Naturally Intervention Report - Adolescent Literacy
Review Details
Reviewed: March 2013
- The study is ineligible for review because it does not use a sample within the age or grade range specified in the protocol.
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Please see the WWC summary of evidence for Read Naturally.
Findings
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Sample Characteristics
Study sample characteristics were not reported.Additional Sources
In the case of multiple manuscripts that report on one study, the WWC selects one manuscript as the primary citation and lists other manuscripts that describe the study as additional sources.
-
Read Naturally. (n. d.). Case 2: University of Minnesota study, Minneapolis, Minn. Retrieved November 5, 2009, from http://www.readnaturally.com/approach/case2.htm.
Read Naturally Intervention Report - Students with a Specific Learning Disability
Review Details
Reviewed: July 2010
- The study is ineligible for review because it does not use a sample aligned with the protocol.
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Please see the WWC summary of evidence for Read Naturally.
Findings
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Sample Characteristics
Study sample characteristics were not reported.An indicator of the effect of the intervention, the improvement index can be interpreted as the expected change in percentile rank for an average comparison group student if that student had received the intervention.
For more, please see the WWC Glossary entry for improvement index.
An outcome is the knowledge, skills, and attitudes that are attained as a result of an activity. An outcome measures is an instrument, device, or method that provides data on the outcome.
A finding that is included in the effectiveness rating. Excluded findings may include subgroups and subscales.
The sample on which the analysis was conducted.
The group to which the intervention group is compared, which may include a different intervention, business as usual, or no services.
The timing of the post-intervention outcome measure.
The number of students included in the analysis.
The mean score of students in the intervention group.
The mean score of students in the comparison group.
The WWC considers a finding to be statistically significant if the likelihood that the finding is due to chance alone, rather than a real difference, is less than five percent.
The WWC reviews studies for WWC products, Department of Education grant competitions, and IES performance measures.
The name and version of the document used to guide the review of the study.
The version of the WWC design standards used to guide the review of the study.
The result of the WWC assessment of the study. The rating is based on the strength of evidence of the effectiveness of the intervention. Studies are given a rating of Meets WWC Design Standards without Reservations, Meets WWC Design Standards with Reservations, or >Does Not Meet WWC Design Standards.
A related publication that was reviewed alongside the main study of interest.
Study findings for this report.
Based on the direction, magnitude, and statistical significance of the findings within a domain, the WWC characterizes the findings from a study as one of the following: statistically significant positive effects, substantively important positive effects, indeterminate effects, substantively important negative effects, and statistically significant negative effects. For more, please see the WWC Handbook.
The WWC may review studies for multiple purposes, including different reports and re-reviews using updated standards. Each WWC review of this study is listed in the dropdown. Details on any review may be accessed by making a selection from the drop down list.
Tier 1 Strong indicates strong evidence of effectiveness,
Tier 2 Moderate indicates moderate evidence of effectiveness, and
Tier 3 Promising indicates promising evidence of effectiveness,
as defined in the
non-regulatory guidance for ESSA
and the regulations for ED discretionary grants (EDGAR Part 77).