A study between Voyager and control schools in Orange County, Florida 2002-2003.
Hecht, S. A. (2003). Unpublished.
-
examining213Students, gradeK
Practice Guide
Review Details
Reviewed: March 2023
- Practice Guide (findings for Voyager Universal Literacy System)
- Quasi-Experimental Design
- Meets WWC standards with reservations because it uses a quasi-experimental design in which the analytic intervention and comparison groups satisfy the baseline equivalence requirement.
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Findings
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Letter Sound Knowledge |
Voyager Universal Literacy System vs. Business as usual |
5 Months |
Full sample;
|
26.00 |
23.80 |
No |
-- | |
|
Letter Name Knowledge |
Voyager Universal Literacy System vs. Business as usual |
5 Months |
Full sample;
|
26.20 |
25.20 |
No |
-- |
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Woodcock Reading Mastery Test - Revised: Word Analysis |
Voyager Universal Literacy System vs. Business as usual |
5 Months |
Full sample;
|
5.30 |
4.80 |
No |
-- | |
|
Woodcock Reading Mastery Test - Revised: Word Identification |
Voyager Universal Literacy System vs. Business as usual |
5 Months |
Full sample;
|
9.40 |
10.40 |
No |
-- |
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Sample Characteristics
Characteristics of study sample as reported by study author.
-
- B
- A
- C
- D
- E
- F
- G
- I
- H
- J
- K
- L
- P
- M
- N
- O
- Q
- R
- S
- V
- U
- T
- W
- X
- Z
- Y
- a
- h
- i
- b
- d
- e
- f
- c
- g
- j
- k
- l
- m
- n
- o
- p
- q
- r
- s
- t
- u
- v
- x
- w
- y
Florida
Study Details
Setting
The study took place in four schools in Orange County, Florida.
Intervention Group
Teachers implemented the Voyager Universal Literacy System. The intervention lasted for about 5 months.
Comparison Group
Teachers taught their regular lessons.
Voyager Universal Literacy System® Intervention Report - Beginning Reading
Review Details
Reviewed: August 2007
- Quasi-Experimental Design
- Meets WWC standards with reservations
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Please see the WWC summary of evidence for Voyager Universal Literacy System®.
Findings
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Comprehensive Test of Phonological Processing (CTOPP): Segmenting Words subtest |
Voyager Universal Literacy System® vs. unknown |
Pretest |
Kindergarten;
|
7.20 |
4.60 |
Yes |
|
|
|
Letter Sound Knowledge |
Voyager Universal Literacy System® vs. unknown |
Pretest |
Kindergarten;
|
26.00 |
23.80 |
Yes |
|
|
|
Letter name knowledge |
Voyager Universal Literacy System® vs. unknown |
Pretest |
Kindergarten;
|
26.20 |
25.20 |
No |
-- | |
|
Blending |
Voyager Universal Literacy System® vs. unknown |
Pretest |
Kindergarten;
|
9.90 |
9.20 |
No |
-- | |
|
Woodcock Reading Mastery Test (WRMT): Word Attack subtest |
Voyager Universal Literacy System® vs. unknown |
Pretest |
Kindergarten;
|
5.30 |
4.80 |
No |
-- | |
|
Dynamic Indicators of Basic Early Literacy Skills (DIBELS): Nonsense Words Fluency subtest |
Voyager Universal Literacy System® vs. unknown |
Pretest |
Kindergarten;
|
29.30 |
30.60 |
No |
-- | |
|
Woodcock Reading Mastery Test (WRMT): Word Identification subtest |
Voyager Universal Literacy System® vs. unknown |
Pretest |
Kindergarten;
|
9.40 |
10.40 |
No |
-- | |
|
Concepts About Print Test |
Voyager Universal Literacy System® vs. unknown |
Pretest |
Kindergarten;
|
12.80 |
13.50 |
No |
-- | |
|
Comprehensive Test of Phonological Processing (CTOPP): Phoneme Elision subtest |
Voyager Universal Literacy System® vs. unknown |
Pretest |
Kindergarten;
|
3.20 |
3.80 |
No |
-- |
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Stanford Binet: Expressive Vocabulary |
Voyager Universal Literacy System® vs. unknown |
Pretest |
Kindergarten;
|
14.30 |
17.00 |
Yes |
|
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Sample Characteristics
Characteristics of study sample as reported by study author.
-
- B
- A
- C
- D
- E
- F
- G
- I
- H
- J
- K
- L
- P
- M
- N
- O
- Q
- R
- S
- V
- U
- T
- W
- X
- Z
- Y
- a
- h
- i
- b
- d
- e
- f
- c
- g
- j
- k
- l
- m
- n
- o
- p
- q
- r
- s
- t
- u
- v
- x
- w
- y
Florida
-
Race Black 80%
Study Details
Setting
Four schools in Orange County, Florida.
Study sample
The study included 429 economically disadvantaged Kindergarten students at two intervention and two comparison schools. The initial study design called for analysis of outcomes for intervention and comparison classrooms within schools and across the four schools. However, the study authors did not report findings on the within school comparisons due to poor implementation of the intervention. The analysis sample for the between school comparisons included 213 students. This left 213 students in the between schools study: 101 students in the intervention group and 112 students in the comparison group. Over 80% of students were African-American, and approximately 80% qualified for free or reduced price lunches.
Intervention Group
The Voyager Universal Literacy System® program was used as the core reading program in intervention classrooms for five months. No other information about implementation of the program is given.
Comparison Group
The two schools in the comparison group used their school’s existing curriculum, either Houghton Mifflin or Success for All. No other information about instruction for the comparison group was given.
Outcome descriptions
Hecht (2003) used the Comprehensive Test of Phonological Awareness (CTOPP) Elision, Segmenting, and Blending subtests as well as the Dynamic Indicators of Basic Early Literacy Skills (DIBELS) test of Nonsense Word Fluency. Letter Name Knowledge, Letter Sound Knowledge, and Concepts about Print measures were also used. In addition, the Woodcock Reading Mastery Test-Revised (WRMT-R) Word Identification and Word Analysis subtests were used as well as the Stanford-Binet Intelligence Scale (4th Edition) Vocabulary subtest. Spelling subtests of the Wide Range Achievement Test were administered, but are beyond the scope of this review. (See Appendix A2.1–2.2 for more detailed descriptions of outcome measures.)
Support for implementation
No information was given about teacher training in this study.
An indicator of the effect of the intervention, the improvement index can be interpreted as the expected change in percentile rank for an average comparison group student if that student had received the intervention.
For more, please see the WWC Glossary entry for improvement index.
An outcome is the knowledge, skills, and attitudes that are attained as a result of an activity. An outcome measures is an instrument, device, or method that provides data on the outcome.
A finding that is included in the effectiveness rating. Excluded findings may include subgroups and subscales.
The sample on which the analysis was conducted.
The group to which the intervention group is compared, which may include a different intervention, business as usual, or no services.
The timing of the post-intervention outcome measure.
The number of students included in the analysis.
The mean score of students in the intervention group.
The mean score of students in the comparison group.
The WWC considers a finding to be statistically significant if the likelihood that the finding is due to chance alone, rather than a real difference, is less than five percent.
The WWC reviews studies for WWC products, Department of Education grant competitions, and IES performance measures.
The name and version of the document used to guide the review of the study.
The version of the WWC design standards used to guide the review of the study.
The result of the WWC assessment of the study. The rating is based on the strength of evidence of the effectiveness of the intervention. Studies are given a rating of Meets WWC Design Standards without Reservations, Meets WWC Design Standards with Reservations, or >Does Not Meet WWC Design Standards.
A related publication that was reviewed alongside the main study of interest.
Study findings for this report.
Based on the direction, magnitude, and statistical significance of the findings within a domain, the WWC characterizes the findings from a study as one of the following: statistically significant positive effects, substantively important positive effects, indeterminate effects, substantively important negative effects, and statistically significant negative effects. For more, please see the WWC Handbook.
The WWC may review studies for multiple purposes, including different reports and re-reviews using updated standards. Each WWC review of this study is listed in the dropdown. Details on any review may be accessed by making a selection from the drop down list.
Tier 1 Strong indicates strong evidence of effectiveness,
Tier 2 Moderate indicates moderate evidence of effectiveness, and
Tier 3 Promising indicates promising evidence of effectiveness,
as defined in the
non-regulatory guidance for ESSA
and the regulations for ED discretionary grants (EDGAR Part 77).