Alamprese, Judith A.; MacArthur, Charles A.; Price, Cristofer; Knight, Deborah (2011). Journal of Research on Educational Effectiveness, v4 n2 p154-172. Retrieved from: https://eric.ed.gov/?id=EJ920176
Reviewed: July 2020
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Woodcock-Johnson Revised Test of Achievement (WJ-R ACH): Letter-Word Identification |
Adult Education vs. Other intervention |
2 Months |
QED sample;
|
2.44 |
-2.18 |
No |
-- | |
|
Wide Range Achievement Test-Revision 3 (WRAT-3): reading subtest |
Adult Education vs. Other intervention |
2 Months |
QED sample;
|
5.45 |
2.63 |
No |
-- | |
|
Woodcock Reading Mastery Test Revised (WRMT-R): Word Attack subtest |
Adult Education vs. Business as usual |
2 Months |
RCT sample;
|
5.49 |
2.54 |
No |
-- | |
|
Letter-Sound Survey |
Adult Education vs. Other intervention |
2 Months |
QED sample;
|
1.74 |
0.60 |
No |
-- | |
|
Wide Range Achievement Test- Third Edition (WRAT-3): Spelling subtest |
Adult Education vs. Business as usual |
2 Months |
RCT sample;
|
1.71 |
0.47 |
No |
-- | |
|
Wide Range Achievement Test-Revision 3 (WRAT-3): reading subtest |
Adult Education vs. Business as usual |
2 Months |
RCT sample;
|
5.45 |
4.06 |
No |
-- | |
|
Test of Word Reading Efficiency (TOWRE): Sight Word Efficiency subtest |
Adult Education vs. Business as usual |
2 Months |
RCT sample;
|
0.99 |
-0.44 |
No |
-- | |
|
Woodcock Reading Mastery Test Revised (WRMT-R): Word Attack subtest |
Adult Education vs. Other intervention |
2 Months |
QED sample;
|
5.49 |
3.89 |
No |
-- | |
|
Developmental Spelling Test |
Adult Education vs. Other intervention |
2 Months |
QED sample;
|
0.78 |
0.24 |
No |
-- | |
|
Test of Word Reading Efficiency (TOWRE): Phonemic Decoding Efficiency subtest |
Adult Education vs. Business as usual |
2 Months |
RCT sample;
|
0.80 |
-0.09 |
No |
-- | |
|
Developmental Spelling Test |
Adult Education vs. Business as usual |
2 Months |
RCT sample;
|
0.78 |
0.46 |
No |
-- | |
|
Woodcock-Johnson Revised Test of Achievement (WJ-R ACH): Letter-Word Identification |
Adult Education vs. Business as usual |
2 Months |
RCT sample;
|
2.44 |
2.15 |
No |
-- | |
|
Letter-Sound Survey |
Adult Education vs. Business as usual |
2 Months |
RCT sample;
|
1.74 |
1.58 |
No |
-- |
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Nelson comprehension test |
Adult Education vs. Business as usual |
2 Months |
RCT sample;
|
2.18 |
2.86 |
No |
-- | |
|
Nelson Word Meaning Test |
Adult Education vs. Business as usual |
2 Months |
RCT sample;
|
1.29 |
4.84 |
No |
-- |
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Passage Reading Test |
Adult Education vs. Business as usual |
2 Months |
RCT sample;
|
1.89 |
2.09 |
No |
-- |
Characteristics of study sample as reported by study author.
| Asian | 15% | |
| Black | 20% | |
| Other or unknown | 30% | |
| White | 35% |
| Hispanic | 24% | |
| Not Hispanic or Latino | 76% |
The study took place in 23 adult literacy programs located in 12 states. These programs included 71 reading classes and 34 instructors.
The initial sample consisted of 349 learners. These learners attended an adult literacy program that provided class-based instruction to English-speaking adults at the intermediate level. Sixty-six percent of the learners were female. The race/ethnicity distribution of learners was 35 percent White, 24 percent Hispanic, 20 percent Black, 15 percent Asian, and 6 percent in an unspecified other category. Thirty-five percent were born and educated outside of the United States. Sixty-three percent had low incomes based on the poverty threshold of $12,000 annual salary. Thirty-one percent were identified as having a learning disability.
The Making Sense of Decoding and Spelling (MSDS) curriculum was designed specifically for adult learners and used to teach decoding and spelling. Instruction lasted approximately 30 weeks, with classes meeting from one to five days per week. The curriculum includes a review of alphabetic decoding skills and principles and teaches a strategy for decoding multisyllabic words. Instruction was primarily delivered to the whole group in scripted lessons, but lessons include paired- and individual-learner reading practice designed to improve reading speed. Each lesson includes progress monitoring assessments. Instructors were given lesson plans with examples and presentation materials.
There were two comparison groups. In both groups, classes lasted about 30 weeks and met one to five days per week. The authors refer to the first group as the control condition, because a lottery was used to construct this group. Instructors in these programs continued to use their existing reading instruction. These classes did not use a published scope and sequence. Teachers typically included some decoding, but placed more emphasis on spelling, vocabulary, and comprehension. The second group received explicit instruction on reading and spelling that was based on an adapted curriculum. The structured decoding curricula were designed for K–3 students, but were adapted for use with adults. Additional information about the curricula was not provided.
The researchers measured attendance and hours of instruction for each of the study groups. The mean number of hours of instruction was approximately 65 among those using a K–3 curriculum adapted for adult learners, 60 among those using existing reading instruction, and 50 among those using the MSDS curriculum. Attendance rates were 57 percent, 51 percent, and 55 percent, respectively. The researchers also looked at whether MSDS was implemented as intended by calculating the percentage of lessons taught, the number of hours of study curriculum offered to learners, and the level of fidelity to the scripted lessons based on classroom observations. The median percentage of lessons taught was 92 percent; the mean total hours of study curriculum was 27; and the mean fidelity scores were 2.18 on a scale of 0 to 3 on the classroom observation form, where a score of 3 means all segments were taught as scripted.
An indicator of the effect of the intervention, the improvement index can be interpreted as the expected change in percentile rank for an average comparison group student if that student had received the intervention.
For more, please see the WWC Glossary entry for improvement index.
An outcome is the knowledge, skills, and attitudes that are attained as a result of an activity. An outcome measures is an instrument, device, or method that provides data on the outcome.
A finding that is included in the effectiveness rating. Excluded findings may include subgroups and subscales.
The sample on which the analysis was conducted.
The group to which the intervention group is compared, which may include a different intervention, business as usual, or no services.
The timing of the post-intervention outcome measure.
The number of students included in the analysis.
The mean score of students in the intervention group.
The mean score of students in the comparison group.
The WWC considers a finding to be statistically significant if the likelihood that the finding is due to chance alone, rather than a real difference, is less than five percent.
The WWC reviews studies for WWC products, Department of Education grant competitions, and IES performance measures.
The name and version of the document used to guide the review of the study.
The version of the WWC design standards used to guide the review of the study.
The result of the WWC assessment of the study. The rating is based on the strength of evidence of the effectiveness of the intervention. Studies are given a rating of Meets WWC Design Standards without Reservations, Meets WWC Design Standards with Reservations, or >Does Not Meet WWC Design Standards.
A related publication that was reviewed alongside the main study of interest.
Study findings for this report.
Based on the direction, magnitude, and statistical significance of the findings within a domain, the WWC characterizes the findings from a study as one of the following: statistically significant positive effects, substantively important positive effects, indeterminate effects, substantively important negative effects, and statistically significant negative effects. For more, please see the WWC Handbook.
The WWC may review studies for multiple purposes, including different reports and re-reviews using updated standards. Each WWC review of this study is listed in the dropdown. Details on any review may be accessed by making a selection from the drop down list.
Tier 1 Strong indicates strong evidence of effectiveness,
Tier 2 Moderate indicates moderate evidence of effectiveness, and
Tier 3 Promising indicates promising evidence of effectiveness,
as defined in the
non-regulatory guidance for ESSA
and the regulations for ED discretionary grants (EDGAR Part 77).
Resources
Newsflash
Contact Us
Facebook
Twitter
YouTube