Appendix A1.1 Study characteristics: Scientific Learning Corporation, 2004 (randomized controlled trial)
| Characteristic | Description |
|---|---|
| Study citation | Scientific Learning Corporation. (2004). Improved language skills by children with low reading performance who used Fast ForWord Language: MAPS for learning. MAPS for Learning, 3 (1), 1–13. |
| Participants | Four-hundred fifty-two academically at-risk students in kindergarten through fifth grade were randomly assigned, within grade and gender, to the treatment group (n=288) or the control group (n=164) at a 1.74: 1 ratio. The original study stated that 85 of the students were categorized as English language learners (53 in the treatment group and 32 in the control group).1 Follow-up correspondence with the developer of the intervention revealed that 98 students were identified as English language learners. Nine English language learning students were also identified as receiving services for special education and were not included in the analysis. Of the remaining 89 students, five from the comparison group and three from the intervention group had incomplete data. There were complete data for a total of 81 English language learning students (52 in the treatment group and 29 in the comparison group). |
| Setting | The study was conducted in nine school districts in the United States. Elementary school teachers identified students as at-risk based on student difficulties with reading and language arts. |
| Intervention | Participants used Fast ForWord Language, an adaptive computer-based training program based on acoustically modified speech and language training. Students were presented with seven exercises as computer games. Exercises began with acoustic reception and moved to more complex skills in syntactic and semantic aspects of language. The difficulty of each task was continuously adapted so that participants would get about 80% of the items correct. Participants used the program about 100 minutes a day, five days a week over an average of 39 days. Each participant worked on multiple 20-minute Fast ForWord Language training exercises during each session. |
| Comparison | The control group used their regular curriculum. No information about the regular curriculum was provided. |
| Primary outcomes and measurement | The study measure in the English language development domain was the Test of Auditory Comprehension of Language Revised Edition (TACL-R). (See Appendix A2.2 for a more detailed description of this outcome measure.) The study measure in the phonological awareness domain was the Phonological Awareness Test (PAT). The WWC review of English language learning interventions does not investigate phonological awareness, so results for this domain are not included in this report. |
| Teacher training | No information about teacher training was provided. However, the teachers were speech-language or educational professionals. |
| 1 Means and standard deviations for this subsample were not provided in the original study, so the WWC contacted the developer to obtain this information. | |
Appendix A1.2 Study characteristics: Troia, 2004 (quasi-experimental design)
| Characteristic | Description |
|---|---|
| Study citation | Troia, G. A. (2004). Migrant students with limited English proficiency: Can Fast ForWord Language make a difference in their language skills and academic achievement? Remedial and Special Education, 25 (6), 353–366. |
| Participants | This study included 191 English language learning first- through sixth-grade students. Students from four schools were randomly assigned to the treatment or control group, while three schools created a matched comparison group for the treatment groups. The WWC could not separate effects for groups formed on a random and non-random (that is, those that used matching without random assignment) basis, so the WWC rated this study as a quasi-experimental design. All participants were migrant students (lived in the Unted States for six years on average) whose native language was Spanish, and their average age was 9.49 years old. There were slightly more boys (53%) than girls (47%) in the study and slightly more treatment group students (n=99) than comparison group students (n=92). A total of 168 students (90 treatment, 78 control) completed the Letter-Word posttest, and 167 students (89 treatment, 78 control) completed the Word Attack posttest. |
| Setting | The study was conducted in seven schools from five rural school districts in central Washington state. This region has many agricultural communities that employ seasonal migrant workers, so the schools enroll a large number of migrant students. |
| Intervention | Participants used Fast ForWord Language, an adaptive computer-based training program based on acoustically modified speech and language training. Students were presented with seven exercises as computer games. Exercises began with acoustic reception and moved on to more complex skills in syntactic and semantic aspects of language. The difficulty of each task was continuously adapted so that participants would get about 80% of the items correct. Participants used the Fast ForWord Language program about 100 minutes a day, five days a week for a minimum of four weeks. Each participant worked on multiple 20-minute Fast ForWord Language training exercises during each session. |
| Comparison | The comparison group used their regular curriculum. No information about the regular curriculum was provided. |
| Primary outcomes and measurement | The study measures in the reading achievement domain included the Woodcock-Johnson Psycho-Educational Battery-Revised Word Identification and Word Attack subtests. The study measures in the English language development domain were the Woodcock-Muñoz Language Survey, the Language Assessment Scales, and the Oral and Written Language Survey.1 (See Appendix A2.1 for a more detailed description of outcome measures.) The study measures in the phonological awareness domain included the Lindamood Auditory Conceptualization (LAC) Test, the Sound Blending subtest of the Woodcock-Johnson Psycho-Educational Battery-Revised, and the Experimental Rhyming and Segmentation Tests. The study measure in the social skill development domain was the Social Skills Rating System (SSRS). The WWC review of English language learning interventions does not investigate phonological awareness or social skill development, so results for these domains are not included in this report. |
| Teacher training | No information about teacher training was provided, except that teachers were instructed not to provide the children with the correct responses. The primary role of the teacher present during the intervention was to troubleshoot any technical difficulties. |
| 1 Two findings (the Language Assessment Scales and the Oral and Written Language Survey) are not included in Appendix A3 because of differential attrition rates between the intervention group and the comparison group. The Woodcock-Muñoz Language Survey was administered in those schools where the Language Assessment Scale was not used (20 students). The WWC could not determine whether this group was representative of the larger sample, so did not present these outcomes in these appendices. | |
Appendix A2.1 Outcome measures in the reading achievement domain
| Outcome measure | Description |
|---|---|
| Woodcock-Johnson Psycho-Educational Battery Revised (WJ-R): Letter-Word Identification | The Woodcock-Johnson Psycho-Educational Battery-Revised (Woodcock & Johnson, 1990) is made up of many subtests. The Letter-Word Identification subtest assesses basic reading skills by having participants read aloud as many letters and real words as possible in five seconds (as cited in Troia, 2004). |
| WJ-R: Word Attack | The Word Attack subtest assesses basic reading skills by having participants read aloud a list of nonsense words (as cited in Troia, 2004). |
Appendix A2.2 Outcome measures in the English language development domain1
| Outcome measure | Description |
|---|---|
| Test of Auditory Comprehension of Language Revised Edition (TACL-R)2 | The TACL-R (Carrow-Woolfolk, 1985) assesses the comprehension of spoken language and consists of three subtests, including word classes and relations, grammatical morphemes, and elaborated sentences (as cited in Scientific Learning Corporation, 2004). |
| 1 The Scientific Learning Corporation included outcome measures on phonological awareness, but these were not included in this review because they were judged to fall outside the reading achievement and English language development domains. 2 Although limited information was provided on the TACL-R in the article by the Scientific Learning Corporation, the TACL-3 was normed on a standardization sample of more than 1,000 children. This information was found at www.proedinc.com/Scripts/prodView.asp?idProduct=1571. |
|
Appendix A3.1 Summary of study findings included in the rating for the reading achievement domain1
| Author's findings from the study | ||||||||
|---|---|---|---|---|---|---|---|---|
| Mean outcome (standard deviation2) | WWC calculations | |||||||
| Outcome measure | Study sample | Sample size (students) | Fast ForWord Language group | Comparison group | Mean difference3 (Fast ForWord Language – comparison) | Effect size4 | Statistical significance5 (at α= 0.05) | Improvement index6 |
| Troia, 2004 (quasi-experimental design) | ||||||||
| Woodcock-Johnson Psycho-Educational Battery Revised (WJ-R): Letter-Word Identification | Grades 1–6 | 168 | 92.47 (14.89) | 90.59 (18.02) | 1.88 | 0.11 | ns | +5 |
| WJ-R: Word Attack | Grades 1–6 | 167 | 94.64 (17.25) | 94.46 (15.85) | 0.18 | 0.01 | ns | 0 |
| Domain average7 for reading achievement | 0.06 | ns | +3 | |||||
|
ns = not statistically significant 1 This appendix reports findings considered for the effectiveness rating and the improvement index. Two findings (the Language Assessment Scale and the Oral and Written Language Survey) were not reported because of problematic attrition rates.2 The standard deviation across all students in each group shows how dispersed the participants' outcomes are: a smaller standard deviation on a given measure would indicate that participants had more similar outcomes. 3 Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the comparison group. 4 For an explanation of the effect size calculation, please see the Technical Details of WWC-Conducted Computations. 5 Statistical significance is the probability that the difference between groups is a result of chance rather than a real difference between the groups. The level of statistical significance was calculated by the WWC and, where necessary, corrects for clustering within classrooms or schools and for multiple comparisons. For an explanation about the clustering correction, see the WWC Tutorial on Mismatch. See the Technical Details of WWC-Conducted Computations for the formulas the WWC used to calculate statistical significance. In the case of Fast ForWord Language, no corrections for clustering or multiple comparisons were needed. 6 The improvement index represents the difference between the percentile rank of the average student in the intervention condition and that of the average student in the comparison condition. The improvement index can take on values between -50 and +50, with positive numbers denoting favorable results. 7 This row provides the study average, which is also the domain average in this case. The WWC-computed domain average effect size is a simple average rounded to two decimal places. The domain improvement index is calculated from the average effect size. |
||||||||
Appendix A3.2 Summary of study findings included in the rating for the English language development domain1
| Author's findings from the study | ||||||||
|---|---|---|---|---|---|---|---|---|
| Mean outcome (standard deviation2) | WWC calculations | |||||||
| Outcome measure | Study sample | Sample size (students) | Fast ForWord Language group | Comparison group | Mean difference3 (Fast ForWord Language – comparison) | Effect size4 | Statistical significance5 (at α= 0.05) | Improvement index6 |
| Scientific Learning Corporation, 2004 (randomized controlled trial)7 | ||||||||
| Test of Auditory Comprehension of Language-Revised (TACL-R) | Grades K–5 | 81 | 45.21 (8.33) | 37.93 (8.07) | 7.28 | 0.88 | Statistically significant | +31 |
| Domain average8 for English language development | 0.88 | Statistically significant | +31 | |||||
|
ns = not statistically significant 1 This appendix reports findings considered for the effectiveness rating and the improvement index.2 The standard deviation across all students in each group shows how dispersed the participants' outcomes are: a smaller standard deviation on a given measure would indicate that participants had more similar outcomes. 3 Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the comparison group. 4 For an explanation of the effect size calculation, please see the Technical Details of WWC-Conducted Computations. 5 Statistical significance is the probability that the difference between groups is a result of chance rather than a real difference between the groups. The level of statistical significance was calculated by the WWC and, where necessary, corrects for clustering within classrooms or schools and for multiple comparisons. For an explanation about the clustering correction, see the WWC Tutorial on Mismatch. See the Technical Details of WWC-Conducted Computations for the formulas the WWC used to calculate statistical significance. In the case of Fast ForWord Language, no corrections for clustering or multiple comparisons were needed. 6 The improvement index represents the difference between the percentile rank of the average student in the intervention condition and that of the average student in the comparison condition. The improvement index can take on values between -50 and +50, with positive numbers denoting favorable results. 7 The article did not include subsample data specifically for English language learning students, so the WWC obtained these data from the study author. 8 This row provides the study average, which is also the domain average in this case. The WWC-computed domain average effect size is a simple average rounded to two decimal places. The domain improvement index is calculated from the average effect size. |
||||||||
Appendix A4.1 Fast ForWord Language rating for the reading achievement domain
The WWC rates interventions as positive, potentially positive, mixed, no discernible effects, potentially negative, or negative.1
For the outcome domain of reading achievement, the WWC rated Fast ForWord Language as having no discernible effects. It did not meet the criteria for positive effects, because it had only one study. In addition, it did not meet the criteria for the other ratings (potentially positive effects, mixed effects, potentially negative effects, and negative effects) because the single study that met WWC standards did not show statistically significant or substantively important effects.
| Rating received |
|---|
|
No discernible effects: No affirmative evidence of effects.
|
| Other ratings considered |
|
Positive effects: Strong evidence of a positive effect with no overriding contrary evidence.
|
|
Potentially positive effects: Evidence of a positive effect with no overriding contrary evidence.
|
|
Mixed effects: Evidence of inconsistent effects as demonstrated through EITHER of the following:
|
|
Potentially negative effects: Evidence of a negative effect with no overriding contrary evidence
|
|
Negative effects: Strong evidence of a negative effect with no overriding contrary evidence.
|
|
1 For rating purposes, the WWC considers the statistical significance of individual outcomes and the domain level effect. The WWC also considers the size of the domain level effect for ratings of potentially positive effects. See the WWC Intervention Rating Scheme for a complete description. |
Appendix A5.1 Fast ForWord Language rating for the English language development domain
The WWC rates interventions as positive, potentially positive, mixed, no discernible effects, potentially negative, or negative.1
For the outcome domain of English language development, the WWC rated Fast ForWord Language as having potentially positive effects. It did not meet the criteria for positive effects, because it had only one study. The remaining ratings (mixed effects, no discernible effects, potentially negative effects, and negative effects) were not considered because Fast ForWord Language was assigned the highest applicable rating.
| Rating received |
|---|
|
Potentially positive effects: Evidence of a positive effect with no overriding contrary evidence.
|
| Other ratings considered |
|
Positive effects: Strong evidence of a positive effect with no overriding contrary evidence.
|
|
1 For rating purposes, the WWC considers the statistical significance of individual outcomes and the domain level effect. The WWC also considers the size of the domain level effect for ratings of potentially positive effects. See the WWC Intervention Rating Scheme for a complete description. |