Appendix A1 Study characteristics
| Characteristic | Description |
|---|---|
| Study citation | Torgesen, J., Wagner, R., Rashotte, C., & Herron, J. (2003). Summary of outcomes from first grade study with Read, Write, and Type and Auditory Discrimination in Depth Instruction and software with at-risk children (FCRR Techn. Rep. No. 2). Retrieved from Florida Center for Reading Research Web site: http://www.fcrr.org/TechnicalReports/RWTfullrept.pdf |
| Participants | The study included 150 low-achieving first grade students in five elementary schools. All students scored in the lowest 35% on a letter-sound knowledge measure and were considered to be most at-risk for developing reading problems. At two schools, students were randomly assigned either to Read, Write & Type!™ (RWT) (n = 16) or to Auditory Discrimination in Depth® (ADD) (n = 16). At three schools, students were randomly assigned to RWT (n = 38), ADD (n = 38), or a comparison group (n = 42) (J. K. Torgesen, personal communication, September 7, 2006). Two students attrited from each of the RWT and ADD groups, and one student attrited from the comparison group. The final analysis samples included 52 RWT and 52 ADD students located at five schools; and 36 RWT, 36 ADD, and 41 comparison students located at three schools. Approximately 34% of the sample were minority children (primarily African-American). Approximately 35% of the sample received free/reduced lunch, but students ranged in terms of their socio-economic status. |
| Setting | The study took place in five elementary schools (location unknown). |
| Intervention | Students assigned to the RWT program received services from October through May. Working in groups of three, the students had four 50-minute sessions per week. A trained RWT teacher devoted approximately half of each session to direct instruction, leading students in warm-up activities outlined in the teacher's manual. For the remainder of the session, students worked individually on the computer practicing the same skills, with the teacher in a support role. The teacher occasionally provided further individualized instruction if a child encountered specific difficulties. The computer program emphasizes phonological awareness, letter sound correspondence, and phonemic decoding in the context of children expressing themselves in written language. |
| Comparison | ADD students received instruction in the same format and duration as the RWT students, but the type of activities differed. The ADD program focuses on developing phonological awareness and phonemic decoding skills through practicing word reading skills out of context, reading phonetically controlled text, and completing computer activities. The regular instruction comparison groups continued receiving the regular instruction and support typically available to them (J. K. Torgesen, personal communication, September 7, 2006). Two of the three schools with regular instruction comparison groups used Open Court's Collections for Young Scholars as the whole-class reading curriculum. |
| Primary outcomes and measurement | The authors assessed students at the end of the study period using a battery of tests. All children in the sample pool were given the phoneme blending, phoneme elision, and phoneme segmenting subtests of the Comprehensive Test of Phonological Processes (CTOPP), and the word attack, word identification, and passage comprehension subtests of the Woodcock Reading Mastery Test. Students in the study were also given the vocabulary subtest of the Stanford Binet Intelligence Scale, which the authors used as a proxy measure for verbal IQ. Other outcomes were reported in the study, but not included in this review either because they were outside the scope of the beginning reading review (developmental spelling, probability of reading disability) or because sufficient information on the measure name, description, or validity and reliability was not reported (word efficiency and non-word efficiency) (see Appendix A2.1–2.2 for more detailed descriptions of outcome measures). |
| Teacher training | Teachers received training for the intervention, but no information was provided on the nature of the training. |
Appendix A2.1 Outcome measures in the alphabetics domain by construct
| Outcome measure | Description |
|---|---|
| Phonological awareness | |
| Comprehensive Test of Phonological Processes (CTOPP): Phoneme Blending Subtest | The phoneme blending subtest measures the child's ability to blend separately presented sounds together to form words. This is a standardized test (as cited in Torgesen et al., 2003). |
| Comprehensive Test of Phonological Processes (CTOPP): Phoneme Elision Subtest | The phoneme elision subtest measures the child's ability to manipulate sounds in words. This is a standardized test (as cited in Torgesen et al., 2003). |
| Comprehensive Test of Phonological Processes (CTOPP): Phoneme Segmenting Subtest | The phoneme segmenting subtest measures the child's ability to isolate and pronounce the sounds in words. This is a standardized test (as cited in Torgesen et al., 2003). |
| Phonics | |
| Woodcock Reading Mastery Test: Word Identification Subtest | The word identification subtest is a measure of word reading vocabulary in which the child reads list of words of increasing difficulty. This is a standardized test (as cited in Torgesen et al., 2003). |
| Woodcock Reading Mastery Test: Word Attack Subtest | The word attack subtest is a measure of phonemic reading ability in which the child reads non-words. This is a standardized test (as cited in Torgesen et al., 2003). |
Appendix A2.2 Outcome measures in the comprehension domain by construct
| Outcome measure | Description |
|---|---|
| Comprehension | |
| Woodcock Reading Mastery Test: Passage Comprehension Subtest | The passage comprehension subtest measures the child's ability to comprehend the meaning of short passages. This is a standardized test (as cited in Torgesen et al. 2003). |
| Vocabulary | |
| Estimated Verbal IQ | The measure is based on the vocabulary subtest of the Stanford Binet Intelligence Scale. The vocabulary subtest consists of both providing names of pictures and definitions of words. This is a standardized test (as cited in Torgesen et al., 2003). |
Appendix A3.1 Summary of findings for the alphabetics domain by construct1
| Author's findings from the study | ||||||||
|---|---|---|---|---|---|---|---|---|
| Mean outcome (standard deviation2) | WWC calculations | |||||||
| Outcome measure | Study sample | Sample size (students) | Read, Write & Type!™ group | Comparison group | Mean difference3 (RWT – comparison) | Effect size4 | Statistical significance5 (at α= 0.05) | Improvement index6 |
| Construct: Phonological awareness | ||||||||
| Torgesen et al., 2003 (randomized controlled trial)7 | ||||||||
| Comparison #1: Read, Write and Type!™ vs. Auditory Discrimination in Depth® group | ||||||||
| CTOPP: Phoneme Blending Subtest | Grade 1 | 104 | 18.90 (4.90) | 18.80 (5.30) | 0.10 | 0.02 | ns | +1 |
| CTOPP: Phoneme Elision Subtest | Grade 1 | 104 | 13.50 (4.50) | 14.30 (4.50) | –0.80 | –0.18 | ns | –7 |
| CTOPP: Phoneme Segmenting Subtest | Grade 1 | 104 | 15.30 (5.30) | 16.20 (6.60) | –0.90 | –0.15 | ns | –6 |
| Comparison #2: Read, Write and Type!™ vs. regular instruction/support group | ||||||||
| CTOPP: Phoneme Blending Subtest | Grade 1 | 77 | 20.10 (4.50) | 18.20 (5.40) | 1.90 | 0.38 | ns | +15 |
| CTOPP: Phoneme Elision Subtest | Grade 1 | 77 | 13.80 (4.20) | 12.50 (4.60) | 1.30 | 0.29 | ns | +11 |
| CTOPP: Phoneme Segmenting Subtest | Grade 1 | 77 | 15.40 (4.70) | 11.70 (4.50) | 3.70 | 0.80 | Statistically significant | +29 |
| Construct: Phonics | ||||||||
| Torgesen et al., 2003 (randomized controlled trial) | ||||||||
| Comparison #1: Read, Write and Type!™ vs. Auditory Discrimination in Depth® group | ||||||||
| Woodcock Reading Mastery Test: Word Attack Subtest | Grade 1 | 104 | 106.30 (13.60) | 109.70 (14.00) | –3.40 | –0.24 | ns | –10 |
| Woodcock Reading Mastery Test: Word Identification Subtest | Grade 1 | 104 | 105.10 (13.40) | 107.10 (14.30) | –2.00 | –0.14 | ns | –6 |
| Comparison #2: Read, Write and Type!™ vs. regular instruction/support group | ||||||||
| Woodcock Reading Mastery Test: Word Attack Subtest | Grade 1 | 77 | 108.30 (12.10) | 99.50 (14.50) | 8.80 | 0.65 | Statistically significant | +24 |
| Woodcock Reading Mastery Test: Word Identification Subtest | Grade 1 | 77 | 107.00 (12.40) | 100.10 (15.60) | 6.90 | 0.48 | ns | +18 |
| Averages8 | ||||||||
| Average for alphabetics, Comparison #1 (Torgesen et al., 2003) | -0.14 | ns | -6 | |||||
| Average for alphabetics, Comparison #2 (Torgesen et al., 2003) | 0.52 | Statistically significant | +20 | |||||
| Domain average for alphabetics domain across comparisons (Torgesen et al., 2003) | 0.19 | ns | +8 | |||||
| 1 This appendix reports findings considered for the effectiveness rating and the average improvement indices in the alphabetics domain. 2 The standard deviation across all students in each group shows how dispersed the participants' outcomes are: a smaller standard deviation on a given measure would indicate that participants had more similar outcomes. 3 Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the comparison group. 4 For an explanation of the effect size calculation, see Technical Details of WWC-Conducted Computations. 5 Statistical significance is the probability that the difference between groups is a result of chance rather than a real difference between the groups. 6 The improvement index represents the difference between the percentile rank of the average student in the intervention condition versus the percentile rank of the average student in the comparison condition. The improvement index can take on values between -50 and +50, with positive numbers denoting results favorable to the intervention group. 7 The level of statistical significance was reported by the study authors or, where necessary, calculated by the WWC to correct for clustering within classrooms or schools and for multiple comparisons. For an explanation about the clustering correction, see the WWC Tutorial on Mismatch. See Technical Details of WWC-Conducted Computations for the formulas the WWC used to calculate statistical significance. In the case of Torgesen et al. (2003), correction for multiple comparisons was needed, so the significance levels may differ from those reported in the original study. 8 The WWC-computed average effect sizes for each comparison and for the domain across comparisons are simple averages rounded to two decimal places. The average improvement indices are calculated from the average effect sizes. |
||||||||
Appendix A3.2 Summary of findings for the comprehension domain by construct1
| Author's findings from the study | ||||||||
|---|---|---|---|---|---|---|---|---|
| Mean outcome (standard deviation2) | WWC calculations | |||||||
| Outcome measure | Study sample | Sample size (students) | Read, Write & Type!™ group | Comparison group | Mean difference3 (RWT – comparison) | Effect size4 | Statistical significance5 (at α= 0.05) | Improvement index6 |
| Construct: Reading comprehension | ||||||||
| Torgesen et al., 2003 (randomized controlled trial)7 | ||||||||
| Comparison #1: Read, Write and Type!™ vs. Auditory Discrimination in Depth® group | ||||||||
| Woodcock Reading Mastery Test: Passage Comprehension Subtest | Grade 1 | 104 | 99.30 (10.50) | 99.30 (12.50) | -0.60 | -0.05 | ns | -2 |
| Comparison #2: Read, Write and Type!™ vs. regular instruction/support group | ||||||||
| Woodcock Reading Mastery Test: Passage Comprehension Subtest | Grade 1 | 77 | 100.20 (9.60) | 95.40 (14.40) | 4.80 | 0.38 | ns | +15 |
| Construct: Vocabulary | ||||||||
| Torgesen et al., 2003 (randomized controlled trial) | ||||||||
| Comparison #1: Read, Write and Type!™ vs. Auditory Discrimination in Depth® group | ||||||||
| Estimated Verbal IQ | Grade 1 | 104 | 95.508 | 95.508 | 0.00 | 0.00 | ns | 0 |
| Comparison #2: Read, Write and Type!™ vs. regular instruction/support group | ||||||||
| Estimated Verbal IQ | Grade 1 | 77 | 95.90 (11.20) | 95.90 (11.30) | 0.00 | 0.00 | ns | 0 |
| Averages9 | ||||||||
| Average for comprehension, Comparison #1 (Torgesen et al., 2003) | -0.03 | ns | -1 | |||||
| Average for comprehension, Comparison #2 (Torgesen et al., 2003) | 0.19 | ns | +8 | |||||
| Domain average for comprehension domain across comparisons (Torgesen et al., 2003) | 0.08 | ns | +3 | |||||
| 1 This appendix reports findings considered for the effectiveness rating and the average improvement indices in the comprehension domain. 2 The standard deviation across all students in each group shows how dispersed the participants' outcomes are: a smaller standard deviation on a given measure would indicate that participants had more similar outcomes. 3 Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the comparison group. 4 For an explanation of the effect size calculation, see Technical Details of WWC-Conducted Computations. 5 Statistical significance is the probability that the difference between groups is a result of chance rather than a real difference between the groups. 6 The improvement index represents the difference between the percentile rank of the average student in the intervention condition versus the percentile rank of the average student in the comparison condition. The improvement index can take on values between -50 and +50, with positive numbers denoting results favorable to the intervention group. 7 The level of statistical significance was reported by the study authors or, where necessary, calculated by the WWC to correct for clustering within classrooms or schools and for multiple comparisons. For an explanation about the clustering correction, see the WWC Tutorial on Mismatch. See Technical Details of WWC-Conducted Computations for the formulas the WWC used to calculate statistical significance. In the case of Torgesen et al. (2003), correction for multiple comparisons was needed, so the significance levels may differ from those reported in the original study. 8 The authors did not present standard deviations for this outcome, however, with the assumption that there were positive standard deviations, the zero mean difference between the RWT and the comparison groups generates a zero effect size. 9 The WWC-computed average effect sizes for each comparison and for the domain across comparisons are simple averages rounded to two decimal places. The average improvement indices are calculated from the average effect sizes. |
||||||||
Appendix A4.1 Read, Write & Type!™ rating for the alphabetics domain
The WWC rates anintervention's effects in a given outcome domain as positive, potentially positive, mixed, no discernible effects, potentially negative, or negative.1
For the outcome domain of alphabetics, the WWC rated Read, Write & Type!™ as having potentially positive effects. It did not meet the criteria for positive effects because only one study met WWC evidence standards. The remaining ratings (mixed effects, no discernible effects, potentially negative effects, negative effects) were not considered because the intervention was assigned the highest applicable rating.
| Rating received |
|---|
|
Potentially positive effects: Evidence of a positive effect with no overriding contrary evidence.
|
| Other ratings considered |
|
Positive effects: Strong evidence of a positive effect with no overriding contrary evidence.
|
| 1 For rating purposes, the WWC considers the statistical significance of individual outcomes and the domain-level effect. The WWC also considers the size of the domain-level effect for ratings of potentially positive or potentially negative effects. See the WWC Intervention Rating Scheme for a complete description. |
Appendix A4.2 Read, Write & Type!™ rating for the comprehension domain
The WWC rates anintervention's effects in a given outcome domain as positive, potentially positive, mixed, no discernible effects, potentially negative, or negative.1
For the outcome domain of comprehension, the WWC rated Read, Write & Type!™ as having no discernible effects. It did not meet the criteria for other ratings (positive effects, potentially positive effects, mixed effects, potentially negative effects, and negative effects) because the one study that met WWC standards did not show statistically significant or substantively important effects.
| Rating received |
|---|
|
No discernible effects: No affirmative evidence of effects.
|
| Other ratings considered |
|
Positive effects: Strong evidence of a positive effect with no overriding contrary evidence.
|
|
Potentially positive effects: Evidence of a positive effect with no overriding contrary evidence.
|
|
Mixed effects: Evidence of inconsistent effects as demonstrated through either of the following criteria.
|
|
Potentially negative effects: Evidence of a negative effect with no overriding contrary evidence.
|
|
Negative effects: Strong evidence of a negative effect with no overriding contrary evidence.
|
| 1 For rating purposes, the WWC considers the statistical significance of individual outcomes and the domain-level effect. The WWC also considers the size of the domain-level effect for ratings of potentially positive or potentially negative effects. See the WWC Intervention Rating Scheme for a complete description. |
Appendix A5 Extent of evidence by domain
| Sample size | ||||
|---|---|---|---|---|
| Outcome domain | Number of studies | Schools | Studies | Extent of evidence1 |
| Alphabetics | 1 | 5 | 146 | small |
| Fluency | 0 | 0 | 0 | n/a |
| Comprehension | 1 | 5 | 146 | small |
| General reading achievement | 0 | 0 | 0 | n/a |
|
na = not applicable/not studied 1 A rating of "medium to large" requires at least two studies and two schools across studies in one domain, and a total sample size across studies of at least 350 students or 14 classrooms. Otherwise, the rating is "small." |
||||
