WWC review of this study

Putting computerized instruction to the test: A randomized evaluation of a “scientifically based” reading program.

Rouse, C. E., & Krueger, A. B. (2004). Economics of Education Review, 23(4), 323-338. Retrieved from: https://eric.ed.gov/?id=EJ730375

  • Randomized Controlled Trial
     examining 
    454
     Students
    , grades
    3-6
No statistically significant positive
findings
Meets WWC standards without reservations

Reviewed: September 2016

Literacy achievement outcomes—Indeterminate effects found
Outcome
measure
Comparison Period Sample Intervention
mean
Comparison
mean
Significant? Improvement
index

State Standardized Reading Test

Fast ForWord® vs. Business as usual

8 Weeks

Full sample;
454 students

44.57

43.03

No

 
 
2
Reading comprehension outcomes—Indeterminate effects found
Outcome
measure
Comparison Period Sample Intervention
mean
Comparison
mean
Significant? Improvement
index

CELF-3

Fast ForWord® vs. Business as usual

8 Weeks

Grade: 4;
86 students

32.09

31.01

No

 
 
2
Reading fluency outcomes—Indeterminate effects found
Outcome
measure
Comparison Period Sample Intervention
mean
Comparison
mean
Significant? Improvement
index

Success for All assessment

Fast ForWord® vs. Business as usual

8 Weeks

Full sample;
373 students

4.11

4.03

No

 
 
1

Characteristics of study sample as reported by study author.


  • Female: 52%
    Male: 48%
  • Race
    Black
    27%
  • Ethnicity
    Hispanic
    65%
    Not Hispanic
    35%

  • Urban
    • B
    • A
    • C
    • D
    • E
    • F
    • G
    • I
    • H
    • J
    • K
    • L
    • P
    • M
    • N
    • O
    • Q
    • R
    • S
    • V
    • U
    • T
    • W
    • X
    • Z
    • Y
    • a
    • h
    • i
    • b
    • d
    • e
    • f
    • c
    • g
    • j
    • k
    • l
    • m
    • n
    • o
    • p
    • q
    • r
    • s
    • t
    • u
    • x
    • w
    • y

    Northeast

Setting

The study took place in four elementary schools in one urban school district in the northeast. The district has roughly 20,000 students with the majority of students being ethnic minorities: 40% African American and 50% Hispanic. Nearly 70% of the students qualify for free/reduced lunch program and 56% speak a language other than English at home. Since Fast ForWord is a type of pull-out instructional program, participants in intervention and comparison groups were not from intact classrooms.

Study sample

There were 49% female participants in the intervention group, and 55% of the comparison group were female. 27% of the intervention and comparison groups were African-American. 64% of the intervention group was Latino, and 67% of the comparison group was Latino. In the district, almost 70% of the students qualify for free or reduced price lunch, but this information was not provided for the intervention and comparison groups. In the intervention group, 15% of students were identified as being in special education, and 16% of the students in the comparison group were identified as special education.

Intervention Group

The FFW is a computer software program distributed by the Scientific Learning Corporation (SLC). The program uses computer games to try and retrain the brain. The games work by slowing and magnifying the acoustic changes within normal speech. The program is comprised of three programs, including FFW language, FFW language-to-reading, and FFW reading. FFW language helps to develop oral language skills that are foundational skills for reading. The FFW language-to-reading helps students make the connection between spoken and written language. FFW Reading helps build readings skills, such as word recognition and fluency. The program goes 6 - 8 weeks and students work between 90-100 minutes per day, five days a week. Students start at a basic level and progress through proficiency as they acquire skills. A student is said to have completed the program after at least 20 days and has completed at least 80% of the 5 - 7 program games. Intervention group participants had regular reading instruction offered by the school, and received an additional 90-100 minutes per day with the Fast ForWord reading computer games (p. 324). Students in the intervention group missed a variety of activities during the Fast ForWord intervention time, specified in Table 1 (p. 327). These missed activities included homeroom, math, science, language arts, specials (art, music, gym). All students were in Success for All, a whole school reform adopted by most schools in the district at the time of the study (p. 327). Study participation was broken into two ""flights"" as illustrated in Table 3 (p. 332). Students had a potential for 37 training days in Flight 1, and about 30 training days in Flight 2. In Table A2, the authors reported that 268 students were in Flight 1, and 244 students were in Flight 2 among the randomized sample size.

Comparison Group

Comparison group participants had regular reading instruction offered by the school, and participated in a variety of activities that the intervention group students missed while doing Fast ForWord. Table 1 (p. 327) detailed the types of activities in which the comparison group participated, and the proportion of the students in the comparison group who were affected. These missed activities included homeroom, math, science, language arts, specials (art, music, gym).

Support for implementation

The school district supported implementation of Fast ForWard in the four schools by licensing 30 computers per school for 1 year at a cost of $30,000. In two schools, the intervention took place in computers in the library, and in two other schools, the intervention took place in a computer lab (p. 326, note 3). The training package cost $100 per site, and any adult who completed the training could supervise the students (did not have to be a certified teacher). The schools were equipped with sufficient power to run the software, color printers, head phones, Y-connectors, a quiet space for students to work, and an adult supervisor trained in the program (p. 326). Study researchers conducted site visits to ensure that computer labs were set up properly and the instructors were adequately trained (p. 327). The Scientific Learning Corporation provided support at the beginning of the evaluation, conducted site visits, and provided telephone support (p. 327).

Reviewed: March 2013

Study sample characteristics were not reported.
No statistically significant positive
findings
Meets WWC standards without reservations

Reviewed: August 2010

Literacy achievement outcomes—Indeterminate effects found
Outcome
measure
Comparison Period Sample Intervention
mean
Comparison
mean
Significant? Improvement
index

Clinical Evaluation of Language Fundamentals- Third Edition (CELF-3): Receptive Language

Fast ForWord® vs. Business as usual

Posttest

grades 3-6;
86 students

31.7

31.01

No

--
More Outcomes

State Standardized Reading Test

Fast ForWord® vs. Business as usual

Posttest

grades 3-6;
454 students

44.18

43.03

No

--

Characteristics of study sample as reported by study author.


  • 73% Free or reduced price lunch
  • Race
    Black
    42%
  • Ethnicity
    Hispanic
    51%
    Not Hispanic
    49%

  • Urban
    • B
    • A
    • C
    • D
    • E
    • F
    • G
    • I
    • H
    • J
    • K
    • L
    • P
    • M
    • N
    • O
    • Q
    • R
    • S
    • V
    • U
    • T
    • W
    • X
    • Z
    • Y
    • a
    • h
    • i
    • b
    • d
    • e
    • f
    • c
    • g
    • j
    • k
    • l
    • m
    • n
    • o
    • p
    • q
    • r
    • s
    • t
    • u
    • x
    • w
    • y

    Northeast

Setting

The study took place in four schools in an urban district in the northeastern United States. Forty percent of the district’s students were African-American and more than 50% were Hispanic. Almost 70% of students in the district qualified for the free or reduced-price lunch program, and 56% of the district’s students spoke a language other than English at home. The authors describe test scores in these schools as well below average and note that schools in the district adopted a whole-school reform, Success for All.

Study sample

Groups were formed through a multistep process. Authors first identified an eligible population of students from four schools within one urban school district, focusing on third- to sixth-grade students who scored in the bottom 20% on the state’s standardized reading test administered in the 2001–02 school year. Consent letters were sent to these students’ parents. Principals in the schools were asked to identify students who could not sit through the daily 90- to 100-minute use of Fast ForWord®, those who had transferred to another school, and those students who might otherwise be unavailable (family away on long trip, for example). The remaining students were randomly assigned to either the treatment or control group, within each grade and school. In all, 237 students in the Fast ForWord® group and 217 students in the comparison group were included in the analysis sample.

Intervention Group

Fast ForWord® was primarily an add-on to regular reading instruction. In three schools, students in the treatment condition were pulled out of their regular classroom instruction for 90–100 minutes of computerized Fast ForWord® instruction per day and, in one school, they used Fast ForWord® for that same amount of time before or after school. Each school had to find a way to fit the use of Fast ForWord® into its unique schedule. In no case were students taken out of Success for All. The study reported students’ outcomes after six to eight weeks of program implementation

Comparison Group

The control group continued to receive the standard curriculum being used in district schools. Because the Fast ForWord® students used Fast ForWord® either during subjects such as math, science, language arts, special subjects (such as art, music, or gym), or homeroom, or—in the case of one school—before or after school, the counterfactual condition for the control group students was mixed.

Outcome descriptions

For both the pretest and posttest, the authors administered the Success for All assessment, the Clinical Evaluation of Language Fundamentals–Third Edition (the receptive portion and the Listening to Paragraph supplemental test), and a state standardized reading test (the authors did not indicate which state). For a more detailed description of test outcome measures, see Appendix A2.4.

Support for implementation

Fast ForWord® staff provided training for Fast ForWord® instructors (those interacting with students) at the beginning of the study. Phone support was also provided for the duration of the study. Detailed information on the training of instructors was not provided.

In the case of multiple manuscripts that report on one study, the WWC selects one manuscript as the primary citation and lists other manuscripts that describe the study as additional sources.

  • Rouse, C. E., Krueger, A. B., & Markman, L. (2004). Putting computerized instruction to the test: A randomized evaluation of a “scientifically based” reading program. Cambridge, MA: National Bureau of Economic Research.

 

Your export should download shortly as a zip archive.

This download will include data files for study and findings review data and a data dictionary.

Connect With the WWC

newsflash icon contact icon facebook icon twitter icon
loading
back to top