WWC review of this study

Evaluation of Rocketship Education’s use of DreamBox Learning’s online mathematics program.

Wang, H., & Woodworth, K. (2011). Menlo Park, CA: SRI International. Retrieved from http://www.dreambox.com/

  • Randomized Controlled Trial
     examining 
    557
     Students
    , grades
    K-1
At least one statistically significant positive finding
Meets WWC standards without reservations

Reviewed: December 2013

Mathematics achievement outcomes—Statistically significant positive effects found
Outcome
measure
Comparison Period Sample Intervention
mean
Comparison
mean
Significant? Improvement
index

Northwest Evaluation Association (NWEA) Measures of Academic Progress (MAP): Mathematics

DreamBox Learning vs. None

Posttest

Grades K and 1;
557 students

159

156.2

Yes

 
 
4

Characteristics of study sample as reported by study author.


  • 81% English language learners

  • 88% Free or reduced price lunch

  • Female: 53%
    Male: 47%
  • Ethnicity
    Hispanic
    87%
    Not Hispanic
    13%

  • Urban
    • B
    • A
    • C
    • D
    • E
    • F
    • G
    • I
    • H
    • J
    • K
    • L
    • P
    • M
    • N
    • O
    • Q
    • R
    • S
    • V
    • U
    • T
    • W
    • X
    • Z
    • Y
    • a
    • h
    • i
    • b
    • d
    • e
    • f
    • c
    • g
    • j
    • k
    • l
    • m
    • n
    • o
    • p
    • q
    • r
    • s
    • t
    • u
    • x
    • w
    • y

    California

Setting

The study was conducted in three Rocketship Education charter schools located in San Jose, California.

Study sample

The study sample included all kindergarten and first-grade students at the three schools that participated in the study, a total of 557 students after attrition from a sample of 583 who were randomly assigned. The number of classrooms included in the study is not specified. Within grade levels, students were randomly assigned to either the intervention or comparison groups at a 4 to 1 ratio. In the baseline sample, 53% of students were female, 87% were Hispanic, 81% were English language learners, 88% were eligible for free or reduced-price meals, 4% were classified as special education, and 10% participated in Response to Intervention (RtI) services.

Intervention Group

The experiment was conducted from mid-October through mid-February during the 2010–11 school year. Intervention students were scheduled to receive 20 to 40 minutes of DreamBox Learning mathematics instruction per day; usage statistics show that students averaged 21.8 hours of usage over the course of the study, or approximately 16 minutes per day. Instructional sessions were conducted in a computer lab. The authors noted that the low-achieving students who were assigned to receive RtI services were scheduled to receive 45 minutes of DreamBox Learning instruction in their after-school RtI programming, regardless of intervention status. For the 42 intervention group students who were assigned to RtI services, this 45 minutes was in addition to the DreamBox Learning instruction provided during the school day, for a total of 26.5 hours of usage over the course of the study on average. Progress and use information provided by the DreamBox Learning software was not used to modify face-to-face mathematics instruction for either the intervention or comparison group.

Comparison Group

Students in the comparison condition received no additional mathematics instruction. However, they received additional literacy instruction via an online program during the time and in the same location as intervention group students using the DreamBox Learning software. The 11 students in the comparison condition who were assigned to RtI services were scheduled to receive 45 minutes of DreamBox Learning instruction in their after-school RtI programming; the authors found that these comparison condition students averaged 5.1 hours of program usage over the course of the study.

Outcome descriptions

The study used math test scores from the MAP assessment developed by the Northwest Evaluation Association (NWEA). The study reports the overall math score, as well as five subtest scores, for problem solving, number sense, computation, measurement and geometry, and statistics and probability. Scores were scaled using the RIT scale, “which is scaled using the Item Response Theory (IRT) and has the same meaning regardless of the grade of the student” (as cited in Wang & Woodworth, 2011, p. 3). The schools administered the assessment in September 2010 (pretest) and January/February 2011 (posttest). For a more detailed description of this outcome measure, see Appendix B.

Support for implementation

DreamBox Learning “does not prescribe a specific role for teachers” (Wang & Woodworth, 2011, p. 3). The computer labs in which students received DreamBox Learning instruction were run by lab coordinators, noncredentialed hourly staff who played a minimal role in instruction. The authors noted that lab coordinators sometimes may have been out of the computer lab, at which times the students would be supervised by support staff.

In the case of multiple manuscripts that report on one study, the WWC selects one manuscript as the primary citation and lists other manuscripts that describe the study as additional sources.

  • Wang, H., & Woodworth, K. (2011). A randomized controlled trial of two online mathematics curricula. Evanston, IL: Society for Research on Educational Effectiveness.

 

Your export should download shortly as a zip archive.

This download will include data files for study and findings review data and a data dictionary.

Connect With the WWC

newsflash icon contact icon facebook icon twitter icon
loading
back to top