The Effects of Odyssey® Math on Grade 4 Student Math Achievement in the Mid-Atlantic Region

Regional need and study purpose. In response to an assessment of regional needs that revealed student math achievement as a high priority, the Regional Educational Laboratory (REL) Mid-Atlantic conducted a randomized field trial to evaluate the impact of a math curriculum software package, Odyssey® Math, on math achievement in grade 4 in the 2007/08 school year. Using a more rigorous research design than previous studies of Odyssey® Math, this study evaluates whether Odyssey® Math causes higher math achievement.

Intervention description. According to the developer, Odyssey® Math is a computer-based math curriculum that includes professional development and online software. Designed to improve math achievement, the program is used in 5,000 schools, more than 700 in the Mid-Atlantic Region alone, and with 3 million students throughout the United States. The software can be used as the main curriculum, a partial substitute for the main curriculum, or a supplement to the main curriculum.

Study design and period. Within each of the 32 schools grade 4 teachers who volunteered to participate in this study and their classrooms were randomly assigned to intervention (Odyssey® Math access) or control (curriculum delivered as usual) groups, resulting in 61 classrooms and 1,399 students in the intervention group and 63 classrooms and 1,477 students in the control group. So that intervention and control groups receive the same amount of instruction, math teachers in the intervention group used Odyssey® Math as a substitute for part of their usual math curriculum for up to 60 minutes a week during 2007/08, while teachers in the control group used only the regular curriculum.

Key outcomes and measures. This study's key outcome, grade 4 student math achievement, was measured by the TerraNova CTBS Basic Battery math subtest. The test consists of 57 selected response items and takes 1 hour and 10 minutes to administer. Its content objectives reflect the National Council of Teachers of Mathematics Standards, state and local curriculum documents, and the conceptual framework of the National Assessment of Educational Progress.

Data collection approach. Trained field research coordinators administered a teacher demographic survey, conducted pretests and posttests, and observed intervention and control classrooms. REL Mid-Atlantic staff administered pretests in week 3 of the 2007/08 school year, and posttests five weeks before the end of the same school year. Data were also collected on the time students spend using the software.

Analysis plan. Because students were nested in classrooms, and classrooms, in schools, multilevel modeling was used to analyze the data. Three models were specified, one for each of the three research questions. Descriptive analyses were conducted to describe teacher and student characteristics, assess baseline equivalence between the intervention and control groups, and evaluate teacher and student withdrawals from the study.

Principal investigators. Kay Wijekumar, PhD, The Pennsylvania State University and John Hitchcock, PhD, ICF International.

Additional Information Region, contact information, and references.

Click here for full printable version