WWC review of this study

Alignment of game design features and state mathematics standards: Do results reflect intentions?

Schenke, K., Rutherford, T., & Farkas, G. (2014). Computers & Education, 76, 215–224.

  • Randomized Controlled Trial
     examining 
    10,860
     Students
    , grades
    2-5

Reviewed: July 2020

At least one finding shows moderate evidence of effectiveness
At least one statistically significant positive finding
Meets WWC standards with reservations
Algebra outcomes—Indeterminate effects found
Outcome
measure
Comparison Period Sample Intervention
mean
Comparison
mean
Significant? Improvement
    index
ESSA
rating

California Standards Test: Algebra and Functions (A&F) strand

ST Math vs. Business as usual

0 Days

Full sample;
10,860 students

11.31

11.30

No

--
Data analysis, statistics, and probability outcomes—Indeterminate effects found
Outcome
measure
Comparison Period Sample Intervention
mean
Comparison
mean
Significant? Improvement
    index
ESSA
rating

California Standards Test: Statistics, Data Analysis, and Probability (SDAP)

ST Math vs. Business as usual

0 Days

Full sample;
10,860 students

3.32

3.25

No

--
Geometry and Measurement outcomes—Indeterminate effects found
Outcome
measure
Comparison Period Sample Intervention
mean
Comparison
mean
Significant? Improvement
    index
ESSA
rating

California Standards Test: Measurement and Geometry (M&G) strand

ST Math vs. Business as usual

0 Days

Full sample;
10,860 students

9.64

9.22

No

--
Number and Operations outcomes—Statistically significant positive effects found
Outcome
measure
Comparison Period Sample Intervention
mean
Comparison
mean
Significant? Improvement
    index
ESSA
rating

California Standards Test: Number Sense I

ST Math vs. Business as usual

0 Days

Full sample;
10,860 students

10.74

10.12

Yes

 
 
6
 
More Outcomes

California Standards Test: Number Sense II

ST Math vs. Business as usual

0 Days

Full sample;
10,860 students

10.66

10.30

No

--

Characteristics of study sample as reported by study author.


  • 70% English language learners

  • 91% Free or reduced price lunch

  • Female: 49%
    Male: 51%
  • Race
    Asian
    4%
    Black
    2%
    Not specified
    89%
    White
    5%
  • Ethnicity
    Hispanic
    85%
    Not Hispanic
    15%
    • B
    • A
    • C
    • D
    • E
    • F
    • G
    • I
    • H
    • J
    • K
    • L
    • P
    • M
    • N
    • O
    • Q
    • R
    • S
    • V
    • U
    • T
    • W
    • X
    • Z
    • Y
    • a
    • h
    • i
    • b
    • d
    • e
    • f
    • c
    • g
    • j
    • k
    • l
    • m
    • n
    • o
    • p
    • q
    • r
    • s
    • t
    • u
    • x
    • w
    • y

    California

Setting

The study took place in 52 elementary schools in southern California during the 2008-2009 and 2009-2010 school years. Schools were eligible to apply for the study if they were in the bottom third of the achievement distribution for the California state standardized test (p. 218).

Study sample

The analytic sample includes students in grades 3-5 with valid pretest and posttest mathematics assessment data. The study reports the following characteristics for the analytic sample, which includes students in 50 schools: Black: 1.61%; Hispanic: 85.06%; White: 5.13%; Vietnamese: 3.79%; other race/ethnicity: 4.42%; male: 50.75%; free/reduced price lunch: 90.61%; and English learner: 70.17% (Table 1, p. 219).

Intervention Group

Spatial Temporal Mathematics (ST Math) is an instructional program developed by the MIND Research Institute for K-5 mathematics instruction; the authors referred readers to Shaw (1999) for additional details. The program is designed to minimize the need to use language as part of the mathematics instruction and presents students with game-like activities that provide mathematics content relevant to state standards. The games each include multiple mathematics puzzles, and students need to complete at least 80% of the puzzles successfully at one level before moving to the next level. The program is self-paced, though teachers can speed up progress by allowing students to skip levels if they get stuck. The authors report that the practice of skipping levels is not common. Some students did not receive the full ST Math curriculum because they did not progress to the later objectives, while other students received a different ordering of the curriculum (pp. 217, 223).

Comparison Group

Students in the comparison group did not use ST Math. The study did not provide any additional information on the type of instruction received by students in the comparison group.

Support for implementation

No support for implementation is described in the study.

Reviewed: October 2015

At least one finding shows moderate evidence of effectiveness
At least one statistically significant positive finding
Meets WWC standards with reservations
General Mathematics Achievement outcomes—Statistically significant positive effects found
Outcome
measure
Comparison Period Sample Intervention
mean
Comparison
mean
Significant? Improvement
    index
ESSA
rating

California Standards Test (CST): Number Sense I (number recognition)

Spatial Temporal Mathematics (ST Math) vs. Business as usual

Posttest

3-5 grade students;
10,860 students

N/A

N/A

Yes

 
 
6
 
More Outcomes

California Standards Test (CST): Number Sense II (number operations)

Spatial Temporal Mathematics (ST Math) vs. Business as usual

Posttest

3-5 grade students;
10,860 students

N/A

N/A

No

--

Algebra and Functions (A&F) strand of the California Standards Test

Spatial Temporal Mathematics (ST Math) vs. Business as usual

Posttest

3-5 grade students;
10,860 students

N/A

N/A

No

--

Measurement and Geometry (M&G) strand of the California Standards Test

Spatial Temporal Mathematics (ST Math) vs. Business as usual

Posttest

3-5 grade students;
10,860 students

N/A

N/A

No

--

California Standards Test (CST): Statistics, Data Analysis and Probability (SDAP)

Spatial Temporal Mathematics (ST Math) vs. Business as usual

Posttest

3-5 grade students;
10,860 students

N/A

N/A

No

--

Characteristics of study sample as reported by study author.


  • 70% English language learners

  • 91% Free or reduced price lunch

  • Female: 49%
    Male: 51%
  • Race
    Asian
    4%
    Black
    2%
    Not specified
    4%
    White
    5%
  • Ethnicity
    Hispanic
    85%
    Not Hispanic
    15%
    • B
    • A
    • C
    • D
    • E
    • F
    • G
    • I
    • H
    • J
    • K
    • L
    • P
    • M
    • N
    • O
    • Q
    • R
    • S
    • V
    • U
    • T
    • W
    • X
    • Z
    • Y
    • a
    • h
    • i
    • b
    • d
    • e
    • f
    • c
    • g
    • j
    • k
    • l
    • m
    • n
    • o
    • p
    • q
    • r
    • s
    • t
    • u
    • x
    • w
    • y

    California
 

Your export should download shortly as a zip archive.

This download will include data files for study and findings review data and a data dictionary.

Connect With the WWC

loading
back to top