WWC review of this study

Alignment of game design features and state mathematics standards: Do results reflect intentions?

Schenke, K., Rutherford, T., & Farkas, G. (2014). Computers & Education, 76, 215–224.

  • Randomized Controlled Trial
     examining 
    10,860
     Students
    , grades
    2-5

Reviewed: July 2020

At least one finding shows moderate evidence of effectiveness
At least one statistically significant positive finding
Meets WWC standards with reservations
Algebra outcomes—Indeterminate effects found
Outcome
measure
Comparison Period Sample Intervention
mean
Comparison
mean
Significant? Improvement
    index
Evidence
tier

California Standards Test: Algebra and Functions (A&F) strand

ST Math vs. Business as usual

0 Days

Full sample;
10,860 students

11.31

11.30

No

--
Data analysis, statistics, and probability outcomes—Indeterminate effects found
Outcome
measure
Comparison Period Sample Intervention
mean
Comparison
mean
Significant? Improvement
    index
Evidence
tier

California Standards Test: Statistics, Data Analysis, and Probability (SDAP)

ST Math vs. Business as usual

0 Days

Full sample;
10,860 students

3.32

3.25

No

--
Geometry and Measurement outcomes—Indeterminate effects found
Outcome
measure
Comparison Period Sample Intervention
mean
Comparison
mean
Significant? Improvement
    index
Evidence
tier

California Standards Test: Measurement and Geometry (M&G) strand

ST Math vs. Business as usual

0 Days

Full sample;
10,860 students

9.64

9.22

No

--
Number and Operations outcomes—Statistically significant positive effects found
Outcome
measure
Comparison Period Sample Intervention
mean
Comparison
mean
Significant? Improvement
    index
Evidence
tier

California Standards Test: Number Sense I

ST Math vs. Business as usual

0 Days

Full sample;
10,860 students

10.74

10.12

Yes

 
 
6
 
More Outcomes

California Standards Test: Number Sense II

ST Math vs. Business as usual

0 Days

Full sample;
10,860 students

10.66

10.30

No

--


Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.

Characteristics of study sample as reported by study author.


  • 70% English language learners

  • Female: 49%
    Male: 51%
    • B
    • A
    • C
    • D
    • E
    • F
    • G
    • I
    • H
    • J
    • K
    • L
    • P
    • M
    • N
    • O
    • Q
    • R
    • S
    • V
    • U
    • T
    • W
    • X
    • Z
    • Y
    • a
    • h
    • i
    • b
    • d
    • e
    • f
    • c
    • g
    • j
    • k
    • l
    • m
    • n
    • o
    • p
    • q
    • r
    • s
    • t
    • u
    • x
    • w
    • y

    California
  • Race
    Asian
    4%
    Black
    2%
    Other or unknown
    89%
    White
    5%
  • Ethnicity
    Hispanic    
    85%
    Not Hispanic or Latino    
    15%

Setting

The study took place in 52 elementary schools in southern California during the 2008-2009 and 2009-2010 school years. Schools were eligible to apply for the study if they were in the bottom third of the achievement distribution for the California state standardized test (p. 218).

Study sample

The analytic sample includes students in grades 3-5 with valid pretest and posttest mathematics assessment data. The study reports the following characteristics for the analytic sample, which includes students in 50 schools: Black: 1.61%; Hispanic: 85.06%; White: 5.13%; Vietnamese: 3.79%; other race/ethnicity: 4.42%; male: 50.75%; free/reduced price lunch: 90.61%; and English learner: 70.17% (Table 1, p. 219).

Intervention Group

Spatial Temporal Mathematics (ST Math) is an instructional program developed by the MIND Research Institute for K-5 mathematics instruction; the authors referred readers to Shaw (1999) for additional details. The program is designed to minimize the need to use language as part of the mathematics instruction and presents students with game-like activities that provide mathematics content relevant to state standards. The games each include multiple mathematics puzzles, and students need to complete at least 80% of the puzzles successfully at one level before moving to the next level. The program is self-paced, though teachers can speed up progress by allowing students to skip levels if they get stuck. The authors report that the practice of skipping levels is not common. Some students did not receive the full ST Math curriculum because they did not progress to the later objectives, while other students received a different ordering of the curriculum (pp. 217, 223).

Comparison Group

Students in the comparison group did not use ST Math. The study did not provide any additional information on the type of instruction received by students in the comparison group.

Support for implementation

No support for implementation is described in the study.

Reviewed: October 2015

At least one finding shows moderate evidence of effectiveness
At least one statistically significant positive finding
Meets WWC standards with reservations
General Mathematics Achievement outcomes—Statistically significant positive effects found
Outcome
measure
Comparison Period Sample Intervention
mean
Comparison
mean
Significant? Improvement
    index
Evidence
tier

California Standards Test (CST): Number Sense I (number recognition)

Spatial Temporal Mathematics (ST Math) vs. Business as usual

Posttest

3-5 grade students;
10,860 students

N/A

N/A

Yes

 
 
6
 
More Outcomes

California Standards Test (CST): Number Sense II (number operations)

Spatial Temporal Mathematics (ST Math) vs. Business as usual

Posttest

3-5 grade students;
10,860 students

N/A

N/A

No

--

Algebra and Functions (A&F) strand of the California Standards Test

Spatial Temporal Mathematics (ST Math) vs. Business as usual

Posttest

3-5 grade students;
10,860 students

N/A

N/A

No

--

Measurement and Geometry (M&G) strand of the California Standards Test

Spatial Temporal Mathematics (ST Math) vs. Business as usual

Posttest

3-5 grade students;
10,860 students

N/A

N/A

No

--

California Standards Test (CST): Statistics, Data Analysis and Probability (SDAP)

Spatial Temporal Mathematics (ST Math) vs. Business as usual

Posttest

3-5 grade students;
10,860 students

N/A

N/A

No

--


Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.

Characteristics of study sample as reported by study author.


  • 70% English language learners

  • Female: 49%
    Male: 51%
    • B
    • A
    • C
    • D
    • E
    • F
    • G
    • I
    • H
    • J
    • K
    • L
    • P
    • M
    • N
    • O
    • Q
    • R
    • S
    • V
    • U
    • T
    • W
    • X
    • Z
    • Y
    • a
    • h
    • i
    • b
    • d
    • e
    • f
    • c
    • g
    • j
    • k
    • l
    • m
    • n
    • o
    • p
    • q
    • r
    • s
    • t
    • u
    • x
    • w
    • y

    California
  • Race
    Asian
    4%
    Black
    2%
    Other or unknown
    4%
    White
    5%
  • Ethnicity
    Hispanic    
    85%
    Not Hispanic or Latino    
    15%
 

Your export should download shortly as a zip archive.

This download will include data files for study and findings review data and a data dictionary.

Connect With the WWC

loading
back to top