An Examination of the Promise of the NumberShire Level 1 Gaming Intervention for Improving Student Mathematics Outcomes [Whole-number concepts intervention vs. control]
Fien, Hank; Doabler, Christian T.; Nelson, Nancy J.; Kosty, Derek B.; Clarke, Ben; Baker, Scott K. (2016). Journal of Research on Educational Effectiveness v9 n4 p635-661. Retrieved from: https://eric.ed.gov/?id=ED576652
-
examining238Students, grade1
Practice Guide
Review Details
Reviewed: February 2020
- Practice Guide (findings for Targeted Math Intervention)
- Randomized Controlled Trial
- Meets WWC standards without reservations because it is a randomized controlled trial with low attrition.
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Findings
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
GA-MN Group Administered Missing Number |
Targeted Math Intervention vs. Business as usual |
0 Days |
Full sample;
|
11.40 |
11.00 |
No |
-- |
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Profusion - R (Revised) |
Targeted Math Intervention vs. Business as usual |
0 Days |
Full sample;
|
58.20 |
52.30 |
Yes |
|
|
|
Easy CBM - NCTM (grade 1) |
Targeted Math Intervention vs. Business as usual |
0 Days |
Full sample;
|
23.20 |
24.10 |
No |
-- |
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
GA-QD Group Administered Quantity Discrimination |
Targeted Math Intervention vs. Business as usual |
0 Days |
Full sample;
|
25.50 |
25.00 |
No |
-- |
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Sample Characteristics
Characteristics of study sample as reported by study author.
-
24% English language learners -
Female: 50%
Male: 50% -
Suburban, Urban
-
- B
- A
- C
- D
- E
- F
- G
- I
- H
- J
- K
- L
- P
- M
- N
- O
- Q
- R
- S
- V
- U
- T
- W
- X
- Z
- Y
- a
- h
- i
- b
- d
- e
- f
- c
- g
- j
- k
- l
- m
- n
- o
- p
- q
- r
- s
- t
- u
- v
- x
- w
- y
Oregon
-
Race Asian 4% Black 7% Other or unknown 29% White 61% -
Ethnicity Hispanic 22% Not Hispanic or Latino 78%
Study Details
Setting
Twenty-six first-grade classrooms from two districts (11 classes in one district and 15 classes in the other) in two cities in Oregon (Eugene and Portland) participated. Students who met inclusion criteria were randomly assigned to treatment or comparison conditions. If assigned to treatment, they received 1:1 tutoring on a computer as a supplement to their core curriculum.
Study sample
The average age of the students was 6.5 years. For treatment students, 4% were Asian, 5% were black, 21% were Latino, 8% were multiracial, and 62% were white. 51% were female, 9% were in SPED, and 25% were ELL. For the comparison students, 4% were Asian, 8% were Black, 23% were Latina, 4% were multiracial, and 61% were White. 49% were female, 10% were SPED, and 22% were ELL. Free and reduced lunch was reported by district, rather than for the sample. In District A 57% of the students received free/reduced lunch and in District B 35.5% of students received free/reduced lunch.
Intervention Group
The intervention period was 8 weeks. Each student receiving the Tier 2 supplemental intervention participated 4 times a week. Authors report the program included 48 sessions over 12 weeks; however, intervention in this study only ran for 8 weeks. Each intervention session was 15 minutes in duration. The intervention was delivered on the computer and included an explicit instructional framework. Four activities were warm-up, teaching event, assessment event, and wrap up. The warm-up reviews previously taught material, the teaching event includes characters in the game provide support as students learn new material and students then practice with supported help from these characters. The last two activities provide extended practice. Virtual representations are included such as number lines and base-10 blocks.
Comparison Group
The intervention was eight weeks in duration. This is a business-as-usual condition and therefore included a wide variety of intervention materials across schools and districts. Some were developed by the district, Touch Math and SRA Explorations was reported in two classrooms. Everyday Mathematics was reported in 6 classrooms. Basic facts worksheets and calendar concepts were also reported by teachers as intervention resource materials.
Support for implementation
All interventionists who facilitated Number Shires 1 received 4 hour of professional development. This included a 2-hour training presentation and two 1-hour site-based visits with project staff.
An indicator of the effect of the intervention, the improvement index can be interpreted as the expected change in percentile rank for an average comparison group student if that student had received the intervention.
For more, please see the WWC Glossary entry for improvement index.
An outcome is the knowledge, skills, and attitudes that are attained as a result of an activity. An outcome measures is an instrument, device, or method that provides data on the outcome.
A finding that is included in the effectiveness rating. Excluded findings may include subgroups and subscales.
The sample on which the analysis was conducted.
The group to which the intervention group is compared, which may include a different intervention, business as usual, or no services.
The timing of the post-intervention outcome measure.
The number of students included in the analysis.
The mean score of students in the intervention group.
The mean score of students in the comparison group.
The WWC considers a finding to be statistically significant if the likelihood that the finding is due to chance alone, rather than a real difference, is less than five percent.
The WWC reviews studies for WWC products, Department of Education grant competitions, and IES performance measures.
The name and version of the document used to guide the review of the study.
The version of the WWC design standards used to guide the review of the study.
The result of the WWC assessment of the study. The rating is based on the strength of evidence of the effectiveness of the intervention. Studies are given a rating of Meets WWC Design Standards without Reservations, Meets WWC Design Standards with Reservations, or >Does Not Meet WWC Design Standards.
A related publication that was reviewed alongside the main study of interest.
Study findings for this report.
Based on the direction, magnitude, and statistical significance of the findings within a domain, the WWC characterizes the findings from a study as one of the following: statistically significant positive effects, substantively important positive effects, indeterminate effects, substantively important negative effects, and statistically significant negative effects. For more, please see the WWC Handbook.
The WWC may review studies for multiple purposes, including different reports and re-reviews using updated standards. Each WWC review of this study is listed in the dropdown. Details on any review may be accessed by making a selection from the drop down list.
Tier 1 Strong indicates strong evidence of effectiveness,
Tier 2 Moderate indicates moderate evidence of effectiveness, and
Tier 3 Promising indicates promising evidence of effectiveness,
as defined in the
non-regulatory guidance for ESSA
and the regulations for ED discretionary grants (EDGAR Part 77).