Skip Navigation
Funding Opportunities | Search Funded Research Grants and Contracts

IES Grant

Title: A Multi-Site Randomized Controlled Trial to Assess the Efficacy of the NumberShire Level 1 Gaming Intervention for Improving Math Outcomes for Students With or At Risk for Math Learning Disabilities
Center: NCSER Year: 2016
Principal Investigator: Clarke, Ben Awardee: University of Oregon
Program: Technology for Special Education      [Program Details]
Award Period: 4 years (7/01/2016-6/30/2020) Award Amount: $3,499,086
Type: Efficacy and Replication Award Number: R324A160125

Co-Principal Investigators: Nelson, Nancy; Doabler, Christian; Fien, Hank; Smolkowski, Keith

Purpose: The purpose of this study is to determine the efficacy of the NumberShire Level 1 gaming intervention, aimed at improving mathematics achievement for first-grade students with or at risk for mathematics learning disabilities. Intervention to strategically accelerate math learning in early elementary school can help prevent costly remediation in later grades. According to the National Assessment of Educational Progress (2013), 45% of all fourth-grade students with disabilities scored below the basic level. More research is needed on the efficacy of early learning interventions covering critical early math content such as whole number concepts. This project seeks to fill this gap by studying the efficacy of one such gaming intervention, NumberShire Level 1, for improving mathematics achievement for first-grade students.

Project Activities: The researchers will test the impact of the NumberShire Level 1 gaming intervention on proximal and distal mathematics outcomes for students in first and second grade using a randomized controlled trial. They will also determine the relationship between moderator variables (e.g., student demographics, self-regulation) and math outcomes and test a theory of change about the relationships between mathematics outcomes and behavioral self-regulation, productive disposition towards mathematics, and intervention dosage.

Products: The products of this project include evidence of the efficacy of the NumberShire Level 1 intervention for improving first-grade students' mathematics achievement, evidence of any moderation by factors such as demographics and self-regulation, and an understanding of the interrelations between underlying processes and the math outcomes. Products will also include peer-reviewed publications, presentations, and other forms of dissemination to practitioners and parents.

Structured Abstract

Setting: The study will be conducted in elementary schools in Oregon, Nevada, and Massachusetts. The schools include high percentages of English language learners and students from economically disadvantaged backgrounds.

Sample: The project will involve approximately 1,272 first-grade students identified as being at risk for mathematics learning disabilities. They will be recruited from 39 classrooms (13 elementary schools in each of the three states), with one teacher and one instructional assistant per classroom also participating.

Intervention: NumberShire Level 1 is an intervention that was developed to target students with or at risk for mathematics learning disabilities. The intervention includes 12 hours of individualized instruction in a game environment, occurring for 15 minutes per day, 4 days a week, for 12 weeks. The key features of the game include evidence-based explicit instructional and technological design and delivery, critical early math content focused on whole number concepts, and an engaging gaming platform. Trained instructional assistants facilitate the intervention in a computer lab or other location where students use the game.

Research Design and Methods: The researchers will conduct a multi-site randomized controlled trial, blocking on classroom, to determine the efficacy of the NumberShire Level 1 game on student mathematics outcomes. Students will be randomly assigned and receive their respective math interventions (NumberShire Level 1 or usual instruction) in the spring of first grade, with data collection post intervention and at follow up in second grade. The research team will also conduct a series of moderation analyses to understand the role of behavioral self-regulation and productive disposition towards mathematics in predicting differences in mathematics outcomes for students at risk for mathematics learning disabilities. Classroom observations and implementation logs will be used to document fidelity of implementation, and total number of practice items students complete during a session will be used as a measure of intervention dosage.

Control Condition: Students in the control condition will receive business-as-usual math instruction.

Key Measures: Student outcome measures will include researcher-developed measures of conceptual understanding and procedural fluency with whole numbers, easyCBM Math (a standardized, computerized assessment), Assessing Student Proficiency in Early Number Sense, NumberShire Proximal Assessment (previously developed by the researchers), and the Test of Early Mathematics Ability. Fidelity of implementation will be measured with the Technology Observation Tool. Behavioral self-regulation will be measured by the Head-Toes-Knees-Shoulders observation assessment and the Child Behavior Rating Scale completed by teachers. Productive disposition towards mathematics will be measured through a modified survey of self-efficacy for math learning, growth mindset in math, and perceived value of math learning.

Data Analytic Strategy: Data will be analyzed using a mixed-model analysis of covariance approach that accounts for the nested structure of the data at posttest after year 1 and at the follow-up in second grade. This model will be extended to look at interactions to test the moderator hypotheses. Multilevel multiple regression analyses will be used to examine the theoretical relationships between the underlying processes (i.e., active engagement in frequent practice, self-regulation, productive disposition) and mathematics outcomes.

Related IES Projects: Development of a Game-based Integrated Learning and Assessment System to Target Whole Number Concepts (Project NumberShire) (R324A120071); Project NumberShire: A Game-Based Integrated Learning and Assessment System to Target Whole Number Concepts (EDIES11C0026); Numbershire II: Math Games for 2nd Graders with or at-Risk for LD (EDIES12C0045); Numbershire II: Development Of A Second Grade Game-Based Integrated Learning System To Target Whole Numbers And Operations In Base Ten And Operations And Operations And Algebraic Thinking (EDIES13C0045)

Products and Publications

Additional online resources and information: NumberShire