Project Activities
The project will implement NSK in elementary classrooms to determine its impact on kindergarten students and assess its continued effects in first grade using a series of randomized controlled trials. The research team will test student-level variables hypothesized to interact with the intervention to predict math outcomes, examine relationships between underlying processes and student outcomes, and explore classroom-level factors that may influence the sustainability of intervention implementation.
Structured Abstract
Setting
Research activities will occur in elementary schools from two large, diverse, urban districts in Kentucky.
Sample
The study will include approximately 1,224 students from about 50 kindergarten classrooms in 20 schools. These students will have or be at risk for learning disabilities and come from racially and ethnically diverse and economically disadvantaged backgrounds, including English learners.
Intervention
NSK is an intervention designed to improve math achievement for students with or at risk for MLD. It consists of 10 hours of individualized instructional game play focused on building students' understanding of whole number concepts and skills identified in the Common Core State Standards for Mathematics (CCSS-M). To progress through the game, students engage in a series of interactive math activities, strategically designed around three key components—evidence-based explicit instructional and technological design and delivery features, critical early math content focused on whole number concepts, and a highly engaging gaming platform. Students assigned to the intervention group will receive NSK in addition to core math instruction 15 minutes per day, 4 days per week, for 10 weeks. Trained, project-supported, instructional assistants will facilitate NSK.
Research design and methods
Using a randomized controlled trial across three cohorts, the research team will assign individual students to the NSK intervention condition or control condition, blocking on classroom. This trial will be used to determine the impact of the intervention on student outcomes as well as explore relationships between the intervention effects on early math achievement and students' behavioral self-regulation (attentional focusing, working memory, and inhibitory control), self-efficacy in mathematics, demographic characteristics, MLD status, and English proficiency levels. They will also examine the effect of various classroom-level factors, including fidelity of implementation and teacher characteristics (such as training, experience using technology for instruction, and perceptions of the usability and relevance of NSK), which research demonstrates plays a role in teachers' uptake and implementation of technology. Following the efficacy trials, in the final year of the project, the researchers will conduct an implementation study to explore the extent to which teachers continue to use NSK with limited research staff involvement and the impact of teacher characteristics on their continued implementation, as well as follow up on student outcomes for those who had participated in the trial during kindergarten.
Control condition
Students assigned to the control condition will receive business-as-usual math intervention in addition to core math instruction. These print-based and/or technology-based, supplemental interventions focus on a variety of CCSS-M topics, delivered by instructional assistants or teachers several times per week, 15–30 minutes each day.
Key measures
This project will collect mathematics screening and outcome data using proximal, curriculum-based measures of conceptual understanding and procedural fluency and distal math outcome measures including the Number Sense Screener, Assessing Student Proficiency in Early Number Sense, the ROOTS Assessment of Early Numeracy Skills, and NWEA's Measures of Academic Progress (MAP) Growth K–2 Mathematics Assessment. The researchers will collect administrative data for student-level moderation variables (English proficiency and MLD status, prior math achievement on MAP, demographic characteristics). They will also use Head-Toes-Knees-Shoulders to assess behavioral self-regulation and adapt the Math and Me Survey to assess self-efficacy. The research team will collect data on fidelity of implementation and document features of math instruction using a researcher-developed direct observation tool and the Classroom Observations of Student–Teacher Interactions. They will use instructor logs to assess fidelity of implementation and NSK-specific adaptation of the Primary Intervention Rating Scale to measure teacher perceptions of the social validity of the intervention. The researchers will summarize gameplay metrics, including average number of logins per week, average number of game sessions attempted per week, average length of game sessions, and total number of game sessions completed as additional, specific measures of fidelity of implementation and student engagement in the intervention.
Data analytic strategy
To test main effects of the NSK intervention, the research team will use a mixed-model analysis of covariance approach that accounts for the nested structure of the data at posttest after kindergarten and at the follow up in first grade. Multiple regression analyses will be used to examine associations between the underlying processes (including gains in behavioral self-regulation and self-efficacy and intervention dosage) and mathematics outcomes.
Cost analysis strategy
Researchers will identify, measure, and value the resources needed to implement NSK, including new expenditures and reallocations of existing resources, using the ingredients method. Using this information, they will estimate single-year costs, expected initial costs, and costs for the reasonable lifespan of the program in per-school and per-student terms. A cost-effectiveness ratio will be calculated for NSK relative to the control condition by dividing program costs by the effect size.
Products and publications
Products: The project team will disseminate the NSK game and supplementary resources for use by educators, as well as share research findings, and practical recommendations with stakeholders using a variety of approaches, including commercialization of the app. Products will also include the final dataset, peer-reviewed publications and presentations, and additional dissemination products that reach education stakeholders.
ERIC Citations: Find available citations in ERIC for this award here.
Related projects
Supplemental information
Co-Principal Investigators: Choo, Sam; Kosty, Derek; Clarke, Ben; Smolkowski, Keith
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.