Skip Navigation
Funding Opportunities | Search Funded Research Grants and Contracts

IES Grant

Title: Efficacy of an Integrated Digital Elementary School Mathematics Curriculum
Center: NCER Year: 2013
Principal Investigator: Roschelle, Jeremy Awardee: SRI International
Program: Education Technology      [Program Details]
Award Period: 4 years (7/1/2013-6/30/2017) Award Amount: $3,496,525
Goal: Efficacy and Replication Award Number: R305A130400

Co-Principal Investigators: Nicole Shechtman, Mingyu Feng, Corinne Singleton

Purpose: The purpose of this project is to evaluate the efficacy of Reasoning Mind (RM), a fully developed digital mathematics curriculum, which can serve as the primary, full-year (grade 5) curriculum. RM uses technology to give students and teachers instant feedback, implement a differentiated instruction model, provide supports for learning, and offer engaging activities. RM is oriented to developing students' understanding of the key topics on the pathway to algebra; topics include place value system, fractions, rational numbers, geometric measurement, and graphing points in a coordinate plane, and the curriculum is aligned with the Common Core State Standards for Mathematics (CCSSM). Researchers in this project will contrast RM's digital approach to typical paper-based curricula, which leave to teachers' discretion differentiated instruction, integration of technology, and selection of additional student supports.

Project Activities: In this study, researchers will implement a randomized field trial to test whether RM, a full-year digital core curriculum increases mathematics achievement for grade 5 students. In addition, the team will examine how duration of intervention and intensity of use interact with student outcomes, and will explore the hypothesized mediating role of technology features in supporting mathematics outcomes.

Products: The products of this project will be evidence of the efficacy of a fully developed digital core curriculum for improving student achievement in grade 5 mathematics. Peer-reviewed publications will also be produced.

Structured Abstract

Setting: This study will be conducted in elementary schools in West Virginia.

Sample: Fifty-two schools will be recruited, and all fifth-grade classrooms in those schools will participate. Approximately 3,500 fifth-grade students will participate in the study.

Intervention: Aligned with the CCSSM, RM focuses on key topics on the pathway to algebra, including place value system, fractions, rational numbers, geometric measurement, and graphing points in a coordinate plane. Students work independently in solving mathematics problems on a computer, and the intervention provides dynamic feedback (e.g., accuracy of responses, hints) and implements a differentiated instruction model based on performance. The system also incorporates motivational components, such as games. In addition, teachers receive up to 60 hours of professional development (e.g., in mathematical content) and implementation support (e.g., understanding progress reports on individual students and entire classroom).

Research Design and Methods: The research team will conduct a randomized control experiment, in which schools will be randomly assigned to treatment or control condition, with participation lasting two school years. In Year 1, researchers will recruit schools to participate in the study, randomly assign participating schools to condition, and will refine research instruments. In the initial year of participation, Year 2 of the study, treatment teachers will receive RM professional development. The research team will examine teachers' implementation of RM, gather information characterizing the control curricula, and explore facilitators and barriers to implementation at classroom and school levels. The team will use information collected in Year 1 to improve the usability of the curriculum and utility of the teacher professional development that will be delivered in Year 2. The test of the efficacy of the intervention will be carried out in Year 2, with the fifth-grade students in their teachers' second year of implementation. The team will continue to collect information about implementation and the control condition, and will measure student interest in mathematics and achievement. The final year of the project will be focused on analyzing the data and reporting the findings.

Control Condition: Grade 5 classrooms in the control condition will follow their typical practice (e.g., use paper textbooks), and teachers will receive professional development according to district policies and offerings. Control schools will be asked to not adopt a blended or differentiated instruction-focused digital curriculum product during the study.

Key Measures: Student achievement will be assessed with the Smarter Balanced summative assessment, which West Virginia has selected as its statewide standardized assessment of mathematics. Students' productive dispositions toward mathematics will also be measured using the Attitudes Toward Mathematics Inventory, and teachers' mathematical knowledge for teaching will be measured using either the Pedagogical Content Knowledge Test or released items from the Learning Mathematics for Teaching Project. Treatment and control teachers will also complete electronic teacher logs. Data measuring student problem solving, use of remote tutoring, and teachers' use of reports will be gathered directly from the RM system records. Measures of implementation will be collected via classroom observations, stakeholder interviews, and teacher survey.

Data Analytic Strategy: Data analyses will include a hierarchical linear regression model of mean differences in grade 5 achievement scores between students in treatment and control conditions, with random assignment at the school level. Covariates will include prior achievement, SES, rural location. Secondary analyses will examine differential impact for students with low or high prior achievement. Exploratory analyses will examine students' use of remote tutoring; teachers' use of RM reports to adjust instruction; students' productive time on task; and implementation compliance of both students and teachers. Regression models will be conducted on school- and student-level prior achievement measures to examine if there were unmeasured confounding factors associated with both the predictor and outcome variables.


Journal article, monograph, or newsletter

Roschelle, J., Gaudino, S. and Darling, S. (2016). Designing for Consistent Implementation of a 5th Grade Digital Math Curriculum. International Journal of Design for Learning, 7(2): 41–56.