Project Activities
The research team will work with the Upgrade/MATHia platform team to add enhancements to existing lessons across a series of studies to explore their effects on metacognitive skills. In experiment 1, the team will investigate whether adding direct instruction and practice with three metacognitive skills (planning, monitoring, and evaluating) benefits students' learning and performance more than business-as-usual instruction in MATHia. In experiment 2, the team will evaluate whether it is more beneficial to receive the integrated support of all three metacognitive skills at once or just the support for one particular metacognitive skill. In experiments 3a and 3b, the team will investigate whether interleaving the three types of skills immediately is more beneficial than focusing on one skill at a time and then interleaving them. Experiment 3a will be run with eighth graders, and 3b will be run with sixth graders to examine developmental differences.
Structured Abstract
Setting
Middle schools who use MATHia, an online environment that serves U.S. schools nationwide, as part of their normal mathematics classroom activities. Schools across the United States use MATHia, and the research team will work with the team at MATHia to recruit a diverse sample.
Sample
Across the series of studies, approximately 600 sixth graders across 30 classrooms and 1700 eighth graders across 86 classrooms will participate. The research team will work with the team at MATHia to recruit a diverse sample with respect to socioeconomic factors, race, and ethnicity based on district- and school-level data.
The research team will seek to identify supports that can be implemented during mathematics learning in MATHia that improve middle school students' metacognition, mathematics knowledge, and motivation.
Research design and methods
For all studies, the team will implement a between-subjects design, with random assignment to condition at the classroom level. Conditions will involve direct instruction and practice with a particular metacognitive skill or a set of skills with only experiment 1 having a business-as-usual control condition. All experiments except experiment 3b will be conducted with eighth-grade students. In experiment 1, the team will compare adding direct instruction and practice with three metacognitive skills (planning, monitoring, and evaluating) to business-as-usual instruction in MATHia. In experiment 2, the team will compare integrated support of all three metacognitive skills at once to just the support for one particular metacognitive skill. In experiments 3a and 3b, the team will compare interleaving the three types of skills throughout learning to focusing on one skill at a time and then interleaving them. Experiment 3a will be run with eighth graders, and 3b will be run with sixth graders. For all experiments, students will answer questions to assess their metacognitive skills, mathematics knowledge, and motivation before and after the interventions. To capture their behaviors, log data of their interactions within the MATHia workspace will also be collected.
Control condition
Only the experiment 1 will have a control condition in which participants will complete their business-as-usual instruction in MATHia. The remaining experiments will compare different types of enhanced instruction.
Key measures
Metacognitive skills will be measured through researcher-developed manipulation checks and the Motivated Strategies Learning Questionnaire (MSLQ). Motivation to learn mathematics will be measured by well-established questionnaires including Achievement Goal Questionnaire Revised (AGQ-R) and an open-ended question that captures their motivational strategies. Mathematics knowledge will be assessed with a pre and posttest that we will develop based on the content in MATHia. The research team will also capture different behaviors in the log data that reflect metacognitive behaviors, such as using hints less frequently, only accessing hints after attempting to solve, and faster times at completing components within the workspace.
Data analytic strategy
The team will use multilevel regression models to examine the main effects of the different conditions on students' metacognitive skills, math knowledge, and motivation.
Cost analysis strategy
The team will comprehensively estimate the cost of each enhancement presented in each study, accounting for all key ingredients of the enhancements and any cost implications of changes to the implementation based on study findings. The team will compare these to the cost of using MATHia in the business-as-usual control condition. They will leverage multiple sources of data and test for sensitivity to ensure results are as accurate and generalizable as possible.
People and institutions involved
IES program contact(s)
Products and publications
Products: The team will disseminate findings to researchers, practitioners, administrators, digital content creators, and policymakers through social media, presentations, and publications. Findings will be shared with the team at MATHia for developers to consider integrating into a broader set of lessons throughout their platform and for teachers to incorporate into their problem-solving lessons in their classrooms.
Additional project information
This project is part of the Digital Learning Platforms to Enable Efficient Education Research Network (Digital Learning Platforms Network), which aims to leverage existing, widely used digital learning platforms for rigorous education research.
Related projects
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
