|Investigating the impact of metacognitive supports on students' mathematics knowledge and motivation in MATHia
|Digital Learning Platforms to Enable Efficient Education Research Network [Program Details]
|2 years (01/01/2024 – 12/31/2025)
Related Network Teams: This project is part of the Digital Learning Platforms to Enable Efficient Education Research Network (Digital Learning Platforms Network), which aims to leverage existing, widely used digital learning platforms for rigorous education research and includes the following other projects — The SEER Research Network for Digital Learning Platforms (R305N210034), The Canvas+Terracotta LMS-Based Experimental Education Research Platform (R305N210035), The ASU Learning at Scale (L@S) Digital Learning Network (R305N210041), MATHia: A Digital Learning Platform Supporting Core and Supplemental Instruction in Middle and High School Mathematics (R305N210045), Revisions to the ASSISTments Digital Learning Platform to Expand Its Support for Rigorous Education Research (R305N210049), Efficient Education Research via the OpenStax Learning Platform (R305N210064), Now I See It: Supporting Flexible Problem Solving in Mathematics through Perceptual Scaffolding in ASSISTments (R305N230034)
Purpose: The purpose of this project is to explore the effects of evidence-based enhancements implemented during mathematics problem solving in UpGrade/MATHia on middle school students' metacognitive skills, mathematics knowledge, and motivation. Mathematics learning is an incremental process that requires intensive metacognitive activity. Students must create a plan to solve a problem, monitor and control their understanding, and evaluate their learning process. Metacognitive skills are essential for learning in mathematics, but unfortunately, many students do not receive instruction on these skills in mathematics classes and can struggle with adequately applying them.
Project Activities: The research team will work with the Upgrade/MATHia platform team to add enhancements to existing lessons across a series of studies to explore their effects on metacognitive skills. In experiment 1, the team will investigate whether adding direct instruction and practice with three metacognitive skills (planning, monitoring, and evaluating) benefits students' learning and performance more than business-as-usual instruction in MATHia. In experiment 2, the team will evaluate whether it is more beneficial to receive the integrated support of all three metacognitive skills at once or just the support for one particular metacognitive skill. In experiments 3a and 3b, the team will investigate whether interleaving the three types of skills immediately is more beneficial than focusing on one skill at a time and then interleaving them. Experiment 3a will be run with eighth graders, and 3b will be run with sixth graders to examine developmental differences.
Products: The team will disseminate findings to researchers, practitioners, administrators, digital content creators, and policymakers through social media, presentations, and publications. Findings will be shared with the team at MATHia for developers to consider integrating into a broader set of lessons throughout their platform and for teachers to incorporate into their problem-solving lessons in their classrooms.
Setting: Middle schools who use MATHia, an online environment that serves U.S. schools nationwide, as part of their normal mathematics classroom activities. Schools across the United States use MATHia, and the research team will work with the team at MATHia to recruit a diverse sample.
Sample: Across the series of studies, approximately 600 sixth graders across 30 classrooms and 1700 eighth graders across 86 classrooms will participate. The research team will work with the team at MATHia to recruit a diverse sample with respect to socioeconomic factors, race, and ethnicity based on district- and school-level data.
Factors: The research team will seek to identify supports that can be implemented during mathematics learning in MATHia that improve middle school students' metacognition, mathematics knowledge, and motivation.
Research Design and Methods: For all studies, the team will implement a between-subjects design, with random assignment to condition at the classroom level. Conditions will involve direct instruction and practice with a particular metacognitive skill or a set of skills with only experiment 1 having a business-as-usual control condition. All experiments except experiment 3b will be conducted with eighth-grade students. In experiment 1, the team will compare adding direct instruction and practice with three metacognitive skills (planning, monitoring, and evaluating) to business-as-usual instruction in MATHia. In experiment 2, the team will compare integrated support of all three metacognitive skills at once to just the support for one particular metacognitive skill. In experiments 3a and 3b, the team will compare interleaving the three types of skills throughout learning to focusing on one skill at a time and then interleaving them. Experiment 3a will be run with eighth graders, and 3b will be run with sixth graders. For all experiments, students will answer questions to assess their metacognitive skills, mathematics knowledge, and motivation before and after the interventions. To capture their behaviors, log data of their interactions within the MATHia workspace will also be collected.
Control Condition: Only the experiment 1 will have a control condition in which participants will complete their business-as-usual instruction in MATHia. The remaining experiments will compare different types of enhanced instruction.
Key Measures: Metacognitive skills will be measured through researcher-developed manipulation checks and the Motivated Strategies Learning Questionnaire (MSLQ). Motivation to learn mathematics will be measured by well-established questionnaires including Achievement Goal Questionnaire Revised (AGQ-R) and an open-ended question that captures their motivational strategies. Mathematics knowledge will be assessed with a pre and posttest that we will develop based on the content in MATHia. The research team will also capture different behaviors in the log data that reflect metacognitive behaviors, such as using hints less frequently, only accessing hints after attempting to solve, and faster times at completing components within the workspace.
Data Analytic Strategy: The team will use multilevel regression models to examine the main effects of the different conditions on students' metacognitive skills, math knowledge, and motivation.
Cost Analysis: The team will comprehensively estimate the cost of each enhancement presented in each study, accounting for all key ingredients of the enhancements and any cost implications of changes to the implementation based on study findings. The team will compare these to the cost of using MATHia in the business-as-usual control condition. They will leverage multiple sources of data and test for sensitivity to ensure results are as accurate and generalizable as possible.
Related IES Projects: Children's Longitudinal Development from Pre-K through High School as Disrupted by COVID-19 (R305A210130)