Project Activities
The researchers will iteratively develop Lumilo 2 through co-design sessions with teachers and students, hone the tool through prototyping, test it through classroom piloting, and conduct experimental classroom studies to rigorously evaluate effects of key features (namely, real-time awareness support and decision support for teachers). The researchers will focus their classroom studies to better understand how the tool affects teachers and how, in turn, the tool-supported teacher behaviors affect key student learning processes and outcomes, including motivation, sense of belonging, and self-efficacy for math learning.
Structured Abstract
Setting
The research will occur in public middle school math classrooms in locations throughout the U.S., focusing on users of MATHia, Carnegie Learning's blended curriculum for middle school and high school mathematics.
Sample
Approximately 60 grade 6 to 8 math classrooms will participate in the piloting and experimental classroom studies. Participating schools will have at least 25 percent minorities and 25 percent low-income students.
The tool (Lumilo 2) will provide real-time awareness support and decision support to teachers as students engage in self-paced work with the tutoring software. Aided by the tool, teachers monitor their classes and help students whom the tutoring software is not well-equipped to help. The tool will be used with two software platforms, one oriented towards research (Lynnette, by Carnegie Mellon University) and one oriented towards use at scale in real educational settings (MATHia, by Carnegie Learning).
Research design and methods
The researchers will conduct design-based research, data mining and machine learning, decision modeling, classroom piloting, and two experimental classroom experiments. These two classroom experiments include one to test the value of awareness support in real time(with up-to-the minute analytics of student learning as opposed to static analytics up to the current day) and one to test the value of adding real-time decision support (where the orchestration tool generates recommendations for whom to help and how). For these two experiments, they will randomly assign classes will be randomly assigned to the experimental condition or the control condition.
Control condition
In the first classroom experiment, teachers in the control condition will use a limited-functionality version of the tool with static rather than real-time analytics. In the second, teachers in the control condition will use a tool with awareness support only.
Key measures
Using analytics of classroom observations and process data from Lumilo 2, the researchers will measure teacher awareness of students' progress and struggle, teacher "visits" to students (as students engage in self-paced work), whether teacher help to students is timely and targeted, efficiency and students' learning processes, effects of teacher visits on these processes, learning outcomes, and student motivation (belonging, self-efficacy for math learning). Key measures of learning outcomes include curriculum-embedded assessments in MATHia that are related to the specific math content being targeted.
Data analytic strategy
The researchers will use machine learning methods to develop suggestions for teacher decision support. For student learning processes, they will use "learning curves," a standard practice in the learning sciences literature on adaptive learning and knowledge component modeling. They will analyze data from the randomized field pilot trials using statistical best practices for the comparison of the experimental and control conditions, and they will use causal modeling techniques to study relations among tool conditions, teacher and student process measures, and student learning outcomes.
Cost analysis strategy
The researchers will conduct an ingredients-approach cost analysis to better understand the incremental costs associated with the Lumilo 2 teacher tool, added to the MATHia curriculum, as compared to implementation of MATHia without the new tool.
People and institutions involved
IES program contact(s)
Project contributors
Products and publications
This project will result in a fully developed mixed-reality teacher orchestration tool, Lumilo 2, running on smart glasses, that will provide real-time awareness support and decision support to teachers, as students engage in self-paced work with tutoring software. The project will also result in peer-reviewed publications and presentations as well as additional dissemination products that reach education stakeholders such as practitioners and policymakers.
Publications:
ERIC Citations: Find available citations in ERIC for this award here.
Related projects
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
