Project Activities
Researchers will first use data mining methods to build automated detectors of engagement based on student behaviors within an online program, LearnBop. The team will collect observations of student engagement in order to validate the computer generated results. In the second set of studies, the research team will collect observational, system logfile, and other extant data to capture system features, classroom implementation factors, and contextual factors. In the final years of the project, the team will complete data analyses, report generation, and other dissemination activities.
Structured Abstract
Setting
The project will analyze primary and secondary data from students in Grades 7 and 8 in a large urban Florida public school district.
Sample
The sample includes approximately 1,500 students (750 per grade) in approximately 60 classrooms (30 per grade).
Intervention
LearnBop is an online adaptive math program that offers a variety of scaffolded problems with hints and pop-up instructional videos when students make repeated mistakes.
Research design and methods
In the first phase, the team will use data mining and machine learning to build automated detectors of engagement based on student behaviors within LearnBop and will collect observations of student engagement. During the second phase, the researchers will collect observational, system logfile, and other extant data to examine system features, classroom implementation factors, and contextual factors.
Control condition
The nature of the research design does not require a control condition.
Key measures
The key measures are automated detectors of engagement, LearnBop system features, classroom implementation factors, contextual factors, and measures of student learning. Within-system measures of student learning are based on student-level performance data on LearnBop problems. Measures of robust learning include scores on state assessments and a study-administered mathematics assessment.
Data analytic strategy
Researchers will synchronize system logfiles and field observations to complete analyses related to detector creationby. Within the logfiles, student actions during each problem will be distilled into data features that represent aspects of student behavior typically associated with engagement (e.g., time between clicks and the use of hints). These features will be used to build a detector for each engagement construct (e.g., off-task behavior, confusion, and frustration), using step regression, decision trees, and other algorithms, with ongoing cross-validation analyses. The team will use three-level hierarchical models (problems nested within students nested within teachers) and structural equation modeling to estimate relationships among malleable factors, the detectors of engagement, learning measures, and contextual factors.
People and institutions involved
IES program contact(s)
Products and publications
Products: The researchers will produce preliminary evidence for how malleable system features and classroom implementation factors predict detectors of engagement and learning, and information describing how engagement may mediate links between malleable system and implementation factors and student learning. Ultimately, the team seeks to generate testable hypotheses about system and implementation improvements that will optimize student engagement and learning.
Supplemental information
Co-Principal Investigator: Baker, Ryan S.
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
