Project Activities
This project involves two studies: a replication study aimed at confirming the results found in the pilot study; and an efficacy study conducted as a fully powered randomized control trial. The replication study will include students in 63 classrooms, randomly assigned at the classroom level to one of three conditions: one-on-one MyST tutoring; small-group human tutoring; or business as usual. The efficacy study will include 84 classrooms (42 in Year 2 of the project and 42 in Year 3) randomly assigned to one of two conditions: one-on-one MyST tutoring; or business-as-usual.
Structured Abstract
Setting
This study will be conducted in elementary schools from three districts in Colorado.
Sample
Approximately 2,200 students in grades three, four, and five will participate in this study.
My Science Tutor (MyST) is a computer-based intervention designed to supplement classroom instruction using the Full Option Science System (FOSS) curriculum. FOSS is a non-textbook based science curriculum focused on allowing students the opportunity to actively construct ideas through inquiry, experimentation, and analysis. MyST uses a life-like 3D character named Marni who asks open-ended questions and responds to students' speech. Marni and the student have a shared dialog about science which allows MyST to continuously assess students' knowledge on a particular topic. MyST uses character animation, automatic speech recognition, dialog modeling, and language and speech generation. Multimedia content is also integrated into the intervention during 20-minute tutorials aligned with FOSS content which are intended to provide the scaffolding for students to think and talk about science with Marni.
Research design and methods
This project involves two studies. The first is a replication study of the pilot study from the original IES funded Development and Innovation grant. For this study, 63 classrooms in third, fourth, and fifth grades will be randomly assigned to one of three conditions: one-on-one MyST tutoring; small-group human tutoring; or business as usual. The second part of this project is a two-cohort efficacy study. In each of Years 2 and 3, 42 third-, fourth-, and fifth-grade classrooms will be randomly assigned to receive either one-on-one MyST tutoring or business as usual. Students will be pre-tested and then receive one academic year of tutoring or business as usual. Students in both groups will be post-tested.
Control condition
The replication study has two control conditions: a small-group human tutoring condition and a business-as-usual control condition. The efficacy study uses a business-as-usual control condition.
Key measures
To assess science knowledge, students will complete the Assessing Science Knowledge (ASK) assessments, which are part of the FOSS curriculum, at both pre-test and post-test. Additionally, the Measures of Academic Process (MAP) will be used as pre- and post-tests, and the Transitional Colorado Assessment Program (TCAP) tests will be used as post-test for fifth-graders (the TCAP is only administered in the spring of fifth grade). Finally, student and teacher surveys will be used to examine perceptions of tutoring and problems with the MyST program.
Data analytic strategy
To examine the efficacy of MyST, researchers will construct a hierarchical linear model within a structural equation modeling framework. The analyses will nest students within classrooms. Additionally, the models will account for change within grade (pre- and post-test) and across grade (third, fourth, and fifth grade).
People and institutions involved
IES program contact(s)
Project contributors
Products and publications
The final product of this project will be evidence of the efficacy of MyST to impact students' science knowledge. Peer-reviewed publications will also be produced.
Related projects
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
