|A Randomized Controlled Study of the Effects of Intelligent Online Chemistry Tutors in Urban California School Districts
|Science, Technology, Engineering, and Mathematics (STEM) Education [Program Details]
|Efficacy and Replication
Purpose: Student knowledge and skills in science in the U.S. needs to be improved. A National Science Board report in 2006 indicates that while science achievement has improved slightly on some measures over the last few years, it is far below goals set by the National Science Boardís Commission on Precollege Education in Mathematics, Science, and Technology almost three decades ago. Moreover, the performance of students in the U.S. is lagging when compared to other countries. The purpose of this project is to test the efficacy of the Quantum Chemistry Tutors, a suite of computer-based cognitive tutors that are designed to give individual tutoring to high school students in 12 chemistry topics. The findings from this study will not only tell us about the efficacy of this specific intervention but will also contribute to a better understanding of the general effects of the use of intelligent tutoring software on student learning, when used in conjunction with a range of commercially available science curricula.
Project: The Quantum Tutors for Chemistry to be evaluated cover topics representative of comprehensive content for a full high school chemistry course. Each Tutor uses natural language to scaffold student thinking and performance through questioning, modeling, illustration, and explanation of issues within the context of the student's own work. The Quantum Tutors are designed to supplement or be incorporated into any existing curriculum. To test the efficacy of the Quantum Chemistry Tutors, 45 high school teachers in schools assigned to the treatment group will teach their usual curriculum with the 12 chemistry tutors and 45 teachers in schools selected for the control group will use only their usual curriculum.
Products: The expected outcomes of this research include published reports on the efficacy of the Quantum Chemistry Tutors for use in increasing studentsí science achievement.
Purpose: The purpose of this project is to test the efficacy of the Quantum Chemistry Tutors, a suite of computer-based cognitive tutors that are designed to give individual tutoring to high school students in 12 chemistry topics.
Setting: The setting for this study is high school chemistry classes in Oakland, San Diego, and Santa Clara, CA.
Population: The study sample will consist of approximately 90 high school chemistry teachers and their students in the urban high schools.
Intervention: The Quantum Chemistry Tutors cover topics representative of comprehensive content for a full high school chemistry course and are designed to provide one-on-one instruction in situations where qualified human tutors are unavailable or impractical. Students access the tutor by logging into an Internet site, and use it entirely within the web browser. Students can use the tutor to get immediate feedback when solving problems and to ask questions in context about the concepts and processes being studied. The tutors adjust explanations and model solutions to work within the student's range of understanding. At the same time, the tutorsí design encourages the student to reach slightly beyond that level of comfort by displaying medium-level questions that help the student develop the confidence to approach a more difficult skill.
All high school chemistry teachers in the treatment condition will teach their students using their regular curriculum and with the Quantum Chemistry Tutors for all 12 topics. Embedded within the software for each tutor is a content index that lists primary and secondary learning objectives. Teachers are not required to change the way they teach to incorporate the tutors. To ensure that there is a baseline level of treatment in the study, teachers selected for the treatment group will use the tutors in both their classroom instruction and for homework. Quantum will conduct online training for treatment-group teachers as well as provide ongoing e-mail and telephone technical support. The online teacher training sessions will be conducted via web conference to review and demonstrate the user management system and one or two particular Quantum Tutors as examples.
Research Design and Methods: The study will use a group-randomized design, whereby participating schools will be randomly assigned, based on a computerized random number algorithm, to either a treatment or a control group. Forty-five high school teachers in schools assigned to the treatment group will teach their usual curriculum with the 12 Quantum Chemistry Tutors and 45 teachers in schools selected for the control group will use only their usual curriculum.
Control Condition: The control condition does not include the Quantum Chemistry Tutors as a supplement to the chemistry curriculum.
Key Measures: Student achievement in science will be measured by the California STAR CST in chemistry, and by a test developed specifically for this study. Intervention implementation will be monitored through the collection of survey data from teachers in both treatment and control classrooms and from observational data from a sample of treatment group classrooms.
Data Analysis Strategy: The primary hypothesis testing analyses will involve fitting conditional mixed effects analysis of covariance (ANCOVA) models, with an additional term to account for the nesting of students within schools. Mixed effects models will also be used to examine how student use of Quantum Chemistry Tutors is related to student performance.
Related IES Projects: Integrated Software for Artificial Intelligence Tutoring and Assessment in Science (R305K040008) and Integrated Software for Artificial Intelligence Tutoring and Assessment in Science (R305A070067)