Purpose: In this study, researchers obtained evidence on the effectiveness of Cognitive Tutor Geometry, a high school geometry curriculum that has shown promise in earlier quasi-experimental studies. The program combines individualized tutorial software with teacher-guided group work and problem-solving. As of the early 2000s, data from the National Assessment of Education Progress showed very low mathematics proficiency among high school students, with large gaps in the performance of students of different racial, ethnic, and income groups. But many educators did not always know how to address the problem because few curricula are supported by rigorous scientific research. At the end of the project, the researchers provided evidence about whether implementation of Cognitive Tutor Geometry in one large district improved student performance on the state's end-of-course geometry assessment.

Structured Abstract

THE FOLLOWING CONTENT DESCRIBES THE PROJECT AT THE TIME OF FUNDING

Setting: The evaluation is being conducted in eight high schools in the Baltimore County Public School District, an urban fringe district that serves students from a wide range of racial/ethnic and socioeconomic backgrounds.

Sample: The participants are approximately 2,000 high school students and 16 math teachers.

Intervention: Cognitive Tutor Geometry is designed to provide individualized instruction to address students' specific needs. The curriculum includes challenging problems that reflect real-world situations and provide opportunities for students to progress from concrete to abstract thinking. Students spend 40 percent of their class time using individualized tutorial software built on a detailed computational model of student thinking, and 60 percent of class time will be devoted to teacher-guided group work and problem solving. As part of implementation, teachers receive four days of teacher training. The curriculum has shown evidence of promise from quasi-experimental studies.

Research Design and Methods: The study is a randomized controlled trial in which two teachers in each participating high school will teach geometry concurrently during two periods per day. One teacher is randomly assigned to teach Cognitive Tutor Geometry during the earlier period, and the other teacher uses the school's existing geometry curriculum. During the later period, the teachers switch curricula. For both periods, students enrolled in geometry are randomly assigned to experimental or control classes. The study will run for 2 years, using the same teachers to the extent possible, so as to capture any improvements in implementation after a year of experience with the curriculum.

Control Condition: Students in the control group receive their high school's usual geometry instruction.

Key Measures: Pre- and post-test data will be collected on student performance on the Maryland State Department of Education Geometry Assessment. In addition, student survey data will be collected on other outcomes that may influence long-term student achievement, such as mathematics confidence and attitudes, and career plans. Data will be collected on student course-taking.  The fidelity of implementation will be monitored using a combination of classroom observations, teacher interviews, and a collection of classroom artifacts including lesson plans, logs, and teacher-developed assessments.

Data Analytic Strategy: The analysis strategy uses hierarchical linear models to estimate the impact of Cognitive Tutor Geometry on test scores in geometry and generalized mixed models to estimate the impact on dichotomous variables including course taking, attitudes, and career plans.

Products and Publications

ERIC Citations:  Find available citations in ERIC for this award here.

WWC Review:

Pane, J. F., McCaffrey, D. F., Slaughter, M. E., Steele, J. L., & Ikemoto, G. S. (2010). An experiment to evaluate the efficacy of Cognitive Tutor Geometry. Journal of Research on Educational Effectiveness, 3(3), 254–281. [WWC Review]

Journal articles

Ikemoto, G. S., Steele, J. L., & Pane, J. F. (2016). Poor implementation of learner-centered practices: A cautionary tale. Teachers College Record, 118(13), 1–34.

Pane, J. F., McCaffrey, D. F., Slaughter, M. E., Steele, J. L., & Ikemoto, G. S. (2010). An experiment to evaluate the efficacy of Cognitive Tutor Geometry. Journal of Research on Educational Effectiveness, 3(3), 254–281.