Skip Navigation

icon of glasses and a book Science, Technology, Engineering, and Mathematics (STEM) Education

Grantees

- OR -

Investigator

- OR -

Goals

- OR -

FY Awards

- OR -

Effectiveness of Cognitive Tutor Algebra One Implemented at Scale

Year: 2007
Name of Institution:
RAND Corporation
Goal: Scale-Up Evaluations
Principal Investigator:
Pane, John F.
Award Amount: $5,999,950
Award Period: 5 years
Award Number: R305A070185

Description:

Purpose: On the 2000 National Assessment of Educational Progress 17 percent of grade 12 students were at or above the proficient level in mathematics; 35 percent scored below the basic level of proficiency. Although many educators recognize the need for improved mathematics instruction, school staff currently have few evidence-based options from which to choose. The primary purpose of this study is to evaluate the impact of the Cognitive Tutor® Algebra I curriculum on mathematics achievement when the curriculum is implemented at scale—that is, when it is implemented across diverse school populations and conditions and with no more support than schools would have access to if they had selected Cognitive Tutor as their algebra curriculum apart from participation in a research project.

Project Activities: The research team is conducting a cluster randomized trial in which schools are randomly assigned either to continue to use their current algebra curriculum (control condition) or to use the Cognitive Tutor Algebra I curriculum. The Cognitive Tutor is designed to promote students' understanding of algebraic concepts and principles, problem solving skills, and mastery of higher-order mathematical concepts. A central component of the Cognitive Tutor is an automated computer-based tutor that provides individualized instruction to address students' specific needs. The individualization is built into the software, and is facilitated by detailed computational models of student thinking in algebra. In addition to studying the impact of Cognitive Tutor, the researchers are examining factors that may affect the implementation of Cognitive Tutor and other technology-based mathematics curricula.

Products: Products from this study include published papers regarding the wide-scale effectiveness of a computer-based algebra program for middle school and high school.

Structured Abstract

Purpose: The primary purpose of this study is to evaluate the impact of the Cognitive Tutor® Algebra I curriculum on mathematics achievement when the curriculum is implemented at scale—that is, when it is implemented across diverse school populations and conditions and with no more support than schools would have access to if they had selected Cognitive Tutor as their algebra curriculum apart from participation in a research project.

Setting: Fifty-four high schools and 68 middle schools from school districts in Texas, Connecticut, New Jersey, Michigan, and Kentucky are participating.

Population: Participating school districts are demographically diverse, with African American populations ranging up to 67 percent, Hispanic populations ranging up to 58 percent, economically disadvantaged populations ranging up to 96 percent, and English learner populations ranging up to 29 percent. For the middle school study, all sixth- to eighth-graders in the designated classes in a given school will participate; for the high school study, all ninth- to 12th-graders in the designated classes in a given school will participate.

Intervention: The Cognitive Tutor is designed to promote students' understanding of algebraic concepts and principles, problem solving skills, and mastery of higher-order mathematical concepts. A central component of the Cognitive Tutor is an automated computer-based tutor that provides individualized instruction to address students' specific needs. The individualization is built into the software, and is facilitated by detailed computational models of student thinking in algebra. Teachers receive four days of training on the new curriculum as part of a standard curriculum package.

Research Design and Methods: The research team is conducting a cluster randomized trial in which schools are randomly assigned either to continue to use their current algebra curriculum (control condition) or to use the Cognitive Tutor Algebra I curriculum.

Control Condition: The control classrooms use the Algebra I curricula currently in place in their schools.

Key Measures: The researchers are using exams from the CTB/McGraw-Hill TerraNova Algebra Assessment System for the primary student achievement outcome measure, as well as for the pre-test. A student survey provides additional information about student confidence and attitudes about mathematics and technology, plans for subsequent course taking and college admission testing, and schooling and career plans.

Data Analytic Strategy: Analyses use hierarchical linear models and generalized linear mixed models. These models allow for school- and teacher-level clustering and capture potential school by treatment interactions. These analyses are being complemented with qualitative analyses to gain a description of processes of curricular implementation and factors influencing implementation.

Publications

Journal article, monograph, or newsletter

Kaufman, J.H., Karam, R., Pane, J., and Junker, B. (2012). How Curriculum and Classroom Achievement Predict Teacher Time on Lecture- and Inquiry-Based Mathematics Activities. Journal of Mathematics Education at Teachers College, 3(2): 56–62.

Pane, J.F., Griffin, B.A., McCaffrey, D.F., and Karam, R. (2014). Effectiveness of Cognitive Tutor Algebra I at Scale. Educational Evaluation and Policy Analysis, 36(2): 127–144.

Nongovernment report, issue brief, or practice guide

Daugherty, L., Phillips, A., Pane, J.F., and Karam, R. (2012). Analysis of Costs in an Algebra I Curriculum Effectiveness Study.Santa Monica, CA: RAND Corporation.

Pane, J.F., Griffin, B.A., McCaffrey, D.F., Karam, R., Daugherty, L., and Phillips, A. (2013). Does an Algebra Course With Tutoring Software Improve Student Learning?.Santa Monica, CA: RAND Corporation.

Working paper

Pane, J.F., Griffin, B.A., McCaffrey, D.F., and Karam, R. (2014). Addendum to Effectiveness of Cognitive Tutor Algebra I at Scale (WR-1050–DEIES). Santa Monica, CA: RAND Corporation Working Paper.