WWC review of this study

Effectiveness of Cognitive Tutor Algebra I at Scale

Pane, John F.; Griffin, Beth Ann; McCaffrey, Daniel F.; Karam, Rita (2014). Educational Evaluation and Policy Analysis, v36 n2 p127-144. Retrieved from: https://eric.ed.gov/?id=EJ1024233

  • Randomized Controlled Trial
    , grades

Reviewed: June 2016

No statistically significant positive
Meets WWC standards with reservations
Algebra outcomes—Statistically significant positive effect found for the domain
Comparison Period Sample Intervention
Significant? Improvement

CTB/McGraw-Hill Algebra Proficiency Exam

Cognitive Tutor Algebra I vs. Business as usual

2 Years

Grades 8–12;





CTB/McGraw-Hill Algebra Proficiency Exam

Cognitive Tutor Algebra I vs. Business as usual

1 Year

Grades 8–12;





Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.

Characteristics of study sample as reported by study author.

  • 10% English language learners

  • Rural, Suburban, Urban
    • B
    • A
    • C
    • D
    • E
    • F
    • G
    • I
    • H
    • J
    • K
    • L
    • P
    • M
    • N
    • O
    • Q
    • R
    • S
    • V
    • U
    • T
    • W
    • X
    • Z
    • Y
    • a
    • h
    • i
    • b
    • d
    • e
    • f
    • c
    • g
    • j
    • k
    • l
    • m
    • n
    • o
    • p
    • q
    • r
    • s
    • t
    • u
    • x
    • w
    • y

    Alabama, Connecticut, Kentucky, Louisiana, Michigan, New Jersey, Texas
  • Race


The study took place in 51 school districts across seven states, including urban districts in Alabama, Connecticut, New Jersey, and Texas; suburban districts in Michigan; and rural districts in Kentucky and Louisiana.

Study sample

On average, White non-Hispanic students comprised 41 - 52% of the sample in each condition, while Black students, the next largest group, comprised 30-41%. Free- or reduced-price lunch status was 38 - 69% of the sample in each condition. Finally, English Language Learner (ELL) status was around 10% in each sample condition.

Intervention Group

Cognitive Tutor® Algebra I was implemented for 2 consecutive school years. The curriculum involved teacher-directed classroom instruction 3 days a week and computer-guided instruction 2 days a week. The software was available for students to use during class and other times during the day. It was self-paced, and students’ progress was based on mastery of the material. During the classroom lessons, students were exposed to topics such as solving linear equations, mathematical modeling with linear and quadratic expressions, problem solving using proportion reasoning, and analyzing data and making predictions. To apply these concepts, students completed worksheets and other activities and engaged in a variety of problem solving strategies. An implementation study indicated that teachers generally implemented all components of Cognitive Tutor® Algebra I, but sometimes emphasized the components differently from the publisher recommendations.

Comparison Group

Students in the comparison condition received each school’s existing Algebra I curriculum, which included curricula published by Glencoe, McDougal Littell, and Prentice Hall.

Support for implementation

Teachers received 4 days of training throughout the study. During a 3-day session prior to the school year, teachers were introduced to the curriculum, software, and tools, and provided instruction on connections between the curriculum and software and how to use the data to inform instruction. Teachers received a fourth day of training during the school year, at which time professional development staff observed classrooms, offered recommendations, and helped with any problems the teachers had. In addition, teachers received training materials, an implementation guide, and a book of resources and assessments.

In the case of multiple manuscripts that report on one study, the WWC selects one manuscript as the primary citation and lists other manuscripts that describe the study as additional sources.

  • Pane, John F.; Griffin, Beth Ann; McCaffrey, Daniel F.; Karam, Rita. (2014). Addendum to Effectiveness of Cognitive Tutor Algebra I at Scale. Working Paper WR-1050-DEIES. RAND Corporation.

  • Pane, J. F., Griffin, B. A., McCaffrey, D. F., Karam, R., Daugherty, L., & Phillips, A. (2013). Does an algebra course with tutoring software improve student learning? (high school experiment). Santa Monica, CA: RAND Corporation.

  • Sales, A. C., & Pane, J. F. (2015, June). Exploring causal mechanisms in a randomized effectiveness trial of the Cognitive Tutor (high school experiment). Paper presented at the 8th International Conference on Educational Data Mining, Madrid, Spain.


Your export should download shortly as a zip archive.

This download will include data files for study and findings review data and a data dictionary.

Connect With the WWC

back to top