WWC review of this study

Effectiveness of Cognitive Tutor Algebra I at scale (high school experiment).

Pane, J. F., Griffin, B. A., McCaffrey, D. F., & Karam, R. (2014). Educational Evaluation and Policy Analysis, 36(2), 127–144. doi:10.3102/0162373713507480

  • Randomized Controlled Trial
     examining 
    5,738
     Students
    , grades
    8-12
No statistically significant positive
findings
Meets WWC standards with reservations

Reviewed: June 2016

Algebra outcomes—Statistically significant positive effects found
Outcome
measure
Comparison Period Sample Intervention
mean
Comparison
mean
Significant? Improvement
index

CTB/McGraw-Hill Algebra Proficiency Exam

Cognitive Tutor® Algebra I vs. Business as usual

2 Years

Grades 8–12;

0.08

-0.08

No

--
More Outcomes

CTB/McGraw-Hill Algebra Proficiency Exam

Cognitive Tutor® Algebra I vs. Business as usual

1 Year

Grades 8–12;

-0.02

0.03

No

--

Characteristics of study sample as reported by study author.


  • 10% English language learners

  • 54% Free or reduced price lunch
  • Race
    Black
    36%
    White
    46%

  • Rural, Suburban, Urban
    • B
    • A
    • C
    • D
    • E
    • F
    • G
    • I
    • H
    • J
    • K
    • L
    • P
    • M
    • N
    • O
    • Q
    • R
    • S
    • V
    • U
    • T
    • W
    • X
    • Z
    • Y
    • a
    • h
    • i
    • b
    • d
    • e
    • f
    • c
    • g
    • j
    • k
    • l
    • m
    • n
    • o
    • p
    • q
    • r
    • s
    • t
    • u
    • x
    • w
    • y

    Alabama, Connecticut, Kentucky, Louisiana, Michigan, New Jersey, Texas

Setting

The study took place in 51 school districts across seven states, including urban districts in Alabama, Connecticut, New Jersey, and Texas; suburban districts in Michigan; and rural districts in Kentucky and Louisiana.

Study sample

On average, White non-Hispanic students comprised 41 - 52% of the sample in each condition, while Black students, the next largest group, comprised 30-41%. Free- or reduced-price lunch status was 38 - 69% of the sample in each condition. Finally, English Language Learner (ELL) status was around 10% in each sample condition.

Intervention Group

Cognitive Tutor® Algebra I was implemented for 2 consecutive school years. The curriculum involved teacher-directed classroom instruction 3 days a week and computer-guided instruction 2 days a week. The software was available for students to use during class and other times during the day. It was self-paced, and students’ progress was based on mastery of the material. During the classroom lessons, students were exposed to topics such as solving linear equations, mathematical modeling with linear and quadratic expressions, problem solving using proportion reasoning, and analyzing data and making predictions. To apply these concepts, students completed worksheets and other activities and engaged in a variety of problem solving strategies. An implementation study indicated that teachers generally implemented all components of Cognitive Tutor® Algebra I, but sometimes emphasized the components differently from the publisher recommendations.

Comparison Group

Students in the comparison condition received each school’s existing Algebra I curriculum, which included curricula published by Glencoe, McDougal Littell, and Prentice Hall.

Support for implementation

Teachers received 4 days of training throughout the study. During a 3-day session prior to the school year, teachers were introduced to the curriculum, software, and tools, and provided instruction on connections between the curriculum and software and how to use the data to inform instruction. Teachers received a fourth day of training during the school year, at which time professional development staff observed classrooms, offered recommendations, and helped with any problems the teachers had. In addition, teachers received training materials, an implementation guide, and a book of resources and assessments.

In the case of multiple manuscripts that report on one study, the WWC selects one manuscript as the primary citation and lists other manuscripts that describe the study as additional sources.

  • Pane, J. F., Griffin, B. A., McCaffrey, D. F., & Karam, R. (2014). Addendum to effectiveness of Cognitive Tutor Algebra I at scale (Working Paper WR-1050-DEIES) (high school experiment). Santa Monica, CA: RAND Corporation.

  • Pane, J. F., Griffin, B. A., McCaffrey, D. F., Karam, R., Daugherty, L., & Phillips, A. (2013). Does an algebra course with tutoring software improve student learning? (high school experiment). Santa Monica, CA: RAND Corporation.

  • Sales, A. C., & Pane, J. F. (2015, June). Exploring causal mechanisms in a randomized effectiveness trial of the Cognitive Tutor (high school experiment). Paper presented at the 8th International Conference on Educational Data Mining, Madrid, Spain.

 

Your export should download shortly as a zip archive.

This download will include data files for study and findings review data and a data dictionary.

Connect With the WWC

newsflash icon contact icon facebook icon twitter icon
loading
back to top