WWC review of this study

Effectiveness of Reading and Mathematics Software Products: Findings From Two Student Cohorts. NCEE 2009-4041

Campuzano, Larissa; Dynarski, Mark; Agodini, Roberto; Rall, Kristina (2009). National Center for Education Evaluation and Regional Assistance. Retrieved from: https://eric.ed.gov/?id=ED504657

  • Randomized Controlled Trial
    , grades

Reviewed: September 2016

No statistically significant positive
Meets WWC standards without reservations
Algebra outcomes—Indeterminate effect found for the domain
Comparison Period Sample Intervention
Significant? Improvement

Educational Testing Service (ETS) End-of-Course Algebra Test

Cognitive Tutor® vs. Business as usual

1 Semester

Full sample;
29 classes




Show Supplemental Findings

Educational Testing Service (ETS) End-of-Course Algebra Test

Cognitive Tutor® vs. Business as usual

1 Semester

Year 2 schools only ;
18 classes





Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.

Characteristics of study sample as reported by study author.

  • Female: 49%
    Male: 51%

  • Urban


The study took place within 4 school districts and across 11 different schools. The school districts were considered urban or urban fringe areas. The average district had 230 schools with roughly 133,000 students.

Study sample

Students in the sample were students taking an algebra course. Of the students in the sample 14% were in grade 8 and 86% were in grade 9. Teachers in the classrooms averaged 13 years of teaching experience (intervention = 14.18; comparison = 11.25) with close to half of the teachers holding a master's degree (41%; 40% in intervention and 43% in comparison). The teachers were mostly female with 60% in the intervention and 57% female teachers in the comparison group. Students in these classrooms were an average age of 15. Forty-nine percent of the students were female (intervention = 51%; comparison = 46%) with 14% in grade 8 and 86% in grade 9. No other characteristics were reported.

Intervention Group

The Cognitive Tutor curriculum includes proportional reasoning, solving linear equations and inequalities, solving systems of linear equations, analyzing data, and using polynomial functions, powers, and exponents. Materials include a textbook and software. Students are presented with problems and asked to solve using graphs and solvers. Students' skill levels are evaluated based on student answers. Students use the product in a computer lab two days a week and use the textbook three days a week.

Comparison Group

Comparison students received business-as-usual.

Support for implementation

Teachers implementing the Cognitive Tutor received four days of training on using the product. Teachers could also receive support by phone. The estimated annual cost, per student, to implement the program was $69. Of that amount 49% was used for license fees and the remaining 57% was used for teacher training and support, technical support, and printed materials and supplies.


Your export should download shortly as a zip archive.

This download will include data files for study and findings review data and a data dictionary.

Connect With the WWC

back to top