The WWC review of interventions for Middle School Math addresses student outcomes in the math achievement domain. The findings below present the authors’ estimates and WWC-calculated estimates of the size and the statistical significance of the effects of I CAN Learn® Pre-Algebra and Algebra on students.7
Kirby (2006) reported a positive and statistically significant effect of I CAN Learn® on the math exam from the Louisiana Educational Assessment Program (LEAP) test. The WWC confirmed this finding after correcting the statistical significance level for clustering. The effect size was large enough to be considered substantively important according to WWC criteria (an effect size at least 0.25).
Kerstyn (2001) reported positive but not statistically significant effects of I CAN Learn® on the Florida Comprehensive Assessment Test (FCAT) math exam for Algebra 1, Algebra 1 Honors, MJ-3 pre-algebra, and MJ-3 Advanced courses. The effect size for each of these samples was not large enough to be considered substantively important according to WWC criteria (an effect size at least 0.25).
Kerstyn (2002) reported a positive and statistically significant effect of I CAN Learn® on the FCAT math exam for students in MJ-3 pre-algebra classes. The WWC confirmed this finding. The author also reported negative but not statistically significant effects for students in the Algebra 1, Algebra 1 Honors, and MJ-3 Advanced courses. The WWC confirmed that these negative effects were neither statistically significant nor large enough to be considered substantively important by WWC criteria (an effect size of at least 0.25). Thus, I CAN Learn® showed a statistically significant positive effect for MJ-3 pre-algebra students and indeterminate effects for Algebra 1, Algebra 1 Honors, and MJ-3 Advanced students.8
Kirby (2004a) reported a positive and statistically significant effect of I CAN Learn® on the General Mathematics exam from the California Standards Test. The statistical significance of this effect was confirmed by WWC analysis. The effect size was large enough to be considered substantively important according to WWC criteria (an effect size of at least 0.25).
Kirby (2004b) reported a positive and statistically significant effect of I CAN Learn® on the Math exam from the Georgia Criterion-Referenced Competency Test. The statistical significance of this effect was confirmed by WWC analysis. The effect size was large enough to be considered substantively important according to WWC criteria (an effect size of at least 0.25).
In sum, in the math achievement domain, the WWC reviewed findings from 11 samples reported in five studies.9 Four of these samples showed statistically significant positive effects, and the remaining seven samples showed indeterminate effects. One of the samples was examined in a study that used a strong design.
The WWC rates the effects of an intervention in a given outcome domain as positive, potentially positive, mixed, no discernible effects, potentially negative, or negative. The rating of effectiveness takes into account four factors: the quality of the research design, the statistical significance of the findings, the size of the difference between participants in the intervention and comparison conditions, and the consistency in findings across studies (see the WWC Intervention Rating Scheme).
|Institute of Education Sciences
