¹ The original sample included 14 schools. However, random assignment occurred in only 13 schools. For the purposes of this review, the study design and findings are reported for the subsample of 13 schools. Further, the study author conducted analyses of subsamples of students in addition to the total sample of 2,400 students in 13 schools. Those analyses focused on students of teachers who taught both conditions and students with matched pretest and posttest scores.
² The study notes that these data were collected in the Orleans Parish Public Schools before Hurricane Katrina. The demographics in the school district have changed since the study was conducted.

¹ The reported study sample was taken from Table 5 (page 9 of the study), while the reported demographic information was taken from Table 4 (page 8 of the study).
² Originally the study included 59 matched pairs of I CAN Learn^® and traditional classes, for a total of 118 classes; however, one of the pairs (MJ-3 #12) was dropped due to the traditional classroom transitioning to be an I CAN Learn^® classroom in the second semester.
³ A Semester 1 exam was also administered to study participants at the end of the first semester. The results are not reported in Appendix A3 with the results of the FCAT, because the psychometric properties of the Semester 1 exam were not reported, and it measured the effect of the intervention at an earlier point than did the FCAT. Because the FCAT measured math achievement at the end of the school year, the WWC reasoned that the FCAT was a more appropriate and valid measure of the intervention’s end-of-year effects.

¹ The study authors provided the WWC with the number of teachers in each condition.

ns = not statistically significant
na = not applicable

¹ This appendix reports findings considered for the effectiveness rating and the average improvement indices for the math achievement domain. Subgroup findings from the same studies are not included in these ratings, but are reported in Appendix A4.
² The standard deviation across all students in each group shows how dispersed the participants' outcomes are: a smaller standard deviation on a given measure would indicate that participants had more similar outcomes.
³ Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the comparison group.
⁴ For an explanation of the effect size calculation, see Technical Details of WWC-Conducted Computations.
⁵ Statistical significance is the probability that the difference between groups is a result of chance rather than a real difference between the groups.
⁶ The improvement index represents the difference between the percentile rank of the average student in the intervention condition and that of the average student in the comparison condition. The improvement index can take on values between –50 and +50, with positive numbers denoting results favorable to the intervention group.
⁷ The level of statistical significance was reported by the study authors or, where necessary, calculated by the WWC to correct for clustering within classrooms or schools and for multiple comparisons. For an explanation about the clustering correction, see the WWC Tutorial on Mismatch. For the formulas the WWC used to calculate statistical significance, see Technical Details of WWC-Conducted Computations. In the cases of Kerstyn (2001) and Kirby (2006), corrections for clustering were needed, so the significance levels may differ from those reported in the original studies. No other studies required corrections for clustering or multiple comparisons.
⁸ The WWC-computed average effect sizes for each study and for the domain across studies are simple averages rounded to two decimal places. The average improvement indices are calculated from the average effect size.
⁹ For Kerstyn (2001), the author reported both classroom-level and student-level posttest means and standard deviations. In order to compute a student-level effect size that could be considered for rating purposes, the WWC used the student-level data and corrected for clustering. For further details, please see Technical Details of WWC-Conducted Computations.
¹⁰ The intervention group value from Kerstyn (2002) is the intercept from the hierarchical linear modeling (HLM) model plus the program coefficient from the HLM analysis.
¹¹ The control group mean from Kerstyn (2002) is the intercept from the HLM model.
¹² The author reported results from the Georgia Criterion-Referenced Competency Test as scale scores and as criterion scores (that is, the percentage that passed the criterion score compared with the percentage that failed), but the WWC focused on the results from the scale scores because they contain more information than categorical scores.

ns = not statistically significant

¹ This appendix presents subgroup findings for measures that fall in math achievement. The standard curriculum and FCAT-exempt subgroup scores from Kerstyn (2002) are presented in Appendix A4, and these combined scores can be found in Appendix A3. Total group scores were used for rating purposes and are presented in Appendix A3.
² The standard deviation across all students in each group shows how dispersed the participants’ outcomes are: a smaller standard deviation on a given measure would indicate that participants had more similar outcomes.
³ Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the comparison group.
⁴ For an explanation of the effect size calculation, see Technical Details of WWC-Conducted Computations.
⁵ Statistical significance is the probability that the difference between groups is a result of chance rather than a real difference between the groups.
⁶ The improvement index represents the difference between the percentile rank of the average student in the intervention condition and that of the average student in the comparison condition. The improvement index can take on values between -50 and +50, with positive numbers denoting results favorable to the intervention group.
⁷ Standard curriculum and FCAT-exempt groups are categorized by the state in terms of their impact on school accountability grades. The standard curriculum students are used in the school accountability grade, while the FCAT-exempt students (including ESE, LEP, and Homebound students) are not.
⁸ The level of statistical significance was reported by the study authors or, where necessary, calculated by the WWC to correct for clustering within classrooms or schools (corrections for multiple comparisons were not done for findings not included in the overall intervention rating). For an explanation about the clustering correction, see the WWC Tutorial on Mismatch. For the formulas the WWC used to calculate statistical significance, see Technical Details of WWC-Conducted Computations. In the case of Kerstyn (2002), no correction for clustering was needed.
⁹ The intervention group value from Kerstyn (2002) is the intercept from the HLM model plus the program coefficient from the HLM analysis.
¹⁰ The control group mean from Kerstyn (2002) is the intercept from the HLM model.

¹ For rating purposes, the WWC considers the statistical significance of individual outcomes and the domain-level effect. The WWC also considers the size of the domain-level effect for ratings of potentially positive or potentially negative effects. For a complete description, see the WWC Intervention Rating Scheme.

¹ The samples in Kerstyn (2001) and Kerstyn (2002) comprised distinct samples of students within the same set of 36 schools.
² A rating of “medium to large” requires at least two studies and two schools across studies in one domain, and a total sample size across studies of at least 350 students or 14 classrooms. Otherwise, the rating is “small.”