⁷The Grossman and Sipe study (1992) examined three cohorts of STEP applicants. The authors conducted the analyses of each of these cohorts separately. The study’s Cohort 1 analysis did not meet evidence standards because of a high level of attrition for that cohort and because the authors did not demonstrate that their analysis sample was equivalent at baseline. The Cohort 2 and Cohort 3 analyses meet WWC evidence standards. The WWC combines Cohort 2 and Cohort 3 results when presenting program effects.
⁸ The extent of evidence categorization was developed to tell readers how much evidence was used to determine the intervention rating, focusing on the number and size of studies. Additional factors associated with a related concept—external validity, such as the students’ demographics and the types of settings in which studies took place—are not taken into account for the categorization. Information about how the extent of evidence rating was determined for STEP is in Appendix A6.
⁹ The Grossman and Sipe (1992) study examined outcomes in the completing school domain (high school graduation). However, these findings did not meet WWC evidence standards because of high attrition and because the authors did not provide evidence that their research groups were initially equivalent within this analysis sample. Therefore, this evidence could not be used to rate STEP’s effectiveness in the completing school domain. These results are reported in Appendix A4.