Appendix A1 Study Characteristics: Cave, Bos, Doolittle, & Toussaint, 1993 (randomized controlled trial)
| Characteristic | Description |
|---|---|
| Study citation | Cave, G., Bos, H., Doolittle, F., & Toussaint, C. (1993). JOBSTART: Final report on a program for school dropouts. New York, NY: MDRC. |
| Participants | JOBSTART served youth who were: (1) 17 to 21 years old, (2) lacking a high school diploma or GED certificate, (3) reading below an 8th-grade level, and (4) economically disadvantaged. To meet enrollment targets, sites could waive the poor-reading-skills criterion for 20% of their enrollees. Applicants were considered economically disadvantaged if they: (1) received public assistance, (2) had a family income at or below the poverty line, or (3) were homeless. From 1985 to 1987, JOBSTART participants were recruited and enrolled by 13 sites in nine states. All participants were high school dropouts, and half had not worked at all in the year prior to enrolling in the program. Most JOBSTART recruits were non-white, 44% African-American, and 44% Hispanic. Just over half enrollees (54%) were women, and about half the women were young mothers. At baseline, 27% of sample members were receiving Aid to Families with Dependent Children or general assistance, while 38% were receiving food stamps. In the JOBSTART demonstration, 2,312 youth were randomly assigned to either JOBSTART or a control condition. The analyses in this report are based on data collected in the 48-month follow-up survey, to which 85% of JOBSTART youth and 83% of control group youth responded. Among the 1,941 respondents, the study authors compared the baseline characteristics of JOBSTART and control group youth, including their gender, ethnicity, family structure, employment history, and whether they received public assistance. They found no statistically significant differences between the research groups on these characteristics. |
| Setting | JOBSTART was evaluated in 13 study sites in nine states: Arizona (Phoenix), California (Los Angeles, Monterey Park, San Jose), Colorado (Denver), Connecticut (Hartford), Illinois (Chicago), Georgia (Atlanta), New York (Buffalo, New York City), Pennsylvania (Pittsburgh), and Texas (Corpus Christi, Dallas). The sites differed in their organization and structure: four of the sites were adult schools (three vocational, one community college), six were community-based organizations, and three were nonresidential Job Corps programs. |
| Intervention | JOBSTART offered a combination of education and occupational preparation services to improve the employment prospects of high school dropouts. Participation was voluntary. JOBSTART was modeled after Job Corps, which according to the study authors, was one of the few programs that had documented success in improving the outcomes of high school dropouts. But Job Corps was relatively expensive and required a residential commitment (a considerable barrier to participation for many youth). JOBSTART offered many of the same components as Job Corps, but was nonresidential. In addition, in an effort to reduce the program cost, JOBSTART did not include some of Job Corps’ most intensive services, such as extensive support services, paid work experience, and financial compensation. JOBSTART emphasized education and occupational preparation. It had four key components: (1) instruction in basic academic skills, (2) occupational skills training, (3) support services to facilitate participation, and (4) job placement assistance. Although all sites were required to offer the four components, implementation of the components varied. Some sites, for instance, offered all components in-house, while others linked participants with outside agencies that provided these services. The four key components of JOBSTART are described in more detail below:
|
| Comparison | Control group youth were not eligible to participate in JOBSTART but could voluntarily participate in other services available in the community. Based on survey responses, many control group youth participated in education and training programs during the follow-up period—but their rate of participation was substantially lower than it was among JOBSTART youth (Cave et al., 1993). During the four years after random assignment, 56% of control group youth reported receiving remedial or occupational instruction, compared with 94% of JOBSTART youth. Over this period, the average amount of participation in education and training was 432 hours for control group youth, compared with 800 hours for the JOBSTART youth (including their participation in JOBSTART as well as other education and training activities). |
| Primary outcomes and measurement | One relevant outcome from the JOBSTART study is included in this summary and used for rating purposes: receiving a high school diploma or GED certificate within 48 months of random assignment. For a more detailed description of the outcome measure, see Appendix A2. The study also examined the program’s effects on employment, hours worked, total earnings, welfare receipt, pregnancy, criminal activity, and drug use. These outcomes, however, do not fall within the three domains examined by the WWC’s review of dropout prevention interventions (staying in school, progressing in school, and completing school). So, they are not included in this report. |
| Staff training | Many JOBSTART staff were former teachers from public schools or community colleges. In many instances, these teachers had experience working with disadvantaged youth or adults. In some instances, they had taught GED preparation classes or remedial education prior to working for JOBSTART. Other JOBSTART staff had previously worked in other employment programs for disadvantaged youth (Auspos, Cave, Doolittle, & Hoerz, 1989). |
Appendix A2 Outcome measures for the completing school domain
| Outcome measure | Description |
|---|---|
| Earned a high school diploma or GED certificate within 48 months of random assignment | This binary measure represents the percentage of students who either received a high school diploma or GED certificate within 48 months of random assignment. This measure was based on the sample member’s response to the 48-month survey. |
Appendix A3 Summary of study findings included in the rating for the completing school domain1
| Author's findings from the study | ||||||||
|---|---|---|---|---|---|---|---|---|
| Mean outcome | WWC calculations | |||||||
| Outcome measure | Study sample | Sample size (students) | JOBSTART group | Comparison group | Mean difference2 (JOBSTART – comparison) | Effect size3 | Statistical
significance4 (at α= 0.05) |
Improvement index5 |
| Cave, Bos, Doolittle, & Toussaint, 1993 (randomized controlled trial)6 | ||||||||
| Earned a high school diploma or GED certificate within 48 months of random assignment (%) | Full sample | 1,941 | 42.0 | 28.6 | 13.4 | 0.36 | Statistically significant | +14 |
| Domain average for completing school7 | 0.36 | Statistically significant | +14 | |||||
|
1 This appendix reports findings considered for the effectiveness rating and the average improvement index for the completing school domain. Subgroup findings are not included in these ratings but are in Appendix A4.1. Appendix A4.2 reports the separate effects of JOBSTART on earning a GED certificate or high school diploma, which were not used in JOBSTART’s effectiveness rating. |
||||||||
Appendix A4.1 Summary of subgroup findings by gender and parenthood status for the completing school domain1
| Author's findings from the study | ||||||||
|---|---|---|---|---|---|---|---|---|
| Mean outcome | WWC calculations | |||||||
| Outcome measure | Study sample | Sample size (students) | JOBSTART group | Comparison group | Mean difference2 (JOBSTART – comparison) | Effect size3 | Statistical
significance4 (at α= 0.05) |
Improvement index5 |
| Cave, Bos, Doolittle, & Toussaint, 1993 (randomized controlled trial)6 | ||||||||
| Earned a high school diploma or GED certificate within 48 months of random assignment (%) | Men | 900 | 42.0 | 28.3 | 13.7 | 0.37 | Statistically significant | +14 |
| Earned a high school diploma or GED certificate within 48 months of random assignment (%) | Women living with own children at baseline | 508 | 42.0 | 26.7 | 15.3 | 0.42 | Statistically significant | +16 |
| Earned a high school diploma or GED certificate within 48 months of random assignment (%) | Women not living with own children at baseline | 533 | 41.6 | 31.3 | 10.4 | 0.27 | Statistically significant | +11 |
|
1 This appendix presents subgroup findings by gender and parenthood status for the intervention’s effects on receiving a high school diploma or GED certificate. The full sample was used for determining the effectiveness rating. These findings are in Appendix A3. |
||||||||
Appendix A4.2 Summary of additional findings for the completing school domain1
| Author's findings from the study | ||||||||
|---|---|---|---|---|---|---|---|---|
| Mean outcome | WWC calculations | |||||||
| Outcome measure | Study sample | Sample size (students) | JOBSTART group | Comparison group | Mean difference2 (JOBSTART – comparison) | Effect size3 | Statistical
significance4 (at α= 0.05) |
Improvement index5 |
| Cave, Bos, Doolittle, & Toussaint, 1993 (randomized controlled trial)6 | ||||||||
| Earned a GED certificate within 48 months of random assignment (%) | Full sample | 1,941 | 37.6 | 21.1 | 16.5 | 0.49 | Statistically significant | +19 |
| Earned a high school diploma or GED certificate within 48 months of random assignment (%) | Full sample | 1,941 | 4.4 | 7.5 | –3.1 | –0.34 | Statistically significant | –13 |
|
1 This appendix presents separate effects of JOBSTART on receiving a high school diploma or GED certificate. The intervention’s combined effect on receiving a high school diploma or GED certificate was used for determining the effectiveness rating and is in Appendix A3. |
||||||||
Appendix A5 JOBSTART rating for the completing school domain
The WWC rates an intervention’s effects in a given outcome domain as positive, potentially positive, mixed, no discernible effects, potentially negative, or negative.1
For the outcome domain of completing school, the WWC rated JOBSTART as having potentially positive effects. It did not meet the criteria for positive effects because only one study showed a statistically significant or substantively important effect in this domain. The remaining ratings (mixed effects, no discernable effects, potentially negative effects, negative effects) were not considered because JOBSTART was assigned the highest applicable rating.
| Rating received |
|---|
|
Potentially positive effects: Evidence of a positive effect with no overriding contrary evidence.
AND
|
| Other ratings considered |
|
Positive effects: Strong evidence of a positive effect with no overriding contrary evidence.
AND
|
1 For rating purposes, the WWC considers the statistical significance of individual outcomes and the domain-level effect. The WWC also considers the size of the domain-level effect for ratings of potentially positive or potentially negative effects. For a complete description, see the WWC Intervention Rating Scheme. |
Appendix A6 Extent of evidence by domain
| Sample size | ||||
|---|---|---|---|---|
| Outcome domain | Number of studies | Schools | Students | Extent of evidence1 |
| Staying in school | 0 | na | na | na |
| Progressing in school | 0 | na | na | na |
| Completing school | 1 | 13 | 1,941 | Small |
|
na = not applicable/not studied 1 A rating of "moderate to large" requires at least two studies and two schools across studies in one domain and a total sample size across studies of at least 350 students or 14 classrooms. Otherwise, the rating is "small." | ||||