Setting
The five study sites were Job Training and Partnership Act local employment and training agencies that operated both STEP and SYETP. These sites were located in Boston,
MA; Fresno, CA; Portland, OR; San Diego, CA; and Seattle, WA. Remedial education and life skills learning classes were typically held at a community school or college. Work
experience was conducted in the community near where the classes were held.
Study sample
The study used a randomized controlled trial research design to examine the effects of STEP in five sites located in four states. The sample included youth who had applied for
STEP in 1986 and 1987 in five research sites. To be eligible, youth had to be 14 or 15 years old and from low-income families and had to have tested below grade level.
The study included three cohorts of youth. The authors analyzed each of these cohorts separately. The Cohort 1 analysis did not meet WWC evidence standards because of
high attrition and a failure to establish baseline equivalence. This analysis is therefore not included in this report. The Cohort 2 and Cohort 3 analyses meet evidence standards.
Since these analyses were conducted in the same sites, the WWC treated these analyses as one study and combined the results when rating the effectiveness of STEP.
The Cohort 2 analysis included 1,635 eligible youth who were randomly assigned to either a treatment group, which was offered the opportunity of participating in the STEP
program during the summers of 1986 and 1987, or to a control group, which was offered a summer job in the federally funded Summer Youth Employment and Training
Program (SYETP). The study lost 382 students to attrition or survey nonresponse, leaving an analysis sample of 1,253 youth.
The Cohort 3 analysis included 1,591 eligible youth who were randomly assigned to either a treatment group, which was offered the opportunity of participating in the STEP
program during the summers of 1987 and 1988, or to a control group, which was offered a summer job in the federally funded SYETP. The study lost 256 students to attrition
or survey nonresponse, leaving an analysis sample of 1,335 youth.
Across these two cohorts, slightly more than half of the study sample was female. About half of the sample was African-American, one-fifth was Asian, and one-fifth was
Hispanic. All students came from low-income households, and about half of the sample came from female-headed households. About one-third of students reported having
repeated a grade, and average test scores indicated that sample students were performing substantially below their grade level in math.
Intervention Group
The STEP program focused on four areas: remediation, life skills and opportunities (LSO), work experience, and school-year support. During each of two summers, participants
were offered approximately 90 hours of remedial instruction in basic reading and math skills, 18 hours of LSO instruction, and at least 80 hours of work experience. The
remediation component provided a minimum of 90 hours of skill-based group and individually paced instruction. The LSO component stressed responsible social and sexual
attitudes and behaviors. In the work experience component, participants were usually assigned to jobs near their remediation site. These jobs were typically in maintenance,
recreation, clerical, and child care aide positions. During the intervening school year, program youth interacted with a designated counselor/advocate. These counselors helped
monitor school attendance and performance of students and referred them to social services as needed.
Comparison Group
The control group members were offered either a one-summer or two-summer job in the federally funded SYETP. The counterfactual condition varied across sites. In general,
members of the control group were provided a part-time job during the first summer for which they were eligible for the program. In two sites, San Diego and Seattle, control
group youth also were provided a job in the second summer. In general, control group youth spent more time working than did the treatment group youth, since treatment
group youth were receiving remedial education and life skills training in addition to work. Overall numbers of hours engaged in study-related activity appear to be roughly the
same for treatment and control group youth, with control group youth participating in employment for an average of 190 hours.
Outcome descriptions
The relevant study outcomes included in this review are whether students dropped out of school and their highest grade completed, based on student follow-up interviews. For
a more detailed description of these outcome measures, see Appendices A2.1 and A2.2.
Support for implementation
Each site hired a lead teacher who was primarily responsible for overseeing the remediation component, hiring teachers, planning and providing teacher training, and assembling a curriculum development team of reading and math teachers to develop curriculum modules for site use. The lead teacher also assumed responsibility for seeing that life skills staff were supported and successfully integrated into program operations. Remediation teachers also received in-service training over the course of the summer program, an average of 4 to 6 hours total. In addition, before this cohort’s second summer of participation, the evaluator’s consultants and staff delivered preservice training to all remediation teachers at participating sites. STEP teachers received an average of between 15 and 20 hours of preservice training (the hours varied by site). In addition
to specific topics outlined in the training guides, teacher training included an orientation to the program, discussions on the integration of the life skills and work experience components, and a review of modules and program logistics. For the life skills component, all life skills instructors attended a two-day comprehensive training workshop. This training included a review of curriculum content, an analysis of instructional techniques, a discussion of classroom management issues, and a series of role-plays and other sensitizing activities. They received minimal in-service training.