Setting
This study took place at 11 high schools in seven evaluation sites: Fort Worth, Texas; Cleveland, Ohio; Washington, DC; Houston, Texas; Memphis, Tennessee; Philadelphia,
Pennsylvania; and Yakima, Washington. Three sites implemented QOP in multiple high schools: Washington, DC (2 high schools), Houston (2 high schools), and Memphis
(3 high schools). All other sites implemented QOP in a single high school.
Study sample
The Quantum Opportunity Program Demonstration Project used a randomized controlled trial research design. The demonstration operated in seven sites and served a single
cohort of entering ninth graders over a five-year period. In six of the seven sites, the programs served ninth graders who entered high school in the fall of 1995. In one site
(Washington, DC) the program served ninth graders who entered high school in the fall of 1996.
QOP served students from high schools with dropout rates of 40% or more. To be eligible for QOP, students in these high schools had to meet the following three criteria:
(1) they were entering ninth graders who were not repeating the ninth grade; (2) they had a grade point average below the 67th percentile of entering ninth graders at the
participating high school; and (3) they did not have severe physical and learning disabilities that would prevent them from participating in the program. A sample of students
meeting these criteria were drawn from lists of entering ninth graders; more than 97% of those identified agreed to participate in the study. The participating youth were then
randomly assigned to either an intervention group that was enrolled in QOP or a control group that was not. Across the seven locations, 580 students were assigned to QOP
group and 489 were assigned to the control group.
Researchers compared the baseline characteristics of QOP and control group students on gender, age, race/ethnicity, and grade point average and found no statistically
significant differences between the research groups. Participants were typically 13 or 14 years old, about two-thirds of participants were African-American, and a quarter
were Hispanic. Participants were evenly split between males and females.
Results summarized in this report are based on high school transcripts and three telephone surveys. One of these surveys was conducted at the end of the five-year
demonstration, another two years after the demonstration had ended, and a third four years after the demonstration’s end. There are two outcomes of interest for the WWC
review of the effectiveness of QOP: total credits earned five years after program entry and high school diploma or GED certificate receipt within nine years of program entry.
Total credits earned are based on transcript data and are available for 86% of QOP students and 77% of control group students. High school completion information is based
on data from all three survey waves, as well as transcript data, and is available for 88% of QOP students and 83% of control group students.2 For credits earned, the rate
of differential attrition exceeded the 5% threshold used for WWC dropout prevention reviews. For high school completion, the rate of differential attrition was equal to this
threshold. Because one measure used to rate QOP’s effectiveness exceeded the differential attrition standard, the WWC rated this study as meeting evidence standards with
reservations. To account for nonresponse, the study authors calculated impacts using weights that adjust for differences between respondents and nonrespondents in baseline
characteristics. However, the WWC did not consider this statistical adjustment sufficient for overcoming the differential attrition.
Intervention Group
QOP Demonstration Project was an intensive, five-year, case management and mentoring program for high school youth that emphasized after-school supplemental education,
developmental activities, and community service. Its primary goals were to increase the likelihood that enrollees would complete high school and enter a postsecondary education
or training program. The program also aimed to reduce risky behaviors such as substance abuse, crime, and teenage parenting.
QOP was operated by community-based organizations in seven sites. It offered a cohort of entering ninth graders services for up to five years and continued to provide
services even if participants dropped out of school or moved out of the school district. The comprehensive program had four primary components: case management and
mentoring, educational and developmental activities, supportive services, and financial incentives. These components are described in more detail below.
1. Case management and mentoring: Case managers typically had caseloads of 15 to 25 participants. Most case managers had office space within the school and were
available to enrollees during the school day, at night, on weekends, and during school vacations. Case managers served as mentors to the enrollees and were typically
assigned to the same enrollee for the full five years of the demonstration. According to an implementation study of QOP, most sites successfully implemented this
component of the model (Maxfield et al., 2003).
2. Education services, community service activities, and developmental activities: QOP’s participation target was 750 hours per year per enrollee. One-third of that time was
to be spent on educational activities (such as tutoring or computer-assisted learning), one-third on community service (such as visiting residents of a local nursing home),
and one-third on developmental activities (such as life skills and employment-readiness training). In most sites, based on program staff’s assessment of participant need,
resources were reallocated from community service activities toward case management and education activities. According to an implementation study of QOP, many
sites did not fully implement QOP’s education component, such as sustained course-based tutoring and computer-assisted instruction in basic reading and math skills
(Maxfield et al. 2003). In addition, in many cases developmental activities designed to teach life skills were primarily recreational activities. Overall, enrollees spent an
average of 174 hours per year on education, community service, and developmental activities—23% of the annual goal of 750 hours. Enrollees who spent little time
participating in QOP activities frequently reported that they thought the program was too much like school or that they had a barrier to participation, such as a job or a
child care or transportation problem (Maxfield et al., 2003).
3. Supportive services: QOP provided transportation assistance to facilitate attendance at program activities, as well as referrals to other resources in the community (such
as mental health services and summer jobs programs). According to an implementation study of QOP, most sites successfully provided transportation services; however,
most did not provide adequate child care support and did not consistently offer health screenings and referrals (Maxfield et al., 2003).
4. Financial Incentives: QOP provided enrollees with a stipend of approximately $1.25 for every hour devoted to program activities other than mentoring or recreation. A
matching amount was deposited in an accrual account, to be used by enrollees after they completed high school and enrolled in college, vocational training, or the
military. Enrollees could also earn bonuses for achieving major milestones, such as a grade point average above a set benchmark. According to an implementation study
of QOP, sites generally implemented this program component successfully (Maxfield et al., 2003).
Comparison Group
Control group members were not eligible to participate in QOP but could participate in other services available in the community. Based on responses to follow-up surveys,
16% of control group members participated in a program for disadvantaged youth other than QOP (Schirm et al., 2003). According to study authors, these other programs
were generally substantially less intensive than QOP.
Outcome descriptions
Two relevant outcomes from the QOP study are included in this summary and were used for rating purposes: the number of credits earned five years after program entry and
high school diploma or GED certificate receipt within nine years of program entry. (See Appendices A2.1 and A2.2 for a more detailed description of outcome measures.) The study also examined the program’s effects on academic outcomes, postsecondary outcomes, risky behaviors, resiliency factors, and attitudes. However, these outcomes do not fall within the three domains examined by the WWC’s review of dropout prevention interventions (staying in school, progressing in school, and completing school). Therefore, these additional outcomes are not included in this report.
Support for implementation
QOP staff attended annual training conferences provided by Opportunities Industrialization Centers of America (OICA) during the demonstration period. The initial training lasted seven days with annual four-day sessions in subsequent years. The Ford Foundation funded technical assistance for all seven QOP demonstration sites to be delivered by OICA. Technical assistance activities included helping sites set up and maintain QOP management information systems, conducting site visits, helping resolve case management issues, and providing sites with developmental curriculum
material and computer-assisted instruction (CAI) CD-ROMs. In addition, the U.S. Department of Labor provided technical assistance on selecting computer-assisted instruction (CAI) software, guidelines for setting up and operating accrual accounts, and quarterly calls with each site to discuss service delivery strategies.
Resources
Newsflash
Contact Us
Facebook
Twitter
YouTube
Helping Students Navigate the Path to College: What High Schools Can Do Practice Guide