Striving for Student Success. The Effect of Project GRAD on High School Student Outcomes in Three Urban School Districts
Snipes, Jason C.; Holton, Glee Ivory; Doolittle, Fred; Sztejnberg, Laura (2006). MDRC. Retrieved from: https://eric.ed.gov/?id=ED493362
-
examining13Students, grades9-12
Project GRAD Intervention Report - Dropout Prevention
Review Details
Reviewed: July 2007
- Quasi-Experimental Design
- Meets WWC standards with reservations
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Please see the WWC summary of evidence for Project GRAD.
Findings
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Ever graduated, looking ahead at least 3 years |
Project GRAD vs. Business as usual |
Posttest |
Cohorts 1-3;
|
32.00 |
34.60 |
No |
-- |
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
9th grade promotion (%) |
Project GRAD vs. Business as usual |
Posttest |
Cohorts 1-4;
|
44.90 |
46.90 |
No |
-- | |
|
9th grade credits |
Project GRAD vs. Business as usual |
Posttest |
Cohorts 1-4;
|
2.40 |
2.70 |
No |
-- |
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Sample Characteristics
Characteristics of study sample as reported by study author.
-
54% English language learners -
Female: 50%
Male: 50% -
Urban
-
- B
- A
- C
- D
- E
- F
- G
- I
- H
- J
- K
- L
- P
- M
- N
- O
- Q
- R
- S
- V
- U
- T
- W
- X
- Z
- Y
- a
- h
- i
- b
- d
- e
- f
- c
- g
- j
- k
- l
- m
- n
- o
- p
- q
- r
- s
- t
- u
- v
- x
- w
- y
Texas
-
Race Black 49% White 7% -
Ethnicity Hispanic 42% Not Hispanic or Latino 58%
Study Details
Setting
The initiative originated at Jefferson Davis High School in Houston, Texas, with the implementation of the model’s components in the 1994/95 school year. Project GRAD was implemented next at Jack Yates High School in the 1996/97 school year, and at Phillis Wheatley High School in the 1997/98 school year. The baseline period for Davis was the two years prior to 1994/95, while the baseline for the other schools was the three years prior to implementation. The analysis focuses on outcomes at these high schools through the 2003/04 school year.
Study sample
The main analysis sample included a series of cohorts of entering ninth grade students from three high schools implementing Project GRAD between 1998 and 2004 and ten matched comparison high schools. The sample consists of students for whom administrative records exist over the time period of the study.1 A group of comparison schools was matched to each Project GRAD school based on performance on standardized achievement tests and demographic composition. The result was a sample of three Project GRAD and ten comparison high schools. The study followed cohorts of students. Cohort 1 included students in the intervention and matched comparison schools who enrolled in the ninth grade during the first year of Project GRAD implementation at the intervention schools. Similarly, Cohort 2 included students in the intervention and comparison schools who were enrolled in the ninth grade during the second year of implementation, Cohort 3 included students who enrolled during the third year, and so on. Given the fixed period for data collection, later cohorts had shorter follow-up periods. To ensure both an adequate follow-up and an adequate sample size for measuring impacts, the WWC used results based on either Cohorts 1 through 4 (for most outcomes) or Cohorts 1 through 3 (for ever graduated, looking ahead at least three years—the number of cohorts was limited by the definition of the outcome measure) to rate the effectiveness of Project GRAD. Results for later cohorts that were followed over a shorter follow-up period are reported in Appendix A4. On average, the three Project GRAD and ten comparison high schools served students who had similar test scores, similar attendance patterns, and similar rates of promotion. There were some differences between the schools, however. Project GRAD schools were smaller than comparison schools (1,333 versus 2,158 students on average). In addition, Project GRAD schools served a larger share of African-American students than comparison schools did (56% versus 44%) and a smaller share of white students (1% versus 10%). The proportion of students who were Hispanic was similar in Project GRAD and comparison schools.
Intervention Group
Project GRAD targets a high school and the middle and elementary schools that feed into it. It combines a number of reforms with a goal of increasing reading and math achievement test scores, improving classroom behavior, providing a safety net for students to help reduce dropout rates, and increasing rates of high school graduation and college enrollment. At the high school level, Project GRAD has two main components: 1. Project GRAD college scholarships are provided to students who have a cumulative grade point average of at least 2.5, graduate within four years, complete a recommended college preparatory curriculum, and participate in two summer institutes. Scholarship amounts and criteria vary by site, averaging $1,000 to $1,500 a year. Each participating school has a scholarship coordinator who provides counseling, tutoring, and college admission preparation. 2. Summer institutes consist of four to six hours a day of instruction and related activities for four to six weeks in the summers. Parental and community improvement components seek to engage parents and the community in the schools and support students, along with social services and academic enrichment programs. Additionally, classroom management programs attempt to produce orderly classrooms focused on learning and promote positive relationships among students, teachers, and other adults. Project GRAD works with the entire feeder system of elementary and middle schools that send students to Project GRAD high schools to address early problems that can affect high school completion. To help students arrive at middle and high school better prepared academically, Project GRAD elementary schools provide professional development and coaches for teachers of reading and math and also implement curricula such as MOVE IT Math™, Everyday Math™, or Success For All™. To improve classroom behavior, Project GRAD schools implement Consistency Management & Cooperative Discipline®, an instructional discipline management system in which the teacher acts as an instructional leader and students have leadership roles. It is based on five elements: prevention of disruptive behavior through classroom management, a caring environment, cooperation, classroom organization, and parental and community involvement activities. Project GRAD also provides staff who deliver school-based social services—guidance, counseling, community outreach, and family case-management services—and facilitate parent involvement. Some sites link with Communities in Schools (CIS), a dropout prevention and social service agency, to provide social service and parent involvement staff members. In sites where there is no local CIS organization, Project GRAD has established a variation of the CIS component called Campus Family Support (CFS), which customizes traditional CIS services to meet the needs within the feeder system. In addition to student services, staff organize activities to enhance communication between teachers and parents.
Comparison Group
Matched comparison schools were Houston high schools that did not implement Project GRAD. Specifically, the analysis identified a set of comparison schools from the same district that were similar in terms of average performance on standardized achievement tests in the years immediately preceding program implementation and the percentages of students in key demographic groups.
Outcome descriptions
Outcomes in two of the domains are included in this study. Two measures related to progressing in school were included: credits earned in 9th grade and promotion from 9th to 10th grade. One measure in the completing school domain was included: ever graduated, looking ahead at least three years. All measures are from administrative records. The study also examined Project GRAD’s effects on attendance and standardized test scores. These outcomes do not fall within the three domains (staying in school, progressing in school, completing school) examined by the WWC’S review of dropout prevention interventions and are not included in this report.
Support for implementation
Teachers at Project GRAD high schools were regular teachers employed by the Houston Independent School District. Information on staff training was not available.
An indicator of the effect of the intervention, the improvement index can be interpreted as the expected change in percentile rank for an average comparison group student if that student had received the intervention.
For more, please see the WWC Glossary entry for improvement index.
An outcome is the knowledge, skills, and attitudes that are attained as a result of an activity. An outcome measures is an instrument, device, or method that provides data on the outcome.
A finding that is included in the effectiveness rating. Excluded findings may include subgroups and subscales.
The sample on which the analysis was conducted.
The group to which the intervention group is compared, which may include a different intervention, business as usual, or no services.
The timing of the post-intervention outcome measure.
The number of students included in the analysis.
The mean score of students in the intervention group.
The mean score of students in the comparison group.
The WWC considers a finding to be statistically significant if the likelihood that the finding is due to chance alone, rather than a real difference, is less than five percent.
The WWC reviews studies for WWC products, Department of Education grant competitions, and IES performance measures.
The name and version of the document used to guide the review of the study.
The version of the WWC design standards used to guide the review of the study.
The result of the WWC assessment of the study. The rating is based on the strength of evidence of the effectiveness of the intervention. Studies are given a rating of Meets WWC Design Standards without Reservations, Meets WWC Design Standards with Reservations, or >Does Not Meet WWC Design Standards.
A related publication that was reviewed alongside the main study of interest.
Study findings for this report.
Based on the direction, magnitude, and statistical significance of the findings within a domain, the WWC characterizes the findings from a study as one of the following: statistically significant positive effects, substantively important positive effects, indeterminate effects, substantively important negative effects, and statistically significant negative effects. For more, please see the WWC Handbook.
The WWC may review studies for multiple purposes, including different reports and re-reviews using updated standards. Each WWC review of this study is listed in the dropdown. Details on any review may be accessed by making a selection from the drop down list.
Tier 1 Strong indicates strong evidence of effectiveness,
Tier 2 Moderate indicates moderate evidence of effectiveness, and
Tier 3 Promising indicates promising evidence of effectiveness,
as defined in the
non-regulatory guidance for ESSA
and the regulations for ED discretionary grants (EDGAR Part 77).