Making the Transition: Interim Results of the National Guard Youth ChalleNGe Evaluation
Millenky, Megan; Bloom, Dan; Dillon, Colleen (2010). MDRC. Retrieved from: https://eric.ed.gov/?id=ED514652
-
examining1,196Students, grades11-12
National Guard Youth ChalleNGe Program Intervention Report - Dropout Prevention
Review Details
Reviewed: September 2010
- Randomized Controlled Trial
- Meets WWC standards without reservations
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Please see the WWC summary of evidence for National Guard Youth ChalleNGe Program.
Findings
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Earned a diploma or GED certificate |
National Guard Youth ChalleNGe Program vs. Business as usual |
Posttest |
survey respondents;
|
0.61 |
0.36 |
Yes |
|
|
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Sample Characteristics
Characteristics of study sample as reported by study author.
-
Female: 16%
Male: 84% -
- B
- A
- C
- D
- E
- F
- G
- I
- H
- J
- K
- L
- P
- M
- N
- O
- Q
- R
- S
- V
- U
- T
- W
- X
- Z
- Y
- a
- h
- i
- b
- d
- e
- f
- c
- g
- j
- k
- l
- m
- n
- o
- p
- q
- r
- s
- t
- u
- v
- x
- w
- y
California, Florida, Georgia, Illinois, Michigan, Mississippi, North Carolina, New Mexico, Texas, Wisconsin
-
Race Black 40% Other or unknown 5% White 41% -
Ethnicity Hispanic 14% Not Hispanic or Latino 86%
Study Details
Setting
The study took place in 10 sites in California, Florida, Georgia, Illinois, Michigan, Mississippi, New Mexico, North Carolina, Texas, and Wisconsin.
Study sample
Between 2005 and 2007, a total of 3,074 high school dropouts between the ages of 16 and 18 were randomly assigned to the National Guard Youth ChalleNGe Program in 10 states; 2,320 to the intervention group and 754 to the control group. There was a single site in each state. A random subsample of 1,508 participants (916 in the intervention group and 592 in the control group) was selected to be given the 21-month follow-up survey. The analysis sample included 1,196 youths (736 in the intervention group and 460 in the control group) who responded to the follow-up survey; 80% of the intervention group and 78% of the control group responded to the survey.Most sample members (84%) were male. They ranged in age from 16 to 18 years old at program entry. The sample was racially and ethnically diverse: 41% were Caucasian; 40% were African-American; 14% were Hispanic; and 5% were from other racial and ethnic groups. Sample members had performed poorly in school before entering the program. About half reported receiving mostly Ds and Fs before dropping out.
Intervention Group
The National Guard Youth ChalleNGe Program attempts to promote positive youth development for high school dropouts through a residential education and training program followed by a structured mentoring program. The intervention consisted of three phases.Phase 1: Two weeks of program orientation and physical and psychological assessment in a residential, quasi-military setting. Phase 2: Twenty weeks of education and training in a residential, quasi-military setting. During this phase, the majority of participants’ time was spent on educational activi-ties, including work toward a GED or high school diploma. Education and training covered the following eight areas: leadership, responsible citizenship, service to community, life-coping skills, physical fitness, health and hygiene, job skills, and academic excellence. Phase 3: One year postresidential phase. Participants worked with program staff to arrange a postresidential placement in employment, continued education, or military service. In addition to participating in the placement activity, each participant was supposed to maintain monthly contact with a mentor, who was chosen by the participant and trained by ChalleNGe program staff.
Comparison Group
Control group members were not eligible to receive National Guard Youth ChalleNGe Program services but could receive other services available in the community. Contrary to the study design, eight control group members enrolled in the program (1% of the full control group). These enrollees were still assigned to the control group in all analyses.
Outcome descriptions
The relevant study outcome included in this review is whether students earned a high school diploma or GED, based on student follow-up interviews. For a more detailed description of this outcome measure, see Appendix A2. The study also examined a number of other outcomes that are not within the scope of the Dropout Prevention protocol.
Support for implementation
Program staff included team leaders who directly supervised the students, teachers who provided classroom instruction, and counselors who provided individual, group, and career counseling. It was common for staff members, particularly team leaders, to be National Guard members or to have military experience. Teachers often came to the program from the local school district or from community colleges. Counselors typically had bachelor’s or advanced degrees in psychology, social work, or other relevant fields.
Additional Sources
In the case of multiple manuscripts that report on one study, the WWC selects one manuscript as the primary citation and lists other manuscripts that describe the study as additional sources.
-
Bloom, Dan; Gardenhire-Crooks, Alissa; Mandsager, Conrad. (2009). Reengaging High School Dropouts: Early Results of the National Guard Youth ChalleNGe Program Evaluation. Full Report. MDRC.
An indicator of the effect of the intervention, the improvement index can be interpreted as the expected change in percentile rank for an average comparison group student if that student had received the intervention.
For more, please see the WWC Glossary entry for improvement index.
An outcome is the knowledge, skills, and attitudes that are attained as a result of an activity. An outcome measures is an instrument, device, or method that provides data on the outcome.
A finding that is included in the effectiveness rating. Excluded findings may include subgroups and subscales.
The sample on which the analysis was conducted.
The group to which the intervention group is compared, which may include a different intervention, business as usual, or no services.
The timing of the post-intervention outcome measure.
The number of students included in the analysis.
The mean score of students in the intervention group.
The mean score of students in the comparison group.
The WWC considers a finding to be statistically significant if the likelihood that the finding is due to chance alone, rather than a real difference, is less than five percent.
The WWC reviews studies for WWC products, Department of Education grant competitions, and IES performance measures.
The name and version of the document used to guide the review of the study.
The version of the WWC design standards used to guide the review of the study.
The result of the WWC assessment of the study. The rating is based on the strength of evidence of the effectiveness of the intervention. Studies are given a rating of Meets WWC Design Standards without Reservations, Meets WWC Design Standards with Reservations, or >Does Not Meet WWC Design Standards.
A related publication that was reviewed alongside the main study of interest.
Study findings for this report.
Based on the direction, magnitude, and statistical significance of the findings within a domain, the WWC characterizes the findings from a study as one of the following: statistically significant positive effects, substantively important positive effects, indeterminate effects, substantively important negative effects, and statistically significant negative effects. For more, please see the WWC Handbook.
The WWC may review studies for multiple purposes, including different reports and re-reviews using updated standards. Each WWC review of this study is listed in the dropdown. Details on any review may be accessed by making a selection from the drop down list.
Tier 1 Strong indicates strong evidence of effectiveness,
Tier 2 Moderate indicates moderate evidence of effectiveness, and
Tier 3 Promising indicates promising evidence of effectiveness,
as defined in the
non-regulatory guidance for ESSA
and the regulations for ED discretionary grants (EDGAR Part 77).