Impact Evaluation of the U.S. Department of Education's Student Mentoring Program. Final Report. NCEE 2009-4047
Bernstein, Lawrence; Rappaport, Catherine Dun; Olsho, Lauren; Hunt, Dana; Levin, Marjorie (2009). National Center for Education Evaluation and Regional Assistance. Retrieved from: https://eric.ed.gov/?id=ED504310
-
examining1,969Students, grades4-8
Grant Competition
Review Details
Reviewed: March 2017
- Grant Competition (findings for Student Mentoring Program)
- Randomized Controlled Trial
- Meets WWC standards without reservations because it is a randomized controlled trial with low attrition.
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Findings
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
% Proficient English Language Arts (ELA) |
Student Mentoring Program vs. Business as usual |
1 Year |
Full sample;
|
49.40 |
50.76 |
No |
-- | |
|
% Proficient Math |
Student Mentoring Program vs. Business as usual |
1 Year |
Full sample;
|
45.69 |
47.10 |
No |
-- |
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Misconduct (any infraction) |
Student Mentoring Program vs. Business as usual |
1 Year |
Full sample;
|
25.00 |
22.91 |
No |
-- | |
|
% Delinquent (Comitting repeated infractions) |
Student Mentoring Program vs. Business as usual |
1 Year |
Full sample;
|
8.64 |
9.13 |
No |
-- | |
|
% Delinquent (Comitting any infraction) |
Student Mentoring Program vs. Business as usual |
1 Year |
Full sample;
|
18.13 |
20.03 |
No |
-- | |
|
Misconduct (repeated infractions) |
Student Mentoring Program vs. Business as usual |
1 Year |
Full sample;
|
14.21 |
15.63 |
No |
-- |
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Truancy |
Student Mentoring Program vs. Business as usual |
1 Year |
Full sample;
|
2.04 |
2.47 |
Yes |
|
|
|
Absenteeism rate |
Student Mentoring Program vs. Business as usual |
1 Year |
Full sample;
|
5.03 |
5.49 |
Yes |
|
|
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Sample Characteristics
Characteristics of study sample as reported by study author.
-
Female: 53%
Male: 47% -
Race Black 41% White 22% -
Ethnicity Hispanic 31% Not Hispanic or Latino 69%
Study Details
Setting
This study was conducted in 32 eligible schools located in the United States. All schools received grant funding for the Student Mentoring Program being evaluated, were willing to participate in the randomized study, and had enough students to assign to each condition, though schools ranged from small to large. More detailed information about each school was not provided.
Study sample
Overall, the sample included 47% boys and 53% girls. The average age of the sample was 11.1 years old; 22 percent identified as White, 41% identified as African-American, and 31% identified as Hispanic. 86% qualified for free or reduced-price lunches and 56% came from two-parent households.
Intervention Group
The mentoring programs did not have to follow a certain curriculum, but program activities had to be designed to: "improve interpersonal relationships with peers, teachers, other adults and family members; increase personal responsibility and community involvement; discourage drug and alcohol use, use of weapons, and other delinquency involvement; reduce dropout rates; and improve academic achievement" (pg. xiv). The program included grantee and mentor activities. Grantee activities were to "identify students at risk; recruit, screen, and train mentors; match mentors and students in close (1:1 preferred) relationships; provide/identify space in the school or other setting for mentoring; and support and monitor relationships to ensure that they benefit students." (p4). Mentor activities were to "provide students with support and general guidance; serve as role models [with regard to demonstrating personal and social responsibility]; and provide students with academic assistance and encourage graduation from secondary school and planning for post-secondary education and training" (p4). The number of sessions, time per session, and duration of program activities varied.
Comparison Group
The comparison condition received business-as-usual and were free to enroll in any mentoring program they pleased.
Support for implementation
The legislation authorizing the Student Mentoring Program permits grantees to "support mentors though technical assistance and suggested programming" (xiv). As part of the Executive Summary, the study reports that the majority of mentors "received pre-match training or orientation and had access to ongoing supports from the program" (xviii). No information about support for implementation was provided.
Impact Evaluation of the U.S. Department of Education's Student Mentoring Program. Final Report. NCEE 2009-4047
Review Details
Reviewed: June 2009
- Quick Review (78 KB) (findings for U.S. Department of Education’s Student Mentoring Program (SMP))
- Randomized Controlled Trial
- Meets WWC standards without reservations
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Findings
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Proficiency on State Math Test |
U.S. Department of Education’s Student Mentoring Program (SMP) vs. Business as usual |
Follow-Up |
Main;
|
45.69 |
47.10 |
No |
-- | |
|
Proficiency on State Reading/English Language Arts (ELA) Test |
U.S. Department of Education’s Student Mentoring Program (SMP) vs. Business as usual |
Follow-Up |
Main;
|
49.40 |
50.76 |
No |
-- | |
|
Scholastic Efficiency and School Bonding (scale score) |
U.S. Department of Education’s Student Mentoring Program (SMP) vs. Business as usual |
Follow-Up |
Main;
|
3.06 |
3.03 |
No |
-- | |
|
Future Orientation (scale score) |
U.S. Department of Education’s Student Mentoring Program (SMP) vs. Business as usual |
Follow-Up |
Main;
|
3.85 |
3.80 |
No |
-- | |
|
Social Studies grade |
U.S. Department of Education’s Student Mentoring Program (SMP) vs. Business as usual |
Follow-Up |
Main;
|
3.53 |
3.56 |
No |
-- | |
|
Science grade |
U.S. Department of Education’s Student Mentoring Program (SMP) vs. Business as usual |
Follow-Up |
Main;
|
3.52 |
3.55 |
No |
-- | |
|
Mathematics grade |
U.S. Department of Education’s Student Mentoring Program (SMP) vs. Business as usual |
Follow-Up |
Main;
|
3.19 |
3.23 |
No |
-- | |
|
English/Language Arts Grade |
U.S. Department of Education’s Student Mentoring Program (SMP) vs. Business as usual |
Follow-Up |
Main;
|
3.57 |
3.61 |
No |
-- | |
|
Absenteeism Rate |
U.S. Department of Education’s Student Mentoring Program (SMP) vs. Business as usual |
Follow-Up |
Main;
|
5.03 |
5.49 |
No |
-- |
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Misconduct (any infraction) |
U.S. Department of Education’s Student Mentoring Program (SMP) vs. Business as usual |
Follow-Up |
Main;
|
25.00 |
22.91 |
No |
-- | |
|
Misconduct (repeated infractions) |
U.S. Department of Education’s Student Mentoring Program (SMP) vs. Business as usual |
Follow-Up |
Main;
|
14.21 |
15.63 |
No |
-- | |
|
Delinquency (any infraction) |
U.S. Department of Education’s Student Mentoring Program (SMP) vs. Business as usual |
Follow-Up |
Main;
|
18.13 |
20.03 |
No |
-- | |
|
Delinquency (repeated infraction) |
U.S. Department of Education’s Student Mentoring Program (SMP) vs. Business as usual |
Follow-Up |
Main;
|
8.64 |
9.13 |
No |
-- | |
|
Delinquency (scale score) |
U.S. Department of Education’s Student Mentoring Program (SMP) vs. Business as usual |
Follow-Up |
Main;
|
3.87 |
3.85 |
No |
-- | |
|
Pro-social behavior (scale score) |
U.S. Department of Education’s Student Mentoring Program (SMP) vs. Business as usual |
Follow-Up |
Main;
|
2.79 |
2.80 |
No |
-- | |
|
Misconduct (scale score) |
U.S. Department of Education’s Student Mentoring Program (SMP) vs. Business as usual |
Follow-Up |
Main;
|
3.20 |
3.20 |
No |
-- | |
|
Truency (unexcused absences) |
U.S. Department of Education’s Student Mentoring Program (SMP) vs. Business as usual |
Follow-Up |
Main;
|
2.04 |
2.47 |
No |
-- |
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Sample Characteristics
Characteristics of study sample as reported by study author.
-
Female: 51%
Male: 49% -
Race White 16%
An indicator of the effect of the intervention, the improvement index can be interpreted as the expected change in percentile rank for an average comparison group student if that student had received the intervention.
For more, please see the WWC Glossary entry for improvement index.
An outcome is the knowledge, skills, and attitudes that are attained as a result of an activity. An outcome measures is an instrument, device, or method that provides data on the outcome.
A finding that is included in the effectiveness rating. Excluded findings may include subgroups and subscales.
The sample on which the analysis was conducted.
The group to which the intervention group is compared, which may include a different intervention, business as usual, or no services.
The timing of the post-intervention outcome measure.
The number of students included in the analysis.
The mean score of students in the intervention group.
The mean score of students in the comparison group.
The WWC considers a finding to be statistically significant if the likelihood that the finding is due to chance alone, rather than a real difference, is less than five percent.
The WWC reviews studies for WWC products, Department of Education grant competitions, and IES performance measures.
The name and version of the document used to guide the review of the study.
The version of the WWC design standards used to guide the review of the study.
The result of the WWC assessment of the study. The rating is based on the strength of evidence of the effectiveness of the intervention. Studies are given a rating of Meets WWC Design Standards without Reservations, Meets WWC Design Standards with Reservations, or >Does Not Meet WWC Design Standards.
A related publication that was reviewed alongside the main study of interest.
Study findings for this report.
Based on the direction, magnitude, and statistical significance of the findings within a domain, the WWC characterizes the findings from a study as one of the following: statistically significant positive effects, substantively important positive effects, indeterminate effects, substantively important negative effects, and statistically significant negative effects. For more, please see the WWC Handbook.
The WWC may review studies for multiple purposes, including different reports and re-reviews using updated standards. Each WWC review of this study is listed in the dropdown. Details on any review may be accessed by making a selection from the drop down list.
Tier 1 Strong indicates strong evidence of effectiveness,
Tier 2 Moderate indicates moderate evidence of effectiveness, and
Tier 3 Promising indicates promising evidence of effectiveness,
as defined in the
non-regulatory guidance for ESSA
and the regulations for ED discretionary grants (EDGAR Part 77).