Impacts of the Student Mentoring Program
Estimation of Overall Impacts of the Student Mentoring Program
We estimated a total of 17 impacts in three domains: (1) academic achievement and
engagement; (2) interpersonal relationships and personal responsibility; and (3)
high-risk or delinquent behavior.
- The Student Mentoring Program did not lead to statistically significant impacts
on students in any of the three outcome domains. The estimated impact on the Student
Mentoring Program on the outcome measures for all three domains is reported in Exhibit
- Three of the impacts were statistically significant before accounting for multiple
comparisons. However, after accounting for multiple comparisons within each of the
three domains, these three impact estimates were no longer statistically significant.
Estimation of Subgroup Effects
Several subgroup analyses were statistically significant after accounting for multiple
- The Student Mentoring Program improved academic outcomes for girls and produced
mixed academic outcomes for boys. There were several positive impacts of
the program for girls. The impact on self-reported scholastic efficacy and school
bonding was positive and statistically significant for girls, with treatment group
girls scoring higher than control group girls. In addition, there was a statistically
significant difference in impacts on the scholastic efficacy and school bonding
measure by gender (effect size for girls = 0.18 versus -0.05 for boys). There was
also a positive, statistically significant effect on future orientation for boys
(effect size = 0.17). However, the difference in impacts between boys and girls
on this measure was not statistically significant.
- For boys, the Student Mentoring Program negatively affected self-reported
prosocial behavior. Boys who were assigned to mentoring reported statistically
significant lower scores on the pro-social behaviors scale compared to their control
group peers. Moreover, there was a statistically significant difference in impacts
on the pro-social behaviors scale by gender (effect size for girls = 0.08 versus
– 0.11 for boys).
- The Student Mentoring Program led to a decrease in truancy for younger students.
Truancy (i.e., unexcused absence) showed a statistically significant improvement
for younger students (below age 12) who were assigned to mentoring compared to same
age peers in the control group (effect size = -0.23). However, the difference in
impacts on truancy between younger and older students (aged 12 and older) was not
statistically significant after accounting for multiple comparisons.
Site-Level Characteristics and Impacts
Although we did not find that the Student Mentoring Programs had statistically significant
impacts on student-level outcomes for our sample as a whole, we wished to determine
whether characteristics of programs and their mentors varied across sites and, if
so, whether we could identify program and mentor characteristics associated with
differences in impacts at the site level. Because sites were not randomly assigned
to different levels of implementation—a primary potential source of impact variation—this
analysis is descriptive and exploratory, not causal, in nature.
For this analysis, it was essential to develop a parsimonious model for testing
for any relationship between program and mentor characteristics (and contextual
factors) and site-level impacts. Therefore, in choosing the final set of site-level
covariates for inclusion in our model, we considered several factors, including
their theoretical importance in influencing impacts, possessing statistically significant
site-level variation, and low site-level correlations among these variables to avoid
problems with multicollinearity.11
The site-level covariates in our analysis included nine factors: (1) average hours
of pre-match training provided to mentors; (2) amount of ongoing mentor support
(average frequency of mentor-supervisor meetings); (3) use of activities in mentor/student
meetings (e.g., percent of mentors reporting almost always/most of the time either
working on homework and/or academic skills with students); (4) percent of mentors
aged 22 or below; (5) percent of mentor/student matches of the same race/ethnicity;
(6) percent of students with self-reported delinquent behaviors at baseline; (7)
percent of students scoring “not proficient” in either math or reading/ELA at baseline;
(8) percent of mentor/student matches lasting 6 months or longer; and (9) average
total hours of mentor/student meetings per month.12
Although we did not explicitly control for multiple comparisons because this was
an exploratory analysis, it is important to note that we conducted 153 individual
hypothesis tests of potential associations between the 9 covariates and the 17 outcome
measures, for roughly 7 or 8 of which we would expect to reject the null hypothesis
at the 0.05 level by random chance alone. In fact, we found 12 statistically significant
The following associations between site-level impacts and each of these site characteristics
were statistically significant at the 95 percent confidence level, holding all other
- The frequency of mentor/supervisor meetings was negatively associated with
sitelevel impacts. All other things equal, the frequency of mentor/supervisor
meetings was negatively associated with site-level impacts on the Pro-social Behaviors
measure from the Student Survey and on grades in math and social studies. They were
also positively associated with site-level impacts on school-reported delinquency.
- The proportion of students with self-reported delinquent behaviors at baseline
had both positive and negative relationships with site-level impacts. The
proportion of students with self-reported delinquent behaviors at baseline was positively
associated with site-level impacts on social studies grades and negatively associated
with site-level impacts on absenteeism and truancy.
However, the proportion of students with self-reported delinquent behaviors at baseline
was also positively associated with site-level impacts on repeated misconduct from
- The proportion of mentors aged 22 or younger was negatively associated with
sitelevel impacts on math grades.
- The proportion of mentor/student matches of the same race/ethnicity was
positively associated with site-level impacts on reading/ELA grades.
- Average monthly hours of mentor/student meetings had both positive and negative
relationships with site-level impacts. Average monthly hours of meeting
were positively associated with site-level impacts on student self-reported future
orientation, but negatively associated with site-level impacts on grades in math