Skip Navigation
Impact Evaluation of the U.S. Department of Education's Student Mentoring Program

NCEE 2009-4047
March 2009

The Study Sample

A total of 32 unique grantees met the above selection criteria and agreed to participate, comprising the final purposive sample. When identifying students for the study, grantees had categorical criteria to determine eligibility, such as grade level or school location. Sites also identified appropriate students in a variety of ways, most often asking school staff (such as teachers or counselors from the participating schools) to identify and refer students in need of mentoring to the program. To obtain an adequate sample size of students from the 32 grantees, a total of 2,573 students were recruited, 1,272 of whom were randomly assigned to receive mentoring services from the program and 1,301 that were randomly assigned to not receive these services.4 Students assigned to the control group were free to seek out other mentoring services in the community.

The majority of grantees participating in the Impact Study were non-profit/community-based organizations or faith-based organizations (66 percent) with an average of 6 years of experience with school-based mentoring programs. The average grantee in the Impact Study served 217 students with an annual budget of approximately $277,000. The majority of the grantees in the Impact Study reported having their school-based mentoring programs being extremely focused on improving student academic outcomes (91 percent), increasing students’ self-esteem (84 percent), providing students with general guidance (72 percent), and improving students’ relationships (63 percent). The majority of students served by the Impact Study grantees were female (57 percent) and a plurality were black or African American (41 percent), and in grades 6 through 8 (44 percent).

In addition to data for the 32 Impact Study grantees, data capturing program characteristics were also collected for a random sample of 100 grantees.5 The purpose of this random sample was to assess if the purposive sample used to assess program impacts was representative of the full universe of grantees funded through the Student Mentoring Program in 2004 and 2005 for some observable characteristics, as well as to provide additional descriptive information to ED.

Compared to this representative sample of randomly selected grantees, the grantees participating in the Impact Study were less likely to be non-profit/community-based organizations or faith-based organizations but more likely to be school districts, had more years of experience running schoolbased mentoring, had a larger annual budget, and served more students. Regarding program focus, grantees in the Impact Study reported being less focused on improving students’ academic outcomes and on teaching risk avoidance than the grantees in the representative sample. In addition, there were differences in the students served with grantees in the Impact Study serving more females and more Asian, Latino, and Pacific Islander students but fewer white students than the grantees in the representative sample. The Impact Study sample may also have differed from the representative sample of grantees in other ways that were not observed.

The student sample for the Impact Study had the following characteristics:

  • Gender: The student sample was 47 percent male versus 53 percent female.
  • Age: The average age of the sample was 11.2 years old.
  • Race/Ethnicity: Forty-one percent of the student sample was black or African American, and 31 percent was Hispanic.
  • Poverty: Eighty-six percent of the sample was eligible for either free or reduced price lunch.
  • Family structure: Fifty-six percent of the student sample came from two-parent households.
  • Risk status: Three-fifths (60 percent) of the student sample was at academic risk, defined by being below proficiency in either reading/English language arts (ELA) or math (or both) at baseline, and one-fourth of the sample (25 percent) was at risk for delinquency, defined by self-reported delinquent behaviors.
  • Prior mentoring experience: Twenty-six percent of the sample reported receiving mentoring in the prior school year.
  • Of the baseline characteristics assessed, only one statistically significant difference between the treatment and control group was observed. A higher proportion of students in the treatment group were eligible to receive free or reduced-price school lunches than in the control group.

Top

4 Of these 32 grantees, 10 provided students in both rounds of recruitment and random assignment. In other words, 10 of the 21 grantees recruited in 2006 had previously been recruited (and were part of our sample) in 2005, leaving us with 32 unique grantees.
5 A total of 12 out of the 100 randomly sampled grantees were also part of the purposive sample.