|Title:||Assessing the Efficacy of the University System of Georgia's African-American Male Initiative|
|Principal Investigator:||Richburg-Hayes, Lashawn||Awardee:||MDRC|
|Program:||Scalable Strategies to Support College Completion Network [Program Details]|
|Award Period:||5 years (1/1/2016-12/31/2020)||Award Amount:||$3,980,372|
|Goal:||Development and Evaluation||Award Number:||R305N160025|
Related Network Teams: Related Network Teams: Nudges to the Finish Line: Experimental Interventions to Prevent Late College Departure (PI: Benjamin Castleman, R305160026)
Description: A Research Network involves several teams of researchers who are working together to address a critical education problem or issue. The objective is to encourage information-sharing, build new knowledge, and assist policymakers and practitioners to strengthen education policies and programs and improve student education outcomes. The College Completion Network is evaluating promising strategies for moving students at open- and broad-access postsecondary institutions beyond college enrollment and entry-level courses to degree completion. Each network research team is evaluating a specific intervention strategy.
Purpose: This research team will refine and evaluate a portion of the University System of Georgia's (USG) African American Male Initiative (AAMI) operating at five broad-access institutions within the system. This project addresses ethnic/racial and gender gaps in degree completion at broad-access postsecondary institutions. African American students at broad-access institutions graduate at a rate significantly below the 50 percent average for students of all ethnicities/races. The team will complete both development and evaluation activities. USG initiated AAMI in 2002 as a means of increasing enrollment, retention, and graduation for black males, who accounted for only 32 percent of USG's black enrollment at that time. The AAMI comprises four components intended to encourage college completion among African American male students: academic skill building, student support services, mentoring, and leadership development. The goals of this project are to: (1) refine the AAMI in ways that that improve its support for African American males; and (2) evaluate the effectiveness of the AAMI to assess whether it increases the percentage of African American male students who complete Associate's or Bachelor's degrees.
Project Activities: The team will refine the AAMI intervention, develop measures, and evaluate the impact and cost of the AAMI intervention. During the development stage of the project, researchers will review program documents and quantitative data collected by the participating colleges, hold semi-structured interviews with program staff, and observe program operations. These data collection activities will produce a qualitative description of each of the four AAMI program components, a qualitative comparison of AAMI and other support services available to students at the five colleges, and quantitative measures of AAMI fidelity and cost, and treatment-control contrast. Based on findings from this initial descriptive analysis, the research team will work with the University System of Georgia's AAMI leadership and campus-level staff to improve AAMI service delivery so that it reflects research evidence on promoting persistence and degree completion for African American males. Researchers will also work with staff to align program components across the five participating colleges.
During the evaluation stage of the project, researchers and administrators will randomly assign students to AAMI (treatment) or business-as-usual campus-based support services (control). The research team will collect data about the students' progress through college for three years. During the final half year of the project, researchers will analyze the collected data, produce an evaluation report, and disseminate their findings to interested policymakers and practitioners.
Products: The products from this project include a revised intervention, an interim report, a final report including implementation and evaluation findings, and a submission to a peer-reviewed journal. The project will also increase capacity at the system and institution levels for implementing the intervention.
Setting: This project will take place at five broad-access postsecondary institutions spread out across the University System of Georgia including: one designated 4-year Minority Serving Institution, one 4-year institution serving a racially-mixed student body, two majority-White 4-year institutions, and one majority-White 2-year institution.
Sample: The sample will include approximately 2,000 self-identified African American male college freshmen across the five participating postsecondary institutions.
Intervention: The USG's AAMI intervention includes four main program dimensions, the delivery of which varies somewhat across campuses. Academic skills enrichment occurs through summer bridge programs, first-year experience courses, learning communities, and co-curricular activities that align academic coursework with career mentoring and role modeling. Student support services include tutoring, academic planning, and performance monitoring. Faculty and support staff provide adult-to-student mentoring. Peer mentoring takes place through one-on-one student mentor-mentee assignments as well as regular, structured group meetings. Leadership development takes place through training for leadership roles within campus-based Registered Student Organizations, including the Student African-American Brotherhood organization, as well as via community-based service learning opportunities.
Research Design and Methods: The research design for the development stage focuses at the program level, and distinguishes between the bundle of components included in AAMI and the "business-as-usual" support services available to all students at each of the participating colleges. This stage uses qualitative methods to observe and compare current AAMI practices to business-as-usual services, and uses quantitative methods to assemble measures of implementation fidelity, treatment-control contrast, and intervention cost.
The evaluation phase will begin in in fall 2017. Researchers and administrators will randomly assign eligible, consenting students to the treatment and control conditions for the study. Students assigned to the treatment will receive the revised AAMI intervention from fall 2017 through spring 2020, and the research team will collect data on program implementation and student outcomes during this period.
Control Condition: The control condition will include "business-as-usual" support services available to students at each of the colleges. Control group students will not have access to components of the AAMI intervention.
Key Measures: During the evaluation, researchers will measure service uptake and dosage levels for all AAMI intervention components, as well as other support services taken up by control group students. Researchers will construct qualitative measures to track the fidelity of implementation of the USG's AAMI program components, and to describe the institutional context in which the program operates. A baseline survey will measure student and background characteristics at the beginning of the study. Researchers will administer two surveys during the evaluation stage to measure treatment group students' perceptions of and interactions with the AAMI intervention as well as control group students' perceptions of the business-as-usual services. These surveys will include the main hypothesized mediators of the intervention's effectiveness: student academic self-efficacy; positive relationships with peers and adults; and a sense of belonging on campus. Outcome measures will include short-term measures of course completions, credits earned, and GPA, as well as long-term measures of year-to-year persistence, transfer, and graduation. Researchers will collect course- and degree-completion data from the institutions, as well as enrollment and degree-completion data from the National Clearinghouse.
Data Analytic Strategy: Researchers will use standard statistical tests as well as OLS regression to compare differences across the treatment and control groups. Using an exploratory mediation analysis, the research team will examine whether the AAMI intervention acts via significant correlations between program components and the hypothesized mediators of the intervention's effectiveness. Researchers will use an implementation analysis to assess whether program impacts increase with increased fidelity to the treatment.