Co-Principal Investigator: Colleen Sommo

Purpose: In this project, the research team will assess whether the City University of New York's (CUNY's) Accelerated Study in Associate Programs (ASAP) intervention causes more college students to complete degrees, measured at seven years after initial college enrollment. This is a follow-up study to a previous efficacy study conducted by MDRC and funded by the New York City Center for Economic Opportunity. Because the ASAP intervention includes a comprehensive set of supports over a long (three-year) period, researchers expect it to exert positive impacts on student outcomes past the period of the initial efficacy trial. The previous study found that students randomly assigned to the ASAP program attained significantly more credits and were almost twice as likely to complete a degree after three years, compared to students in the control group. Because many students had completed enough credits to put them close to graduation, or had recently transferred to a 4-year institution, this research project is needed to assess students' accomplishments in the ASAP program given a longer time frame, and how these accomplishments compare to students receiving the standard set of supports available to them at CUNY campuses. If this evaluation finds ASAP effective, other college systems may be interested in implementing similar programs.

Project Activities: This follow-up study extends the previous randomized controlled trial by lengthening the period of observation from three to seven years. During the first year of the project, the team will produce a detailed report on implementation of the ASAP program, while tracking students' progress through college. During the second year of the project, the team will continue to track students' progress and produce an interim report comparing outcomes across the treatment and control groups at six years from the beginning of the study. During the third year of the study, the team will produce a final report comparing outcomes seven years from program enrollment, and costs of degree production in the ASAP program versus business-as-usual practices at CUNY.

Products: Researchers will produce evidence of the efficacy of the ASAP program for low-income community college students at CUNY. The researchers will produce a peer-reviewed implementation report, an interim outcomes report, and a peer-reviewed publication of their seven-year impact findings.

Structured Abstract

Setting: The research will take place at 3 colleges within the CUNY system.

Sample: Approximately 900 low-income community college students, evenly divided between the treatment and control groups, are included in the study. Over four-fifths of the students are traditional-age college students, between the ages of 17 and 24, and over three-quarters of the students are either Hispanic or African American. To participate in the study, students had to come from families with income below 200 percent of the federal poverty level, be in need of at least one developmental education course, be willing to attend college full time, be new or relatively new to college (having previously earned no more than 12 credits), and planning to enroll in an ASAP-eligible major.

Intervention: ASAP is a comprehensive, integrated, and highly monitored three-year program with four types of program components. Students are required attend college full time, encouraged to complete developmental education coursework early in their college careers, and to enroll in summer and winter courses. Student support services include enhanced advising, tutoring, and career planning services. Financial supports include a tuition waiver, a public transportation pass, and free textbooks. Structured course enrollments guarantee students blocked course schedules for the first year, and a success seminar for the first three semesters which emphasizes study skills, time management, and engagement with faculty.

Research Design and Methods: This follow-up study builds upon the previous efficacy study which randomly assigned equally-sized groups of students to the treatment and control conditions, and was deemed to meet What Works Clearinghouse evidence standards. Researchers used data from a baseline questionnaire administered at the time of entry into the study to confirm equivalence of the treatment and control groups. Given the strong research design and baseline equivalence of the two student groups, researchers will estimate intervention impacts for students in the evaluation sample using a standard linear regression model that they employed during the initial efficacy study. Because students volunteered for the initial study, and thus do not represent the CUNY student population as a whole, researchers will access data for the entire pool of students from which students volunteered for the study, and will estimate a propensity score model to measure the likelihood of each student joining the study. In turn, researchers will convert the participation propensities to weights for use in regression models that will re-estimate ASAP's impacts for the CUNY student population. Across analyses, the project will assess the ASAP intervention as a bundle, without breaking it into component parts.

Control Condition: Control group members were eligible to receive all of the colleges' standard services, but not the enhanced services and program components available to students enrolled in ASAP.

Key Measures: Researchers will rely on previously-collected demographic data collected from a baseline survey administered prior to randomization as well as items provided from CUNY's Institutional Research Database (IRDB). College attainment measures will include course completions and credits earned, from transcripts provided by IRDB. Degree completion measures including earning a certificate, an associate's degree, a bachelor's degree, and earning any certificate or degree will come from IRDB for students who remain in CUNY and from the National Student Clearinghouse for students who move outside CUNY. Measures of program cost will be computed from measures provided by the CUNY Central ASAP office for students receiving the intervention, and from IPEDS data for students in the control group.

Data Analytic Strategy: The researchers will employ linear regression models with dummy variables to account for the blocking strategy they used when randomizing students to the treatment and control conditions. Split sample regressions with a statistical test for significant sub-group differences will be used for the moderation analysis, which will assess whether treatment impacts differ for students of color and white students, for first- and continuing-generation college students, for older and younger students, and for working and non-working students. To estimate program impacts within the target population, researchers will employ inverse probability weights to match observable characteristics of students in the analytic sample to students in the target population at CUNY.

Project Website: Evaluation of Accelerated Study in Associate Programs (ASAP) for Developmental Education Students