Co-Principal Investigator: Colleen Sommo

Purpose: The project will extend an ongoing randomized control trial, known as the Performance-Based Scholarship (PBS) Demonstration that is evaluating the impacts of incentive scholarships on postsecondary outcomes for low-income college students. The PBS Demonstration began in 2008 and has recruited over 10,000 students across four states. Funded by other sources, the PBS Demonstration tracks college students' academic outcomes for up to two years after random assignment. This project will further examine two years of outcome data for students who took part in the demonstration in three of the states.

Project Activities: To examine longer-term impacts of PBS, the project will gather and analyze later data on students who took part in the PBS Demonstration. The project will follow students who are part of the PBS Demonstration in three states: (1) at three community colleges in Ohio, (2) at one four-year university in New Mexico, and (3) in California where students can use the scholarship at the community or four-year college of their choice. In Ohio, the PBS supports students for up to one year of college and this project will track the outcomes of one cohort of students after the scholarship program ends in their second and third years of college. In New Mexico, the scholarship supports students for up to two years of college and this project will collect data on the outcomes of one cohort of students after the scholarship ends for an additional two years of college. California offers six variations of the scholarship that run from one-semester to two years. The PBS Demonstration has relied on a student survey to collect data for the California program and this project will collect student transcript data to measure student academic performance.

Products: The project will determine whether the PBS effects student persistence in postsecondary education after the scholarship has ended. These results will be reported in peer reviewed journals.

Structured Abstract

Setting: The Performance-Based Scholarship (PBS) data will be collected on students in three states including: one urban and two suburban community colleges in Ohio; one four-year public university in New Mexico; and a statewide program in California.

Population: The PBS Demonstration enrolled over 10,000 students in four states (students in three states will be followed in this project). These students represent different target populations including low-income mothers eligible for Temporary Assistance for Needy Families (TANF) in Ohio, low-income students aged 16 to 26 years old in New Mexico, and low-income high school seniors in California. The sample to be followed in this project includes approximately 2,300 students in Ohio, 1,100 students in New Mexico, and 2,300 students in California.

Intervention: While the specifics of the PBS differ by state, the underlying approach is similar. Performance benchmarks that are to be met in every term are set for students. These benchmarks can include attendance, meeting with an adviser, grades, and credits completed. The performance-based scholarships (PBS) are linked to meeting these benchmarks. Aid is disbursed in multiple payments over a term with a payment given after a benchmark is met (except in Ohio where one end of term payment is given). Performance in one term does not affect payments in future terms. The payments go directly to the student rather than to the institution.

In general, students receiving PBS have to enroll at least half-time and earn a "C" or better average in order to receive scholarship payments. Depending on the site, scholarship payments range from $900 to $4,000 and are generally provided in addition to all other financial aid for which students qualify.

Control Condition: Control group students do not receive PBS but can receive financial aid for which they qualify.

Research Design and Methods: This project will collect follow-on data of students who were randomly assigned to PBS or a control group by the PBS Demonstration. In the PBS, Demonstration eligible students were identified and invited to take part. Those agreeing to participate in the Demonstration were randomly assigned to the PBS group or the control group using an assignment ration of 50/50 in New Mexico and 60/40 in Ohio. In California, students who attended program workshops, met the eligibility requirements, and agreed to take part were assigned to one of six treatments or the control group. The original California sample included over 5,000 students and about two-thirds were randomly assigned to the control group.

Key Measures: The primary outcome measures will be student persistence including continued enrollment, transfer to four-year institutions by community college students, and receipt of certificates and degrees. The sources for these measures will be the student transcript data from the National Student Clearinghouse, the colleges, and the state intermediaries.

Data Analytic Strategy: The primary analytic method to determine program impacts will be comparing average outcomes for program and control group members using standard statistical tests such as t-tests and ordinary least squares regression. Separate analyses will be carried out by state and students will be pooled within state. Mediator analyses, using survey data collected by the PBS Demonstration, will be conducted to examine the role of increased student effort (increased study time or reduced work hours), adoption of a positive stereotype from receiving a scholarship, and use of the scholarship to address financial barriers to academic success (e.g., child care expenses, transportation).

Products and Publications

Journal article, monograph, or newsletter

Mayer, A.K., Patel, R., and Gutierrez, M. (2016). Four-Year Degree and Employment Findings From a Randomized Controlled Trial of a One-Year Performance-Based Scholarship Program in Ohio. Journal of Research on Educational Effectiveness, 9(3), 283–306.