Role: Research Team

Co-Principal Investigators: Shari Garmise and Travis York (Association of Public and Land-grant Universities); Jed Richardson (University of Wisconsin-Madison); Douglas Webber (Temple University)

Related Network Teams: College Completion Network Lead (PI: Eric Bettinger, R305N170003); Nudges to the Finish Line: Experimental Interventions to Prevent Late College Departure (PI: Benjamin Castleman, R305N160026); The Men of Color College Achievement (MoCCA) Project (PI: Lashawn Richburg-Hayes, R0305N160025); A Scalable Growth Mindset Intervention to Raise Achievement and Persistence in Community College (PI: Gregory Walton, R305A150253); An Experimental Evaluation of Corequisite Developmental Education in Texas (PI: Trey Miller, R305H170085)

A Research Network involves several teams of researchers who are working together to address a critical education problem or issue. The objective is to encourage information-sharing, build new knowledge, and assist policymakers and practitioners to strengthen education policies and programs and improve student education outcomes. The College Completion Network is evaluating promising strategies for moving students at broad-access postsecondary institutions beyond college enrollment and entry-level courses to degree completion. Each network research team is evaluating a specific intervention strategy.

Purpose: This project will refine and evaluate completion grants, a form of financial aid that some postsecondary institutions provide to students who are close to completing their degrees yet at risk of departure from college due to insufficient funding to cover their tuition and student fees. Completion grants combine relatively small amounts of financial grant aid ($500 to $1500) with "skin in the game" requirements,which encourage students to take responsibility for succeeding in college. The main objectives during the development phase of the project are to refine the targeting of grants to students who can benefit from them; improve the messaging, requirements, and supports associated with the grants; and conduct a pilot impact evaluation of completion grant programs. During the evaluation stage, the main objective is to assess whether the completion grants cause students who receive them to complete their degrees at a higher rate than students who do not have access to the grants. Policymakers at the institution and system levels will likely be interested in this relatively low-cost intervention if it proves beneficial.

Project Activities: During the 1.5 year development stage, researchers will work with the Association of Public Land-grant Universities (APLU) and seven universities to identify and assess the key features of completion grant programs including how they are targeted and delivered to students, their costs, their influence on students' financial aid packages, and how the participating universities implement them. During the second half of the development stage, researchers will work with APLU and three universities to conduct a relatively small pilot efficacy study that will measure the impact of existing completion grant programs on short-term outcomes including semester-to-semester persistence, social-psychological well-being, and on-time graduation. Findings from the development stage will inform completion grant program features to be tested during the evaluation stage, which will take place during the 2018-19 academic year. During the evaluation stage, researchers will work with APLU and eight universities to conduct a randomized-controlled trial of completion grants to assess their impacts on short- and medium-term student outcomes, the universities' implementation procedures, and the costs and relative cost-effectiveness of the programs. The research team will disseminate its findings from both stages of the project to a broad range of policymakers and researchers.

Products:  The products from this project include a fully-formed completion grant program, two early reports and a peer-reviewed paper drawing on findings from the development stage, and a peer-reviewed paper reporting on average and sub-group impacts based on findings from the evaluation stage.

Structured Abstract

Setting: This project will take place with ten participating open- and broad-access postsecondary institutions, all of which are members of the APLU and are located across the United States.

Sample: The development stage will include seven universities, three of which will participate in a pilot efficacy study. The sample for the pilot efficacy study will include approximately 100 students per institution receiving a completion grant and an additional 150 students per institution assigned to the control group. The evaluation stage will include eight universities, with each providing approximately 250 completion grants and assigning an additional 375 students assigned to the control group, for a total sample size of approximately 5,000 students with about 2,000 students receiving grants. All students in the pilot and evaluation samples will have earned approximately three-quarters (90) or more of the credits needed for a college degree at the time they are selected for participation in the study.

Intervention: The completion grants comprise four components: 1) financial aid; 2) messaging; 3) "skin-in-the-game" requirements; and 4) additional supports (e.g. advising) aimed at moving students to degree completion. The completion grants include a modest amount of financial aid ($500-$1500) targeted toward students who are within one year (30 credits) of completing a 4-year college degree (120 credits), and who lack funding from other sources (i.e. family contributions, and state or federal grant programs) to cover their tuition and student fee expenses. Messaging includes framing of the completion grants as well as the strategies that institutions use to communicate with students about the grants and the expectations attached to them. The "skin-in-the-game" requirements encourage students to take responsibility for succeeding in college. These requirements range from students making the relatively easy-to-fulfill promise to meet with their academic adviser to the more difficult pledge to complete a degree. In the latter case, some universities convert the completion grant to a loan if a student does not complete his or her degree. The additional supports (e.g. academic advising) are typically available to all students at the participating universities, but may take on special meaning in the context of the grants because students are required to make use of them.

Research Design and Methods: Researchers will randomly assign students to the treatment and control conditions for the pilot efficacy study and the evaluation. For both studies, the control group will be somewhat larger (60 versus 40 percent of the sample) due to the expense of providing the grants and the advantage of increasing statistical power. The research team will use a block randomization design in which the sample will be stratified by income status to guarantee adequate statistical power for detecting a significant difference in the impact of completion grants on low-income students relative to other students in the sample. The research team will use detailed case studies, drawing on administrator interviews and document review, to portray implementation of completion grants during the development and evaluation stages of the project. The study's design also includes an in-depth examination of the resources used and costs incurred by institutions to implement their programs.

Control Condition: The control condition will include "business-as-usual" financial aid, messaging, and supports provided to all students at each of the universities. Students assigned to the control condition will not receive completion grants or any of the messaging and additional requirements connected to them.

Key Measures: During the development and evaluation stages, researchers will work with APLU and institution-based researchers to refine their predictive analytic models for identifying students who could benefit from completion grants. This process will require collecting administrative data to assess students' progress to degree, their extent of financial need, and the content of their financial aid packages. During both stages, researchers will identify key implementation features of the grants through a combination of semi-structured interviews with program staff and administrators and review of program data. The research team will assess impacts of the grants on students' social-psychological well-being through surveys administered one semester after students receive the grants. During the pilot efficacy study, the team will rely on administrative data from the universities to measure short-term persistence outcomes. During the evaluation, the team will supplement administrative data with data from the National Student Clearinghouse, to identify students who complete a degree at a participating university as well as those that transfer to other universities. The team will construct measures of one-time and recurring program expenses through surveys of financial administrators, review of financial documents, and consultation with university staff. The team will develop cost per participant and cost effectiveness indicators for each completion grant program.

Data Analytic Strategy: For the predictive analytic models that assess risk of late college departure, researchers will use probit regression to compute the probability of dropping out due to financial hardship. Researchers will also use probit regression to estimate impacts on the probability of degree completion within the pilot efficacy study and the evaluation, and will include the covariates and interaction terms necessary for sub-group analysis. The research team will use the ingredients method to estimate the cost of the completion grant programs, as well as the cost of implementing them, during both stages of the project. Researchers will use resource cost modeling to develop program-specific resource profiles and cost estimates.