Project Activities
During the 1.5 year development stage, researchers will work with the Association of Public Land-grant Universities (APLU) and seven universities to identify and assess the key features of completion grant programs including how they are targeted and delivered to students, their costs, their influence on students' financial aid packages, and how the participating universities implement them. During the second half of the development stage, researchers will work with APLU and three universities to conduct a relatively small pilot efficacy study that will measure the impact of existing completion grant programs on short-term outcomes including semester-to-semester persistence, social-psychological well-being, and on-time graduation. Findings from the development stage will inform completion grant program features to be tested during the evaluation stage, which will take place during the 2018-19 academic year. During the evaluation stage, researchers will work with APLU and eight universities to conduct a randomized-controlled trial of completion grants to assess their impacts on short- and medium-term student outcomes, the universities' implementation procedures, and the costs and relative cost-effectiveness of the programs. The research team will disseminate its findings from both stages of the project to a broad range of policymakers and researchers.
Structured Abstract
Setting
This project will take place with ten participating open- and broad-access postsecondary institutions, all of which are members of the APLU and are located across the United States.
Sample
The development stage will include seven universities, three of which will participate in a pilot efficacy study. The sample for the pilot efficacy study will include approximately 100 students per institution receiving a completion grant and an additional 150 students per institution assigned to the control group. The evaluation stage will include eight universities, with each providing approximately 250 completion grants and assigning an additional 375 students assigned to the control group, for a total sample size of approximately 5,000 students with about 2,000 students receiving grants. All students in the pilot and evaluation samples will have earned approximately three-quarters (90) or more of the credits needed for a college degree at the time they are selected for participation in the study.
The completion grants comprise four components: 1) financial aid; 2) messaging; 3) "skin-in-the-game" requirements; and 4) additional supports (e.g. advising) aimed at moving students to degree completion. The completion grants include a modest amount of financial aid ($500-$1500) targeted toward students who are within one year (30 credits) of completing a 4-year college degree (120 credits), and who lack funding from other sources (i.e. family contributions, and state or federal grant programs) to cover their tuition and student fee expenses. Messaging includes framing of the completion grants as well as the strategies that institutions use to communicate with students about the grants and the expectations attached to them. The "skin-in-the-game" requirements encourage students to take responsibility for succeeding in college. These requirements range from students making the relatively easy-to-fulfill promise to meet with their academic adviser to the more difficult pledge to complete a degree. In the latter case, some universities convert the completion grant to a loan if a student does not complete his or her degree. The additional supports (e.g. academic advising) are typically available to all students at the participating universities, but may take on special meaning in the context of the grants because students are required to make use of them.
Research design and methods
Researchers will randomly assign students to the treatment and control conditions for the pilot efficacy study and the evaluation. For both studies, the control group will be somewhat larger (60 versus 40 percent of the sample) due to the expense of providing the grants and the advantage of increasing statistical power. The research team will use a block randomization design in which the sample will be stratified by income status to guarantee adequate statistical power for detecting a significant difference in the impact of completion grants on low-income students relative to other students in the sample. The research team will use detailed case studies, drawing on administrator interviews and document review, to portray implementation of completion grants during the development and evaluation stages of the project. The study's design also includes an in-depth examination of the resources used and costs incurred by institutions to implement their programs.
Control condition
The control condition will include "business-as-usual" financial aid, messaging, and supports provided to all students at each of the universities. Students assigned to the control condition will not receive completion grants or any of the messaging and additional requirements connected to them.
Key measures
During the development and evaluation stages, researchers will work with APLU and institution-based researchers to refine their predictive analytic models for identifying students who could benefit from completion grants. This process will require collecting administrative data to assess students' progress to degree, their extent of financial need, and the content of their financial aid packages. During both stages, researchers will identify key implementation features of the grants through a combination of semi-structured interviews with program staff and administrators and review of program data. The research team will assess impacts of the grants on students' social-psychological well-being through surveys administered one semester after students receive the grants. During the pilot efficacy study, the team will rely on administrative data from the universities to measure short-term persistence outcomes. During the evaluation, the team will supplement administrative data with data from the National Student Clearinghouse, to identify students who complete a degree at a participating university as well as those that transfer to other universities. The team will construct measures of one-time and recurring program expenses through surveys of financial administrators, review of financial documents, and consultation with university staff. The team will develop cost per participant and cost effectiveness indicators for each completion grant program.
Data analytic strategy
For the predictive analytic models that assess risk of late college departure, researchers will use probit regression to compute the probability of dropping out due to financial hardship. Researchers will also use probit regression to estimate impacts on the probability of degree completion within the pilot efficacy study and the evaluation, and will include the covariates and interaction terms necessary for sub-group analysis. The research team will use the ingredients method to estimate the cost of the completion grant programs, as well as the cost of implementing them, during both stages of the project. Researchers will use resource cost modeling to develop program-specific resource profiles and cost estimates.
People and institutions involved
IES program contact(s)
Products and publications
Products: The products from this project include a fully-formed completion grant program, two early reports and a peer-reviewed paper drawing on findings from the development stage, and a peer-reviewed paper reporting on average and sub-group impacts based on findings from the evaluation stage.
Additional project information
This project is a research team in the College Completion Network.
Supplemental information
Co-Principal Investigators: Shari Garmise and Travis York (Association of Public and Land-grant Universities); Jed Richardson (University of Wisconsin-Madison); Douglas Webber (Temple University)
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
