Co-Principal Investigator: Davis, Elisabeth

Purpose: This project willconceptually replicate and test whether Castleman and Page's “summer melt” text messaging intervention increases college enrollment, persistence, and credential attainment for college-intending seniors at high-poverty high schools when implemented at scale in routine educational settings. Researchers will also test a persistence enhancement that will provide additional supports during the first year of college. Text messaging is a popular and inexpensive strategy for reaching out to college-going students who may have limited access to the information in their schools and communities. While research findings tend to confirm that such interventions can induce larger percentages of low-income students to enter college, recent data shows limited evidence of such campaigns boosting rates of subsequent college degree attainment. The research team will rigorously test whether a well-designed campaign targeted to low-income students during their senior year of high school can increase their college enrollment and subsequent success. The research team will also test whether an extension of the campaign (through the first year of college) can prevent students from dropping out and boost rates of persistence and degree attainment beyond those of students who receive only the summer melt intervention.

Project Activities: During the first year of the project, the research team will randomly assign students to the treatment and control conditions and begin the summer melt intervention at high schools in early spring 2020. The team will continue the intervention through expected matriculation in fall 2020. Once students have matriculated, the team will redraw a random sample of students who received the summer melt intervention and offer them the persistence enhancement. The team will then implement the persistence intervention for this cohort for an additional year. The team will repeat the same sequence of treatment assignment and implementation for the second cohort beginning in early spring 2021, and they will track postsecondary progress for students in both cohorts through spring 2024. During the third through fifth years of the project, the team will share its findings with policymakers and administrators in the three participating states, share its findings more broadly with policymakers and practitioners across the country, and present its findings to research audiences.

Products: The research team will generate rigorous evidence about the effectiveness of text-messaging campaigns for reducing summer melt and promoting persistence beyond matriculation. The team will disseminate its findings to practitioners in the three participating states as well as to practitioners at broad-access institutions across the US via accessible policy briefs, web content, and practitioner conferences. The team will also disseminate its findings through research conferences and peer-reviewed publications. In addition, the team will document and release a final study dataset within one year of publishing its final impact report.

Structured Abstract

Setting: The project will take place in high schools, colleges, and universities within Alabama, Connecticut, and Indiana.

Sample: The sample will include approximately 18,000 high school seniors planning to attend public four-year postsecondary institutions, and 18,000 high school seniors planning to attend community colleges, distributed across two cohorts, the first in spring 2020 and the second in spring 2021.

Intervention: The summer melt intervention will begin during students' senior year of high school. For high school seniors who have applied to and been accepted by a four-year institution in their state, the intervention will begin in early spring with reminders to ensure that they complete a Free Application for Federal Student Aid (FAFSA), confirm enrollment, and register for college orientation. The summer melt intervention will text students with approximately 12 messages that highlight important steps that lead to starting college in the fall. For students planning to attend two-year institutions, the intervention will begin in early June of their senior year of high school. The first texts will remind students to complete a FAFSA so that they can begin to receive financial aid in the fall term. During the summer, the team will send texts with prompts and advice for completing the enrollment process, taking required college placement tests, and beginning classes. For a subset of students who receive the summer melt intervention and matriculate to college, the persistence intervention will begin during their fall after their senior year of high school and sustain for one year. The team will customize messages to each student's state and institution.

Research Design and Methods: This is a blocked randomized-controlled trial (RCT) study design with randomization at two time points. First, within the sampling frame of students at participating high schools who complete the intake survey, researchers will randomly assign students within schools (blocks) to receive either the summer melt text-messaging intervention (the treatment) or to not receive the intervention (the control condition). During early fall after senior year, within the sampling frame of students who have received the summer melt intervention and matriculated to college, researchers will randomize students to receive the persistence intervention (or not). At both randomization points, researchers will re-randomize, if necessary, to achieve balance across conditions on key subgroups of interest, such as urbanicity and prior access to college-going supports.

Control Condition: Students in the control group for the summer melt intervention will continue to receive business-as-usual college-application communications from their high schools but will not receive the summer melt messaging from this project. Students in the control group for the persistence intervention will continue to receive business-as-usual college-going support communications from their colleges but will not receive the persistence messaging from this project.

Key Measures: Researchers will administer an online survey to all students in participating high schools to inform to-be-developed measures of students' college intentions. Questions on the survey will also gauge the amount and types of college-going information that students have had access to prior to the study. State system administrative data and administrative data collected from the National Student Clearinghouse will be used to track students' postsecondary progress and degree completion from fall 2020 through spring 2024. Back-end data from the text-messaging provider will inform measures of implementation fidelity and student engagement with the summer melt and persistence text-messaging campaigns. Administrative data from state departments of education as well as student survey data will allow researchers to identify key student subgroups such as low-income and first-generation college students. Administrator interviews and state-level budget data will facilitate calculation of intervention costs.

Data Analytic Strategy: To estimate the effects of random assignment to receive the summer melt text messaging intervention, the team will employ multivariate regression models with a binary treatment indicator, student-level covariates, and a unique categorical variable for each student's high school. To estimate the effects of random assignment to receive the persistence intervention, researchers will use an identical model, except that a categorical variable for the student's college will replace the high school variable, in order to estimate college fixed effects. The binary treatment indicator will facilitate estimation of treatment impacts, and researchers will use additional treatment-by-covariate interaction terms to estimate impacts for specific student groups. The categorical school (or college) variable will aid estimation of a fixed effects parameter to soak up variation in student outcomes across schools (or colleges) that might otherwise result in biased estimation of treatment impacts.

Cost Analysis: The project team will use the resource cost model to first estimate intervention costs and will then compute cost-effectiveness ratios to estimate the cost per unit of outcome produced by the intervention.