Project Activities
The researchers will conduct two phases of activity. In the first phase, they will conduct a systematic literature review of nudging intervention studies. In the second, they will use meta-analytic methods to explore sources of intervention effect heterogeneity by grouping studies by intervention type and outcome domain.
Structured Abstract
Setting
The systematic review will include studies of students located in high schools and colleges across the United States.
Sample
The samples in the included studies will be U.S. or Canadian postsecondary students from both 2- and 4-year colleges or postsecondary-adjacent individuals, defined as high school seniors receiving postsecondary-related nudges, recent high school graduates, students who dropped out of postsecondary programs, and those who have delayed postsecondary enrollment.
The type of intervention under investigation is nudge interventions. The researchers will consider for inclusion in the meta-analysis several different types of nudges wherein information aimed at triggering active decision making is added to the decision environment. Informational nudges are composed of light-touch (i.e., easily consumable, bite-sized, short) targeted information delivered via text message or artificial intelligence (AI) chat bots aimed at increasing the likelihood of a specific behavior or action without taking choice away from the recipient (for example, student) with the intent of leading to a desired outcome. Nudges may alter or seek to influence the choice environment, for example by providing students with best options for course selection based on historical data and personalized student information. Early warning system nudges may employ predictive analytics to enter an early warning flag in a college's data system for students who are falling behind in their classes. Social comparison nudges seek to level up students' engagement by providing them with information that compares their performance or progress with that of their peers.
Research design and methods
This project will have two phases of research activity. The first phase will be a systematic literature review to find randomized controlled trials evaluating the effects of nudging interventions on postsecondary student outcomes. The second phase will use meta-analytic methods to explore sources of effect heterogeneity in these studies (for example, how effects vary by the nudge type). The researchers will test confirmatory hypotheses related to characteristics of nudging interventions, including the type of nudge, the sender, the stakes, the personalization of the nudges, and the socioeconomic status of participants. Exploratory analyses will examine other factors (for example, institutional characteristics, other student characteristics, outcome domain).
Control condition
All studies included in the analysis must include a comparison group that received business-as-usual programming (that is, did not receive the evaluated nudge intervention) or an active placebo control (for example, completed a personality test instead of receiving nudges to complete financial aid packages).
Key measures
Key outcome measures include college application and FAFSA completion; college readiness; college enrollment; progressing in college; progressing in developmental education; college degree attainment; workforce outcomes (for example, earnings, employment, general employability skills); industry-recognized credentials, certificates, or licenses; and the uptake of student academic resources (for example, visiting on-campus tutoring centers).
Data analytic strategy
The researchers will use mixed-effects meta-regression models to investigate how nudge intervention effects vary by moderators such as intervention design (for example, extent of personalization) and outcome type (for example, immediate behavior or longer-term academic outcome). These models assume that variation in effect sizes is due to the fixed effects of moderators (for example, intervention design), random effects of residual effect heterogeneity, and within-study sampling error. Following best practices for meta-regression, the researchers will use these models to control simultaneously for multiple moderators, including methods covariates (for example, control group type, attrition). They will use robust variance estimation to account for multiple effect sizes per study via a correlated and hierarchical working model for dependence.
Cost analysis strategy
The researchers will seek to extract cost estimates from each nudge intervention. Assuming sufficient data, they will explore relationships between cost and program effectiveness using meta-regression models that include methods covariates, as described in our data analytic strategy.
People and institutions involved
IES program contact(s)
Project contributors
Products and publications
The project will result in evidence of the promise of nudging strategies for promoting postsecondary success along with evidence regarding the promise of specific nudging strategies. The project will also result in a final dataset to be shared, peer-reviewed publications and presentations, and a video to detail findings and provide implications and recommendations for administrators and practitioners.
Publications:
ERIC Citations: Find available citations in ERIC for this award here.
Related projects
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
