Project Activities
First, the research team assembled the comprehensive database, including aligning outcome measures across the component studies, coding the features of each intervention, and merging the 30 studies into a unified database for analyses. Next, the research team carried out the research agenda (goals 1 and 2 above), disseminating findings to researchers, practitioners, and funders. Finally, the research team made the project dataset available to outside researchers on a restricted-use (for legal/IRB reasons) basis.
Structured Abstract
Setting
The studies took place at over 45 broad-access postsecondary institutions (mostly community colleges) throughout the United States.
Sample
The sample includes over 60,000 students in 30 postsecondary education experiments conducted by MDRC from 2003 to 2019. The sample includes low-income individuals, first-time college students, and students in need of remediation.
The synthesis examined predictive relationships between intervention features and intervention impacts for community college students. Features included increased financial supports, increased advising, increased tutoring, learning communities, student success courses, promoting full-time and summer enrollment, and instructional reforms. In addition, the research team examined the relationship between intervention comprehensiveness and impacts.
Research design and methods
This project was a secondary analysis of individual-level data from a series of multi-site RCTs. The design built on two methodological strategies for analyzing data from multiple evaluations: (1) the multi-site random-effects model developed by Bloom, Hill, and Riccio (2003) for their secondary synthesis of data from MDRC's welfare/work experiments; and (2) the fixed-intercept, random-coefficient (FIRC) model developed by Bloom, Raudenbush, Weiss, & Porter (2017) for studying cross-site variation in program effects. The intent of the design is to find the crucial components of intervention strategies and assess their predictive relationship to impacts on postsecondary outcomes.
Control condition
Most control group students in the component studies were offered all their college's business-as-usual services but not the program services.
Key measures
Analyses focused on a parsimonious set of postsecondary academic outcomes including enrollment and credit accumulation, maximizing the number of studies and sample members included in the analyses.
Data analytic strategy
The research team used 2-level random-effects models to quantify and predict impact variation across the 39 interventions in THE-RCT database (students at level 1, studied interventions at level 2). This model first supports estimation of how much variation in impacts exists across the 39 interventions. Next, this model supports estimation of the relationship between the intensity of a given feature of an intervention (e.g., increased financial support) and the impact of the intervention on a given student outcome (e.g., credit accumulation).
Key outcomes
Findings reported in Weiss, Bloom, and Singh (2022) indicate that the impacts of community college interventions on credit accumulation during the first two semesters of college and persistence to the third semester of college increase with
- the comprehensiveness of the intervention, as measured by its number of intervention components
- promotion of full-time enrollment (during fall and spring) and summer enrollment
People and institutions involved
IES program contact(s)
Products and publications
Publications:
ERIC Citations: Find available citations in ERIC for this award here.
Select Publications:
Journal articles Weiss, M. J., Bloom, H. S., & Singh, K. (2022). What 20 Years of MDRC RCTs Suggest About Predictive Relationships Between Intervention Features and Intervention Impacts for Community College Students. Educational Evaluation and Policy Analysis. doi:10.3102/01623737221139493
Available data:
Student-level data, a study-level database, a codebook, and a user guide are available through the Inter-university Consortium for Political and Social Research (ICPSR): https://www.icpsr.umich.edu/web/ICPSR/studies/37932.
Related projects
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
