Project Activities
The research team will obtain data from CPS and work with the Chicago Scholars program to randomize two cohorts of students to receive an offer of entering the program in their junior year of high school. During the trial, Chicago Scholars will be responsible for delivering the intervention to students. To assess implementation, the research team will administer surveys to all students in the treatment and control groups during their first and second years after scheduled high school graduation and will collect program records from Chicago Scholars staff. To assess impact, the research team will pre-register the study, collect data from the National Student Clearinghouse, and conduct confirmatory and exploratory analyses. The team will also carry out a cost analysis.
Structured Abstract
Setting
This study will take place in Chicago, IL, the third largest city in the US. Participants will be recruited from and attend Chicago-area public high schools.
Sample
The sample will consist of at least 1,100 low-income, first-generation, and under-represented minority youth from Chicago, IL, across two cohorts, recruited for enrollment during their junior year of high school when they are 16-18 years old.
The Chicago Scholars model is a 7-year leadership development program that provides college and career counseling, mentoring, workshops, and a supportive community of both adult resources and peers spanning high school, college, and post-college. The Chicago Scholars program currently serves low-income, first-generation, and under-represented minority youth from Chicago, IL beginning in their junior year of high school. Program supports continue through high school and college and provide one year of post-college support. Prior to the pandemic, the program delivered all supports in-person. Despite certain elements being adapted for virtual settings for 2020-2021, all program components have been retained.
Research design and methods
This study is a randomized controlled trial (RCT) of the Chicago Scholars program model assessing student postsecondary outcomes related to the first three years of programming: on-time college enrollment, college persistence through to sophomore year, quality of college match, and college graduation track record. The RCT is supplemented by an implementation analysis to measure program fidelity and understand additional services received by study participants from other sources. It is expected that some control and intervention participants will participate in other college success programs or other activities that impact the outcomes. Survey data will document the rates and types of receipt of these other programs for use in the impact analysis. The cost study will be conducted using the ingredients method.
Control condition
Participants assigned to the control condition will not receive any services from the Chicago Scholars program. Control participants will be referred to other college access and support services available in the Chicago area.
Key measures
Administrative data from CPS and Chicago Scholars applications will be used to construct baseline, pre-program measures of student characteristics and academic performance. Chicago Scholars records and surveys of student engagement will be used to categorize and quantify Chicago Scholars program components as well as additional services and support programs beyond Chicago Scholars. National Student Clearinghouse records will be used to construct the primary outcomes: on-time college enrollment and college persistence. Researchers will also draw on Integrated Postsecondary Educational Data System (IPEDS) data to construct additional outcome measures based on colleges in which students enroll, such as the selectivity of the college and its graduation rate for a student's own demographic group.
Data analytic strategy
The causal analysis of study outcomes will rely on random assignment of well-qualified students to treatment and control groups. An intent-to-treat (ITT) analysis will estimate the impact of admission to Chicago Scholars in comparison to non-receipt of the Chicago Scholars program services. A treatment-on-the-treated (TOT) analysis will use administrative and survey data on specific program receipt to compare receiving Chicago Scholars programming with control group students who receive either some specific program support, or no other comparable support. To assess the fidelity of implementation to the Chicago Scholars program and the service contrast between the program and other available supports, the team will analyze administrative program data and survey data from treatment and control students.
Cost analysis strategy
The research team will determine key program ingredients through a combination of analysis of Chicago Scholars budget materials, and qualitative follow-up with Chicago Scholars staff, students, and partners. Attention will be given to distinguishing costs as faced by Chicago Scholars in operational maturity, and those faced in a would-be replication. The analysis will consider programmatic and societal perspectives when accounting for costs. The team will assess cost effectiveness as the estimated per-student impact on each study outcome, for each $1,000 of program cost (in net present value). Costs of programs that are identified as taken up by control group students will be considered in these calculations. The team will provide ranges of both cost and cost-effectiveness, considering confidence intervals for estimated program effects and variation in cost-per-student.
People and institutions involved
IES program contact(s)
Products and publications
Researchers will generate rigorous evidence on the effectiveness of the Chicago Scholars program to facilitate progress through postsecondary education for low-income, first-generation, and under-represented minority youth. The team will produce accessible reports and briefs as well as peer-reviewed publications. Findings will be disseminated to policymakers, stakeholders, practitioners, and researchers through a range of venues.
Additional project information
Previous award details:
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
