Project Activities
Researchers will begin by collecting VCCS administrative data on student enrollment and credential attainment in noncredit CTE programs, their progression into other noncredit and for-credit programs within VCCS, and programmatic characteristics of noncredit programs. They will merge the college administrative data with college attainment data from the National Student Clearinghouse (NSC) and individual quarterly employment and earnings data from the unemployment insurance (UI) database maintained by the Virginia Employment Commission. Researchers will also collect more detailed program data that are not readily available in the administrative data through a survey of administrators in the top 10 most highly enrolled programs at VCCS. Drawing on these different sources of data, the project will start with three preliminary inquiries that intend to delineate "basic facts" about noncredit CTE programs: 1) description of enrollment, participation, and academic outcomes for all students and subgroups in noncredit CTE programs at VCCS; 2) description of key program design and implementation factors for noncredit CTE programs, and their variation across programs and colleges, along with stakeholders' perceptions of these program features; and 3) a quasi-experimental analysis to estimate the labor market returns to noncredit CTE programs. Drawing on the "basic facts" garnered through the three preliminary inquiries, the research team will link these multiple sources of data together to estimate the relationship between malleable noncredit CTE program factors and student academic and labor market outcomes. In addition to the quantitative analyses, the team will also conduct interviews with program stakeholders to provide insight into the policy and institutional conditions that underlie variation in availability of services, instructional models, and employer engagement with specific programs. Student focus groups will also provide insight into how students experience delivery of program features.
Structured Abstract
Setting
The project will take place within the Virginia Community College System.
Sample
Researchers will employ two analytic samples. The first one, a statewide sample, will comprise the full set of noncredit CTE programs and all students enrolled in them from academic years 2017-2018 through 2023-2024. Given an average annual enrollment of over 7,000 students, this sample will include a total of approximately 49,000 students. To provide a more nuanced understanding of program design features, the second "deep dive" sample will include the 10 largest noncredit CTE programs at VCCS during the 2023-24 academic year and the students enrolled in them. Researchers expect to administer surveys to approximately 150 administrators of the top 10 programs distributed across colleges within the VCCS system and follow a sample of approximately 5,000 students enrolled in these programs.
Factors
Building on the existing literature on supporting community college students, researchers will focus on five domains of program factors: instructional characteristics; academic advising and non-academic support services; financial aid; career services; and employer relationships and work-based learning opportunities.
Research design and methods
To delineate "basic facts" about noncredit CTE programs, the research team will start with a series of descriptive analyses to describe the characteristics of students enrolled in noncredit CTE programs and their typical outcomes, as well as the variation in the five domains of program features across programs. The team will also conduct quasi-experimental analyses to investigate the impact that noncredit programs have on students' academic and labor market outcomes, for all students and key subgroups. Drawing on the "basic facts" about noncredit CTE programs, the main analyses will use multilevel modeling that combines program fixed effects with student random effects to identify program factors that are predictive of student outcomes and are malleable by programs or institutions. Researchers will also use interview data to delineate the varying approaches used by programs to design and deliver noncredit CTE programs.
Control condition
The descriptive analyses do not include a control condition. The quasi-experimental analysis will draw on variations in program factors across different training institutions that offer the same program. Researchers will compare student success outcomes in training institutions with high program factors to student outcomes in institutions with low program factors. Accordingly, programs with weak engagement of program factors will serve as the comparison group.
Key measures
Researchers will explore three sets of outcomes: 1) noncredit CTE program completion and credential attainment using the VCCS administrative data; 2) subsequent academic progression metrics in for-credit programs using a combination of the VCCS and NSC data, and 3) employment and earnings in the labor market using the UI data. To measure program factors, researchers will construct dichotomous and continuous measures of instructional characteristics and financial aid from the VCCS administrative data, as well as measures of advising, non-academic, and career support services from administrator surveys. From administrator interviews, researchers will code measures of the operating channels (mechanisms) through which specific program factors may affect student outcomes—such as student program engagement and access to services. Such measures may include, for example, presence or absence of cohort structure, frequency of contact between students and coaches/case managers and career services staff, and number and quality of service referral partners. The team will also explore student characteristics that may moderate impacts on student outcomes.
Data analytic strategy
Researchers will use descriptive analyses of key student outcomes and program features, including the extent of variation in program outcomes by student characteristics, occupational fields, and institutional features; a comparative individual fixed effect model to estimate the labor market returns to noncredit CTE; and a model that combines program fixed effects with student random effects to explore the relationship between program factors and student outcomes.
People and institutions involved
IES program contact(s)
Products and publications
Products: The team will disseminate findings regularly to administrators at VCCS and to national audiences. Working through leadership meetings at VCCS, the CTE Center housed at MDRC, and national conferences, the team will share products including policy briefs, conference presentations, and practitioner toolkits targeting audiences both within VCCS and nationwide. The team will also generate peer-reviewed publications.
Supplemental information
Co-Principal Investigators: Castleman, Benjamin; Tessler, Betsy; Finnegan, Catherine
In addition, researchers will use deductive coding to code and analyze interview data collected from program administrators, student service staff and providers, and employers. Researchers will start with an established codebook and reorganize it as the coding proceeds. To achieve a deep understanding regarding program implementation, researchers will look for themes across the coded data that address such questions as how the offer of program services, access to the program, and experiences in the program vary by student characteristics, with a focus on historically marginalized students.
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
