|Title:||The Evaluation of Career and College Promise|
|Principal Investigator:||Edmunds, Julie||Awardee:||University of North Carolina, Greensboro|
|Program:||Evaluation of State and Local Education Programs and Policies [Program Details]|
|Award Period:||5 years (07/01/2019 – 06/30/2024)||Award Amount:||$4,968,353|
Co-Principal Investigators: Unlu, Fatih; Gold, Kimberly; Shah-Coltrane, Sneha
Partner Institutions: RAND Corporation, North Carolina Community College System, North Carolina Department of Public Instruction
Purpose: In this project, the partnership will examine the implementation, impact, and cost of all components of Career and College Promise (CCP), a set of three structured dual enrollment pathways in North Carolina (College Transfer, Career and Technical Education, and Cooperative Innovative High Schools, which includes early colleges). The partnership will build the capacity of the participating state agencies to support a cross-sector research agenda; allow for an in-depth and comprehensive response to legislative mandates; and enable a thorough investigation of long-term outcomes of CPP, including postsecondary degree attainment, employment, and earnings. Studying the CCP program will also contribute to the broader research base on the implementation and impact of dual enrollment programs and pathways.
Partnership Activities: The state agencies will participate in all aspects of the research, including design, developing instrumentation, data collection and analyses. Each state agency will support a research specialist who will have capacity development as part of their role. The partnership will embed cross-sector communication about research within existing administrative structures.
Project Activities: Using state administrative datasets, the partners will examine the impact of the dual enrollment pathways using rigorous quasi-experimental (QE) designs and will extend a prior experimental study of the impacts of the early college program by examining students' long-term outcomes. The team will also examine implementation of the pathways using multiple methods, including descriptive analyses of administrative data, surveys, and site visits. Finally, they will conduct a comprehensive cost-effectiveness analysis of each pathway.
Products: The partnership will produce estimates of the impacts of the CCP pathways on student outcomes, estimates of the cost effectiveness of CCP pathways, and a final dataset or documentation about how the data can be accessed. The partnership will also produce peer-reviewed publications, presentations, reports, and policy briefs.
Setting: The project is set in state of North Carolina, in all of the high schools and colleges there.
Sample:The study will focus on different samples depending on the research question; the largest sample will include all NC high school students who participate(d) in CCP between the 2012-13 and 2022-23 school years and a matched set of comparison students.
Intervention:The interventions are the three CCP pathway options.
Research Design and Methods:The partners will examine the impact; implementation; and cost and cost effectiveness of the CCP pathways. The team will use both a randomized controlled trial (RCT) and quasi-experimental (QE) methods using propensity score weighting, depending on the pathway and research question. The researchers will examine cost using the ingredients method, supplemented by analyses of administrative data.
Control Condition:Weighted comparison groups of eligible students who did not participate in any of the pathways will be created for each pathway group. The weights will be based on propensity scores, which represent the probability of participating in a given pathway, estimated using both student- and school-level covariates. For the subset of early colleges in the experimental study, the control group will consist of students who applied to the attend the early college but were randomly selected not to attend and enrolled in different schools.
Key Measures: Student-level covariates include grade level, multiple demographic characteristics, and baseline measures of student engagement (such as attendance, suspensions), and academic readiness (e.g., test scores at middle school end of grade exams, prior GPA). School-level covariates include school size and urbanicity, averages of student demographics and academic performance, and measures of school conditions and climate. Outcomes include college credits accrued in high school and high school graduation, as well as outcomes measured post high school such as postsecondary enrollment and degree attainment and wage and earnings.
Data Analytic Strategy: Impact analyses will account for nested data structures via hierarchical linear modeling. The researchers will also conduct moderator and mediator analyses to test their theories of change for each pathway.
Cost Analysis: The researchers will estimate the total cost of the CCP initiative at the system-level considering all cost components, including a variety of resources that support the design, delivery, monitoring and sustaining of the intervention. They will apply the principles of the “ingredients method” for cost analysis, which requires collecting information on the resources required for program implementation and their prices (or economic values) based on actual expenditures and off-budget items (e.g., donated space). Similar information on resources used and their valuation will be collected for the business-as-usual condition, so that incremental cost of the intervention will be captured. Finally, the researchers will compare the cost of the CCP intervention with the outcomes achieved as measured by the impact study, using cost-effectiveness analysis (CEA) and benefit-cost analysis (BCA, also known as cost-benefit analysis).
Related IES Projects:Impact of Early College High School (ECHS) Model on Postsecondary Performance and Completion (R305A140361)
This project is a member of the CTE Research Network, R305N180005.