Description:

Co-Principal Investigators: Adriana Villavicencio, Meryle Weinstein, Sean Corcoran, Rebecca Unterman, Shaun Dougherty

Purpose: In this project, researchers will estimate the causal impact of New York City's (NYC) Career Technical Education (CTE) programs on students' career and work-related learning experiences, social and behavioral competencies, high school diploma receipt, and transitions to college and the labor market. The study will examine variation across more than 200 of NYC's CTE programs to identify malleable factors that are associated with impacts. Despite the proliferation of CTE programs, only a few studies can make causal claims about the impact of CTE participation on student outcomes. Findings from this study will inform CTE policy and practice for New York City, New York State, the federal government, and other states about the direction and evolution of urban educational reforms focused on leveraging CTE as a mechanism to improve college and career readiness while aligning with labor market needs.

Project Activities: Researchers will complete three studies. In the first study, the team will complete an implementation and process study, using structured interviews with CTE program directors, data from the NYC Department of Education (DOE)'s new Senior Exit Survey, and administrative data from the NYC DOE to provide a detailed description of the CTE landscape in NYC, assess its evolution during the study period, and set a foundation to investigate the mechanisms undergirding any relationship the researchers find between CTE participation and student outcomes. The researchers will next carry out a student choice and enrollment study, using data from NYC's High School Admission Process to school examine the choices that 8th graders make on their high school applications, and to explore their preferences for CTE programs. Finally, the research team will complete an impact study using NYC DOE data. In this study, they will estimate the causal effect of NYC CTE programs on students' CTE courses, work-based learning, internships, job shadowing, career fairs, and career counseling. The research team will also examine the impact of CTE programs on intermediate outcomes including: measures of student social and behavioral competencies and progress toward high school graduation. Finally, this study will examine the impact of CTE programs on longer-term outcomes including: receipt of NY state academic and CTE-endorsed Regents diplomas; enrollment, persistence and completion of 4-year college, 2-year college, or license/certificate programs; and postsecondary employment and earnings.

Products: The research team will produce a descriptive account of the state of and variability in CTE offerings in NYC, generate program-level variables to assist in mediator/moderator analyses, and describe the types of students who apply to and attend CTE programs in NYC. The research team will also produce evidence, using rigorous quantitative methods, of the effects of participating in NYC high school CTE programs on students' career and work-related learning experiences, social and behavioral competencies, high school diploma receipt, and transitions to college and the labor market. The project will also examine the annual recurring costs of CTE programs compare to the costs of operating the high school programs that students would have attended had they not been admitted to a CTE program. Researchers will produce peer-reviewed publications of the findings, datasets, and documentation about how to acquire a restricted-use agreement to access the data.

Structured Abstract

Setting: This project will take place in high schools in New York City.

Sample: Participants in this study include approximately 60,000 NYC students who enroll in one of more than 200 CTE programs as first-time 9th graders between 2014 and 2016. The sample will also include a similar number of students who enroll in other high school options as a control group. When tracking long-term outcomes, the researchers will also include another 120,000 students who enrolled in CTE programs between 2008 and 2013 and can be followed further into college and careers.

Research Design and Methods: The research team will rely on structured interviews, survey data, and extant administrative data to carry out the implementation and process study and the student choice and enrollment study. They will use a naturally occurring randomized control trial design that results from the NYC high school admissions process to generate causal estimates of CTE impacts on both proximal and long-term student outcomes. Then, to obtain a generalized estimate of impacts and to explore impact heterogeneity, they will use propensity-score matching (PSM) methods to first replicate the RCT results and then apply those methods to the full sample of CTE students (i.e., including those admitted through non-lottery methods).

Intervention: Students in the treatment condition participate in NY State Department of Education-endorsed CTE programs implementing components aligned with five tenets of NYC and federal CTE quality: 1) Preparing students for college and careers; 2) Engaging business and industry; 3) Establishing clear pathways from secondary to postsecondary education or training; 4) Creating opportunities for work-based learning; and 5) Helping students obtain industry-recognized credentials.

Control Condition: Approximately 70 percent of the study's control group is likely to be enrolled in academic high school programs with limited access to CTE-related courses and activities. The remaining 30 percent is likely to be exposed to some or all features of CTE programs either by being assigned to another CTE program of study or by taking CTE courses in addition to their academic requirements for graduation.

Key Measures: Researchers will use structured interviews with CTE program directors to build measures of CTE tenets, program characteristics, and implementation strategies. They will measure student outcomes from administrative records and surveys in three domains: exposure to CTE-program components; achievement and social competencies during high school; and academic and technical diploma receipt and post-secondary education and employment.

Data Analytic Strategy: The team will generate RCT-based impact estimates using linear regression models with outcomes as a function of CTE assignment. They will use PSM methods to replicate the RCT and then apply those methods to the full sample of CTE students. They will generate all estimates with multi-level models with fixed effects to account for variation across assignment blocks, random effects for each CTE program, and student-level covariates for added precision. The analysis will estimate both average treatment effects (ATE) and local average treatment effects (LATE).

This project is a member of the CTE Research Network, R305N180005.