IES Blog

Institute of Education Sciences

It All Adds Up: Why and How to Measure the Cost of Career & Technical Education

Cost analysis is a critical part of education research because it communicates what resources are needed for a particular program or intervention. Just telling education leaders how promising a program or practice can be does not tell the whole story; they need to know how much it will cost so that they can prioritize limited resources. Since 2015, cost analysis has been required for IES-funded Efficacy/Impact studies (and for Development Innovation studies as of 2019) and is included in the IES Standards for Excellence in Education Research.

In this guest blog for CTE Month, two members of the CTE Research Network’s cost analysis working group, David Stern, an advisor to the network, and Eric Brunner, a co-PI of one of the research teams, discuss how costs associated with CTE programs may differ from those of standard education and how to measure those costs.

Photo of David SternWhy is cost analysis different in Career & Technical Education (CTE) research?

Due to additional, non-standard components needed in some types of career training, CTE can cost much more than the education resources needed in regular classrooms. For instance, CTE classes often use specialized equipment—for example, hydraulic lifts in automotive mechanics, stoves and refrigerators in culinary arts, or medical equipment in health sciences—which costs significantly more than equipment in the standard classroom. Having specialized equipment for student use can also constrain class size to be smaller, resulting in higher cost-per-pupil.  High schools and community colleges may also build labs within existing buildings or construct separate buildings to house CTE programs with specialized equipment. These required facility expenses will need to be recognized in cost calculations.

CTE programs can also provide co-curricular experiences for students alongside classes in career-related subjects, such as work-based learning, career exploration activities, or integrated academic coursework. Schools are usually required to provide transportation for students to workplaces, college campuses for field trips, or regional career centers, which is another expense. Finally, the budget lines for recruiting and retaining teachers from some higher paying career areas and industries (such as nursing or business) may exceed those for average teacher salaries. All of these costs add up. To provide useful guidance for the field, CTE researchers should measure and report the cost of these features separately.

Photo of Eric BrunnerHow is resource cost different from reported spending? 

There are also some hidden costs to account for in research on CTE. For example, suppose a school does not have a work-based learning (WBL) coordinator, so a CTE teacher is allowed one of their 5 periods each day to organize and oversee WBL, which may include field trips to companies, job shadowing experiences, internships, or a school-based enterprise. The expenditure report would show 20% of the teacher’s salary has been allocated for that purpose. In reality, however, a teacher may devote much more than 20% of their time to this. They may in fact be donating to the program by spending unpaid time or resources (such as transportation in their own vehicle to visit employer sites to coordinate learning plans) outside the workday. It is also possible that the teacher would spend less than 20% of their time on this. To obtain an accurate estimate of the amount of this resource cost at a particular school, a researcher would have to measure how much time the teacher actually spends on WBL.  This could be done as part of an interview or questionnaire.

Similarly, high school CTE programs are increasingly being developed as pathways that allow students to move smoothly to postsecondary education, such as via dual enrollment programs or directly to the labor market. Building and sustaining these pathways takes active collaboration between secondary and postsecondary educators and employers. However, the costs of these collaborations in terms of time and resources are unlikely to be found in a school expenditure report. Thus, an incremental cost analysis for CTE pathway programs must go beyond budgets and expenditure reports to interview or survey program administrators and staff about the resources or “ingredients” that programs require to operate. A recent example of a cost study of a CTE program can be found here.

Are there any resources for calculating CTE Costs?

In this blog, we have presented some examples of how the costs associated with CTE programs may differ from those of a standard education. To help CTE researchers conduct cost analysis, the CTE Research Network has developed a guide to measuring Incremental Costs in Career and Technical Education, which explains how to account for the particular kinds of resources used in CTE. The guide was developed by the working group on cost analysis supported by the CTE Research Network.


The Career and Technical Education (CTE) Research Network has supported several cross-network working groups comprised of members of network research teams and advisors working on issues of broad interest to CTE research. Another CTE Network working group developed an equity framework for CTE researchers, which was described in a blog for CTE month in February, 2023.

This blog was produced by Corinne Alfeld, NCER program officer for the CTE research topic and the CTE Research Network. Contact: Corinne.Alfeld@ed.gov.

Training the Next Generation of CTE Researchers: A Conversation with the CTE Research Network

IES funded the Expanding the Evidence Base for Career and Technical Education (CTE) Research Network (CTERN) in FY 2018 in order to increase the quality and rigor of CTE research, specifically by (1) coordinating IES-funded researchers studying CTE using causal designs and (2) training new researchers in causal methods to address CTE-related research questions. In this guest blog, the Network Lead’s PI, Katherine Hughes, and Training Lead, Jill Walston, from the American Institutes for Research (AIR), discuss the evolution of the institute across four years of training supported by the grant and what they learned about the components of effective training, in the hopes of sharing lessons learned for future IES-funded trainings.

About the Summer Training Institute

Each summer since 2020, CTERN has held summer training institute on causal research methods in CTE.  Across four summers, we had 81 trainees, including junior faculty, researchers in state or university research offices or institutes, doctoral students, and researchers in non-profit organizations. During the institutes, we had expert CTE researchers and national and state CTE leaders deliver presentations about CTE history, policies, theories, and recent research.

The major focus of the training was on research designs and statistical methods for conducting research that evaluates the causal impact of CTE policies and practices on student outcomes. The participants learned about conducting randomized controlled trials—considered the gold standard for causal research—as well as two quasi-experimental approaches, regression discontinuity and comparative interrupted time series designs. After presentations about the approaches, students worked with data in small groups to complete data analysis assignments designed to provide practical experience with the kinds of data and analyses common in CTE research. The small groups had dedicated time to meet with one of the instructors to discuss their analyses and interpret findings together. The combination of presentations and practical applications of data analysis with real data, and time in small groups for troubleshooting and discussion with CTE researchers, made for a rich experience that students found engaging and effective. The students received an IES certificate of course completion to mark their accomplishment.

Making Continuous Improvements Based on Lessons Learned

We had a continuous improvement mindset for our summer institute. After each week-long session was completed, the CTE research network director, training coordinator, and instructors met to review their perceptions of the training and most importantly the feedback students provided at the end of the week. We applied the lessons learned to make improvements to the agenda, communications, and student grouping approaches to the plans for the following summer.

Over the course of the four years of the summer institute training, we made a number of adjustments in response to feedback.

  • We continued to offer the institute virtually. The institute was originally intended to be held in person; an earlier blog describes our necessary pivot to the online format. While we could have safely changed to an in-person institute in 2022 and 2023, feedback from our students showed that the virtual institute was more accessible to a geographically diverse group. Many trainees said they would not have even applied to the institute if they would have had to travel, even with a stipend to help cover those costs.
  • We added more time for the students to get to know one another with virtual happy hours. Compared to in-person trainings, virtual trainings lack those natural opportunities for informal communications between students and with instructors that can foster engagement, trust, and joint purpose. While we couldn’t replicate in-person networking opportunities, we were able to improve the experience for the students by being intentional with informal gatherings.
  • We expanded the time for the small groups to meet with their instructors. Students reported that this office hour time was very valuable for their understanding of the material and in interpreting the output of the analyses they ran. We extended this time to optimize opportunities for discussion and problem solving around their data analysis assignments.   
  • We made improvements to the data assignment guidance documents. In the first year, students reported that they spent more time on figuring out initial tasks with the data which left less time for running analyses and interpreting their output. We modified our guidance documents that accompanied the assignments to spell out more explicitly some of the initial steps to shorten the time students spent on set-up and maximize their time doing the important work of coding for the analyses and examining output. We also provided links to resources about the statistical packages used by the students for those that needed time to brush up on their skills before the training began.
  • We doubled down on efforts to stay connected with the trainees and supported ways to have them stay connected to each other. For example, we let them know when CTERN’s researchers are presenting at conferences and invite them to connect with us and each other at these conferences. We’re now organizing a LinkedIn group to try to develop a community for our training alumni.

Our summer training institutes were a great success. We look forward to continuing this opportunity for researchers into the future, with a new version to be offered in the summer of 2025 by the CTE Research Network 2.0.


Jill Walston, Ph.D., is a principal researcher at the American Institutes for Research with more than 20 years of experience conducting quantitative research, developing assessments and surveys, and providing technical support to researchers and practitioners to apply rigorous research and measurement practices. Dr. Walston is the lead for training initiatives for the IES-funded Career and Technical Education Research Network.

Katherine Hughes, Ph.D., is a principal researcher at the American Institutes for Research and the principal investigator and director of the CTE Research Network and CTE Research Network 2.0. Dr. Hughes’ work focuses on career and technical education in high schools and community colleges, college readiness, and the high school-to-college transition.

This blog was produced by Corinne Alfeld (Corinne.Alfeld@ed.gov), a Program Officer in the National Center for Education Research (NCER).

 

Variation Matters: A Look at CTE Under Distinctive Policy and Programming Conditions

Young diverse students learning together at stem robotics class - Hispanic Latina female building electronic circuits at school

February is Career and Technical Education (CTE) month! As part of our 20th anniversary celebration, we want to highlight the great work our CTE Research Network (CTERN) continues to accomplish. This guest blog was written by James Kemple, Director of the Research Alliance for New York City Schools and the principal investigator (PI) of a CTERN research project that is examining CTE in New York City.

While “college and career readiness” are familiar buzzwords in K-12 education, it has often seemed like system leaders shout “college” and whisper “career.” During the last decade, as it has become clear that a high school diploma has limited value in the 21st century labor market, career and technical education (CTE) has become a more prominent way to explicitly prepare students for both college and career. For the CTE field to evolve productively, valid and reliable evidence should inform policy and practice, for example by identifying conditions under which CTE may be more or less effective and for whom.

CTE and College and Career Readiness in New York City

One such project is our ongoing study of New York City’s CTE programs. The current phase of the study focuses on 37 CTE-dedicated high schools, which are structured to ensure that all enrolled students participate in a CTE Program of Study from 9th through 12th grade. These programs are organized around an industry-aligned theme (for example, construction, IT, health services, etc.) and offer a sequence of career-focused courses, work-based learning opportunities, and access to aligned college-level coursework. Our study uses an especially rigorous approach to compare the experiences and outcomes of nearly 19,000 NYC students who were assigned to a CTE-dedicated high school between 2013 and 2016 with those of similar students who also applied to CTE programs but were assigned to another high school during the same period.

When our research team looked at the overall impact of 37 CTE-dedicated high schools in NYC, we found that CTE students graduated from these high schools and enrolled in college at rates that were similar to their counterparts in non-CTE high schools. On average, therefore, being in a CTE high school did not steer students away from a college pathway.

Variations in CTE Programs

A much more interesting story emerged when we took a closer look at variation in student experiences and outcomes. In fact, some of the schools produced statistically significant reductions in immediate college enrollment, while others produced increases in the rate at which students enrolled in college.  Why might this be?

The study team identified two possible reasons. First, the schools in our sample differed based on the policy context in which they were created: 21 of the schools were established after 2008 as the NYCDOE undertook a major expansion of CTE in the midst of a larger overhaul of the city’s high schools that included closing persistently low-performing schools, opening new small schools in their place, and creating a universal high school admissions system that gave students access to schools across the city. In contrast to the 16 longstanding CTE high schools—some of which dated back to the early 1900s—these new high schools were smaller, with more thematically aligned sets of CTE programs, and non-selective admission processes. Most of the longstanding CTE schools used test scores, grades, or other performance measures as part of their admissions criteria.

Second, the schools in the study differed in terms of their intended career pathways—and the extent to which these career pathways require a post-secondary credential for entry-level jobs. Notably, nine of the newer (post-2008) high schools focused on career pathways that were likely to require a bachelor’s degree (referred to “college aligned”). CTE programs in the remaining 12 newer high schools and all of the CTE programs in the 16 longstanding high schools focused on either “workforce-aligned” career pathways—allowing students to enter the labor market directly after high school—or “mixed” pathways that require additional technical training or an associate degree for entry-level jobs. Interestingly, each of these groups of schools included a mix of CTE career themes. For example, some health- or technology-focused CTE programs reflected college-aligned pathways, while other programs with these themes reflected workforce-aligned pathways.

We found that the newer, smaller, less selective CTE schools with more tightly aligned career themes had positive effects on key outcomes—particularly those that were focused on college-intended career paths. These schools produced a substantial, positive, statistically significant impact on college enrollment rates. Students in these schools were nearly 10 percentage points more likely to enroll in a four-year college than those in the non-CTE comparison group.

By contrast, the larger, more selective CTE schools, with a range of work-aligned career pathways, were associated with null or negative effects on key outcomes. Notably, these schools actually reduced four-year college enrollment rates.

Applying Lessons Learned

The extraordinary diversity of NYC’s CTE landscape and its student population provides a unique opportunity to gather information about program implementation, quality, accessibility, and costs, and about how these factors influence CTE’s impacts on college and career readiness. A recent report from the project provides new insights into strategies for learning from variation in CTE programs and contexts, as well as particular policies and programming conditions that may enhance or limit college and career readiness.

Recent efforts to enhance CTE, including those underway in NYC, wisely focus on such key elements as rigorous and relevant CTE course sequences, robust work-based learning opportunities, and articulated partnerships with employers and post-secondary education institutions. The findings from this study point to additional conditions that are likely to interact with these curricular and co-curricular elements of CTE—such as providing students with smaller, more personalized learning environments; using inclusive (less selective) admissions policies; and aligning high school requirements with post-secondary options. It will be crucial for policymakers to attend to these conditions as they work to strengthen students’ pathways into college and careers.

Finally, it is important to note that we do not yet have all the information needed to fully discern the impact of NYC’s diverse CTE options. Data on employment and earnings will be crucial to understanding whether students in these schools opted to enter the workforce instead of, or prior to, enrolling in college—and how these decisions affected their longer-term trajectories.


This blog was produced by Corinne Alfeld (Corinne.Alfeld@ed.gov), program officer, NCER.

 

CTE Research Through an Equity Lens

This image depicts six considerations for centering equity in CTE research:  Ensure transparency: Be clear about the why, the what, and the who Involve the community: Obtain feedback from research participants throughout the process Develop diverse teams: Ensure teams represent varied perspectives and are trained in equity-based research perspective Take a systems approach: Be cognizant of historical issues of inequity within vocational education Acknowledge and attend to bias: Consider how bias is present in different parts of research Demonstrate respect: Bring an asset-based perspective

February is Career and Technical Education (CTE) month! As part of our 20th anniversary celebration, we want to highlight the great work our CTE Research Network (CTERN) continues to accomplish. The blog below highlights NCER’s conversation with the Equity Working Group of the IES-funded CTE Research Network

The Equity Working Group (EWG) of the CTE Research Network (CTERN) has published a new resource for researchers on using an equity lens in developing and conducting CTE research: The Equity Framework for CTE Research. CTERN is hosting a free webinar on February 21st at 3:00 Eastern to provide an overview of the framework and how people can use it. In this blog, members of the Equity Working Group answered questions about the framework and why it is important. 

The framework has a focus on equity, but equity can mean different things to different people. How does the EWG define equity in this framework?

We strongly believe that every student should have the opportunity to engage in quality educational experiences. Students who are interested should have access to CTE programs, regardless of their background characteristics. And school systems should invest in students so that they can succeed in these programs. Ultimately, we find ourselves quoting the Wisconsin Department of Public Instruction’s definition because it neatly captures our position: “Every student has access to the educational resources and rigor they need at the right moment in their education across race, gender, ethnicity, language, disability, sexual orientation, family background, and/or family income.”

Why did the EWG members believe that there was a need for an equity framework for CTE research?

CTE has a long and complicated history, including extensive tracking under its previous incarnation as vocational education. The CTE Equity Working Group was very conscious of this history and wanted to take steps to help ensure that CTE research was helping to ameliorate current inequities. As we say in the framework, “We believe that infusing equity throughout our research is critical to ensuring that research can make a difference in promoting equitable learning experiences and outcomes for all students who participate in CTE.”

We also recognized that many researchers (including ourselves) want to use an equity lens to do their research but lack practical guidance in what that looks like. The working group believed that a framework with concrete examples and tips would help CTE researchers have a clearer picture of what to do and would provide a tool for helping them think differently about their work.

How did the EWG create the framework?

This was a collaborative process that grew out of our first CTE Research Network meeting in 2018 or 2019. A group of us realized that incorporating an equity lens into our work would help us better answer questions that matter to communities. We decided to form a working group, which ended up including around 20 or so researchers, practitioners, and policy staff. We read a lot of good frameworks from different organizations on improving our research practices, so we decided to invest our energy in seeing how it may be applied to a CTE context.

How is the framework structured and what are some key takeaways?

It is important to note what this framework is and is not. This framework is not intended as a methodological primer or a replication of existing research guidance; it is intended to encourage researchers to think about their own work through an equity lens.

The framework starts with a brief history of equity in CTE, a description of the process of creating the framework, a list of vocabulary (we believe having a common language is critical), and a statement of the values that underlie the framework.

The rest of the framework is then organized by six stages of research: 1) project management; 2) research design, 3) measurement and data collection, 4) data analysis, 5) cost and resource equity, and 6) reporting and dissemination. In each section, we include a description of how to implement the stage with an equity-focused lens, with questions for researchers to consider and potential barriers. Throughout, we have included examples from current and future CTE research. We are looking for more examples, so people should feel free to reach out to us at jedmunds@serve.org to share how they are doing this work.

In creating summary products to go along with the framework, we identified six themes that cut across the different stages: ensure transparency, involve the community, develop diverse teams, take a systems approach, acknowledge and attend to bias, and demonstrate respect. These themes are summarized in an infographic.

How do you hope that people will use the framework?

We hope this will help start or further conversations among CTE researchers. We structured the framework around each stage of the research process, so anyone engaging in this work can find elements to incorporate or questions to consider individually and as a team, regardless of where they are in their work right now. For studies just getting off the ground, we did our best to illustrate how researchers can build an equity approach from the start of a project through its completion.

What are some examples of how the framework changed individual EWG members’ research practices?

Julie A. Edmunds (co-facilitator): Working on the framework has crystallized three high-impact equity-focused practices that I now try to infuse throughout my work. First, I pay much more attention to the role of systems in inequities. I try to look at upstream factors that might be causing disparities in educational outcomes as opposed to just documenting gaps that might exist between sub-groups. Second, when presenting those gaps (which we still do because it is useful information), I am much more conscious about how those gaps are displayed. For example, we focus on making sure that “White” is not considered the default category against which all others are compared. Third, we are creating processes to ensure that we share our findings with people who gave us the data. For example, we are sending practitioner-friendly products (such as briefs or infographics) to the school staff we interviewed whose insights formed the basis for some of our findings.

John Sludden (member): The framework has helped us think about our internal processes and keeps us focused on our audience, who we’re doing this for. I’m an analyst on the project, and I’ve been empowered to ask questions, conduct analyses, and present to our research partners at the New York City Department of Education. We’re currently thinking about ways to communicate findings to different audiences. At the moment, we’re working on a plan to share findings with principals of CTE high schools in New York City. Organizationally, we are also working on ways to directly engage students in the city, who know more about the system than we ever will. Similar to Julie, analytically, we have spent a lot of our time and attention on looking at the conditions under which students have not been well-served by the system, and ways that students may be better served by CTE.


This blog was produced by Corinne Alfeld (Corinne.Alfeld@ed.gov), program officer, NCER.

Career and Technical Education in STEM for Students with Learning Disabilities: Research Updates and Implications

Career and Technical Education (CTE) Month® is celebrated every February to raise awareness about the role that CTE has in preparing students for college and career success and the achievements of CTE programs across the country. In recognition of this year’s CTE Month®, we caught up with Dr. Michael Gottfried, University of Pennsylvania, to discuss his CTE research.

Through NCSER’s Career and Technical Education for Students with Disabilities special topic area, Dr. Gottfried was awarded a grant to examine whether participating in STEM CTE courses in high school is related to pursuing and persisting in STEM majors and/or careers for students with learning disabilities (SWLDs), a project featured initially in a March 2020 blog. During our recent conversation, he shared updates with us about his CTE project as well as the policy and practice implications of his research.

When we discussed this project in 2020, you shared your research goals and what you had learned so far. Could you provide us with an update?

Since the last time that we chatted, we have made some great progress on this project. We have had several papers accepted for publication. In some of our work, we were interested in the STEM CTE coursetaking patterns of SWLDs in high school. We found that SWLDs are more likely to participate in CTE courses compared to students without disabilities. Yet, when looking at the specific category of STEM CTE courses, there is no evidence that SWLDs are more likely to participate in high school STEM CTE courses compared to students without disabilities.

We have also looked at specific outcomes for SWLDs in STEM CTE courses. For instance, we examined computer science STEM CTE coursetaking for SWLDs. Participation was associated with growth in STEM self-efficacy and STEM utility (usefulness of what is learned for practical application) for SWLDs, whereas it related to positive development of STEM self-efficacy and STEM identity, but not STEM utility, for students without learning disabilities.

After we discovered that little was known about the association between STEM CTE coursetaking and college STEM persistence for SWLDs, we wanted to explore this area. So far, we have found that SWLDs who earned more units of STEM CTE in high school were more likely to seriously consider and ultimately declare STEM majors in college that are related to high school STEM CTE courses, such as information technology or engineering technology.

You and your colleagues recently published a paper in Education Research based on your NCSER-funded research. Could you summarize the findings in this paper and the implications for policy and practice?  

In our paper, we set out to identify whether there were any observable changes in CTE participation over time. The unique aspect of this study was that it combined national data from the Education Longitudinal Study of 2002 and the High School Longitudinal Study of 2009 with administrative data from the state of Washington. Key findings indicated that CTE participation declined nationally between the graduating class of 2004 and the graduating class of 2009 except in the area of applied science, technology, engineering, mathematics, and medical/health (STEMM) CTE, which includes courses in information technology, engineering technology, and health sciences. Data from Washington tended to be less varied in nature compared to national data, with fewer discernible trends, though in general STEM CTE did appear to have an upward trend for all students.

Our work also has direct relevance to policy. Recent changes in the Carl D. Perkins Career and Technical Education for the 21st Century Act emphasized the need to focus on increasing access and participation in STEM-related CTE coursework. While there does appear to be an upward trend in participation in these STEM fields, the tradeoff may be coming at the expense of other CTE areas of study. Combined with the increasing pressure for students to complete more and more academic coursework in a push for college readiness, this decrease in non-STEM CTE participation is particularly noteworthy. Finally, our work helps highlight the importance of examining CTE trends at the state and national levels. Different states have different needs and different graduation requirements that may or may not include CTE participation. As such, given the overall call to increase CTE participation for SWLDs, we encourage future research that explores the implications of these trends for this population.

What do you hope that school leaders, CTE teachers, and students will learn from all the research you are conducting?

The research has numerous implications for policy and practice. First, the results will be important for policymakers as they consider new or revised educational policies to support the pursuance and persistence of SWLDs into STEM fields. Education policymakers in particular need to understand the effects of STEM CTE coursetaking for SWLDs at multiple time points (transition into college, during college, and post-college). Understanding these issues more completely will make for well-informed policy decisions that promote short- and long-term success in STEM for SWLDs. This, in turn, has larger social policy implications with respect to upward mobility and lifelong success.

This project also has important implications for practice. By sharing these results, we hope to support education practitioners in making the adjustments necessary to improve the use of educational resources to ensure that SWLDs are prepared for and engaged in fields with high growth potential. For instance, many states have begun to accept STEM CTE courses for graduation requirements, which increases the likelihood students will take these courses. As STEM CTE courses prove important for SWLDs across the pipeline, then states and districts might consider how to best encourage students to take and succeed in these courses.

What additional research is needed to improve CTE policy and practice?

The current work can inform the future development of an intervention, assessment, or decision to evaluate an intervention. Evidence that STEM CTE coursetaking is associated with higher likelihood of college enrollment and the pursuit of STEM pathways for SWLDs supports the need to study interventions that encourage STEM CTE coursetaking for these students. For example, a randomly selected set of SWLDs who do not take traditional STEM could be counseled into taking STEM CTE courses or placement tests could be used to assign students to STEM CTE or traditional STEM courses. In both cases, students could then be followed into college and beyond to compare education and career outcomes using rigorous research designs. The results could provide additional, strong evidence for the value of STEM CTE coursetaking on postsecondary STEM outcomes.

This blog was authored by Akilah Nelson (akilah.nelson@ed.gov), Program Officer at NCSER, and Michael Gottfried (mgottfr2@upenn.edu), Associate Professor at the University of Pennsylvania.