IES Blog

In honor of CTE month, we wanted to provide more information about the recent additions to NCER’s CTE research portfolio: 7 new grants awarded across 2 grant programs in FY2021. A previous blog last summer announced these grants; this blog briefly describes each project (the hyperlinks will take you to the full online project abstract). Be sure to read to the end for links to other CTE work across IES!

Within the Education Research Grants program (305A), the following projects were funded in 2021:

College and Career Readiness: Investigating California's Efforts to Expand Career Technical Education Through Dual Enrollment

(PI: Michal Kurlaender, University of California, Davis)

This project is examining the result of policy changes due to California Assembly Bill 288 (AB288), enacted in 2015 to create the College and Career Access Pathways (CCAP) partnership. CCAP increased the prominence of Career and Technical Education (CTE) at the high school level by allowing high schools and community colleges to enter joint partnerships and offer dual enrollment courses that count towards both a high school diploma and an associate degree from California community colleges. 

Postsecondary and Labor Market Effects of Career and Technical Education in Baltimore City Public Schools

(PI: Marc Stein, Johns Hopkins University)

This project uses a unique selection process into CTE Centers within a large school district, linked with longitudinal state data, to provide strong evidence on the benefits and mechanisms of CTE participation on secondary education, postsecondary education, and labor market outcomes.

SREB Career and Technical Education Leadership Academy Study

(PI: James Stone, Southern Regional Education Board)

This project is developing, piloting, and studying the promise of the Southern Regional Education Board (SREB) Career and Technical Education Leadership Academy to increase the capacity of school leaders at career and technology centers to work with their teachers as instructional leaders and thereby improve CTE outcomes for their students.

An Experimental Evaluation of the Efficacy of Virtual Enterprises

(Co-PIs: Fatih Unlu, RAND Corporation and Kathy Hughes, AIR)

In this project, the research team will provide the first causal evidence on the efficacy and cost-effectiveness of Virtual Enterprises, a virtual school-based enterprise (SBE) program. SBEs are one form of work-based learning (WBL) in which students run a business that produces and sells goods or services. SBEs may offer unique benefits relative to other types of WBL by providing opportunities for more students to participate, reducing the need for transportation, and allowing students more room to make and learn from their mistakes. Note: Since this is an efficacy (causal impact) study, it has joined the CTE Research Network.

Sub-baccalaureate Career and Technical Education: A Study of Institutional Practices, Labor Market Demand, and Student Outcomes in Florida

(PI: Angela Estacion, WestEd)

The purpose of this project is to address existing policy and research gaps by, first, administering a statewide survey to catalogue the institutional practices that Florida community and technical colleges use to align CTE programming to the labor market. Second, by combining the survey data with student-level program participation and outcome data, the project team will ascertain the degree to which institutional practices and labor market conditions in students' geographical areas are correlated with students' choices and outcomes. Finally, the project team will analyze qualitative data collected from case studies of Florida community and technical colleges to describe the practices cited in the survey data and understand the process of aligning courses and programs with local labor market demand.

Within the “Using SLDS to Support State Education Policymaking” (305S) grant program, the following projects were funded in 2021:

The Distributional Effects of Secondary Career and Technical Educational (CTE) Programs on Postsecondary Educational and Employment Outcomes: An Evaluation of Delaware's CTE Programs of Study

(PI: Luke Rhine, Delaware Department of Education)

The Delaware Department of Education and University of Delaware is examining variability in participation rates among student subgroups in Delaware public high school CTE programs and link CTE high school participation to high school graduation, postsecondary enrollment, employment, and wages. Stay tuned to delawarepathways.org for more information as the project unfolds.

Analyzing and Understanding the Educational and Economic Impact of Regional Career Pathways

(PI: Jonathan Attridge, Tennessee)

The research team is conducting an evaluation of Tennessee Pathways (along with earlier career pathway programs), a state initiative to align K-12 education, postsecondary education, and employers so that high school students have a clear pathway to move into the workforce.

For a full list of CTE-related grants funded by NCER and NCSER across years, topics, and grant competitions, you can explore our “funded projects” search pages for NCER or NCSER. Here is a recent blog post about students with disabilities in CTE from NCSER. And don’t forget to visit the CTE Research Network frequently for many new CTE-related findings and resources!

NCEE and NCES have published multiple reports on CTE, including Career and Technical Education Credentials in Virginia High Schools: Trends in Attainment and College Enrollment Outcomes, as well as some great blogs such as this one on exploring the growing impact of career pathways from NCEE. You can see more on career readiness from NCEE here and from NCES here.

We are so pleased with the growth of this portfolio since our first call for more CTE research 5 years ago! However, there continues to be a need to better understand CTE. For instance, research is still needed on CTE measures and assessments, quality of programs (content, instruction, and opportunities for WBL), and variation in impacts across student subgroups and career clusters.

For more information about CTE research grants, including feedback on new project proposal ideas, please contact NCER program officer Corinne.Alfeld@ed.gov.

Since its inception in 2017, the CTE portfolio in the National Center for Education Research (NCER) at IES has grown to 11 research grants and a research network! Several other CTE-related grants have been funded under other topics, such as “Postsecondary/Adult Education” and “Improving Education Systems” in the education research grants program, and in other grant programs such as “Using SLDS to Support State Policymaking.” Two CTE-related grants under the latter program were awarded in FY21—

• The Distributional Effects of Secondary Career and Technical Educational (CTE) Programs on Postsecondary Educational and Employment Outcomes: An Evaluation of Delaware's CTE Programs of Study (PI: Luke Rhine, Delaware)
• Analyzing and Understanding the Educational and Economic Impact of Regional Career Pathways (PI: Jonathon Attridge, Tennessee)

The newest grants funded in FY21 in the CTE topic of the Education Research Grants program include—

• College and Career Readiness:  Investigating California's Efforts to Expand Career Technical Education Through Dual Enrollment (PI: Michal Kurlaender, University of California at Davis) 
• Postsecondary and Labor Market Effects of Career and Technical Education in Baltimore City Public Schools (PI: Marc Stein, Johns Hopkins University) 
• SREB Career and Technical Education Leadership Academy Study (PI: James Stone, SREB)
• Sub-baccalaureate Career and Technical Education: A Study of Institutional Practices, Labor Market Demand, and Student Outcomes in Florida (PI: Angela Estacion, WestEd)
• An Experimental Evaluation of the Efficacy of Virtual Enterprises (PI: Fatih Unlu, RAND) 

As a causal impact study, the last project (on Virtual Enterprises) has been invited to join NCER’s CTE Research Network as its sixth and final member. Funded in 2018 to expand the evidence base for CTE, the CTE Research Network (led by PI Kathy Hughes at the American Institutes for Research) includes five other CTE impact studies (one project’s interim report by MDRC was recently reviewed by the What Works Clearinghouse and was found to meet standards without reservations). You can read more about the network’s mission and each of its member projects here.  

On AIR’s CTE Research Network website, you can find several new resources and reports, such as: 

• Ready for Causal Research: A National Evaluability Assessment of Career and Technical Education Programs (Final Report). This report describes four CTE programs identified as most immediately viable for causal research—and all these programs are willing to be studied!  
• Identification and Counterfactuals for Program Evaluation of Career and Technical Education. Identifying a control group, or counterfactual, is tricky in a subject like CTE into which students self-select. To guide researchers, this paper presents a series of case studies assessing the counterfactual for a selection of experimental and quasi-experimental CTE program evaluations. 
• Improving Measurement in Career and Technical Education to Support Rigorous Research. This paper catalogs the measures of CTE exposure that prior studies have used, documents measures and outcomes in more recent causal studies, and offers a framework for how to measure CTE exposure and outcomes in the future. 

The CTE Research Network has also been conducting training, including workshops in causal design for CTE researchers and online modules on data and research for CTE practitioners, shared widely with the field by a Network Lead partner, the Association for Career and Technical Education (ACTE). 

Last but certainly not least, if you are interested in getting your CTE project funded by IES, see the new FY22 research grant opportunities on the IES funding page. To apply to the CTE topic in the Education Research Grants program specifically, click on the PDF Request for Applications (ALN 84.305A). Contact Corinne Alfeld with any questions you might have.

Written by Corinne Alfeld (Corinne.Alfeld@ed.gov), NCER Program Officer 

In celebration of CTE^® (career and technical education) month, we would like to highlight the launch of an NCER project that aims to help us understand how to best support adults seeking additional CTE education and training.

Through their exploratory project, Literacy, Numeracy, and Problem-Solving Skills in Technology-Rich Environment in the STEM-Related Subbaccalaureate Programs in the United States, researchers will use a mixed-method design to gather information about the distribution of basic skills (literacy, numeracy, and problem solving) for adults in STEM occupations and students enrolled in STEM-related sub-baccalaureate programs at community colleges. Their goal is to help identify the needs of students and the programming practices at community colleges that may promote basic skill development in STEM programs.

The team will be leveraging data from the Program for the International Assessment of Adult Competencies (PIAAC) to survey the distribution of skills and abilities across a nationally representative sample of adults in STEM fields. They will also be collecting primary data from adults and programs in multiple locations, including Indiana, Ohio, and Washington states.

To help inform the public about their project, the researchers have created a short YouTube video for the public. This project began in July 2020 and may have initial results ready as early as late 2021.

Written by Meredith Larson (Meredith.Larson@ed.gov), Program Officer for Postsecondary and Adult Education, NCER.

The Career and Technical Education (CTE) Research Network just entered its third year, and it is time to share recent accomplishments with the IES community! As a reminder, the CTE Research Network (led by the American Institutes for Research, or AIR) was created to expand the evidence base on the impact of CTE programs on student outcomes using causal research methods.

Research

In June 2020, AIR released a preliminary report of CTE sites that are ready for causal evaluation. Designed to support researchers interested in studying the impact of CTE on student outcomes, the report details the history, theory of change, student enrollments, and other information for 4 selected CTE sites around the country. The Network hopes that researchers will use the information in the report to design evaluation studies of these programs.

In July 2020, a fifth research project joined the network. A team from MDRC, led by Rachel Rosen, was recently funded by IES to study the Impact of Technology-Based Career Advising Tools on High School Students' CTE Choices and Academic Performance. In partnership with Communities in Schools (CIS), the study will use a three-arm, school-level random assignment research design (RCT) to assess the effects of Navience and YouScience on students' self-expressed attitude and interest in career pathways, CTE course taking patterns, and engagement with and progress towards graduation. We welcome the team to the Network and look forward to learning whether and how these advising tools influence student thinking about career options, choice of relevant CTE coursework and work-based learning options, and decisions about CTE concentration in available pathways and programs of study.

A small group of researchers from different Network teams collaborated on and recently released a technical working paper on counterfactuals in CTE. It can be challenging to identify comparison groups for CTE students because it is an elective into which they self-select. The paper describes a variety of rigorous methods of comparing CTE students to valid counterparts and provides case studies that illustrate how to use these methods.

Training

The CTE Research Network is committed to increasing the number of researchers trained to study CTE using causal methods. It is notoriously challenging to isolate the effects of CTE from other influences on student outcomes. In August 2020, the Network hosted 18 researchers for a week-long virtual summer training institute on how to design a causal study to examine the impact of CTE. During the week, participants learned how to implement randomized-control trials (RCTs), regression discontinuity designs (RDDs), and comparative interrupted time series (CITS) in a CTE context. After learning about each method, participants worked in small groups to apply the method to real data and had access to the instructors to ask questions. The feedback about the training was overwhelmingly positive. The lecture portions of the training will be posted soon to the training page of the Network’s website. Another week-long training institute will be held in summer 2021 (hopefully, in person!)

The Network is currently developing a series of online modules for CTE practitioners and state agency staff to strengthen capacity to access, conduct, understand, and use CTE research. There will be a presentation to preview the modules at the Association of Career and Technical Education (ACTE) Best Practices and Innovations Conference on October 9, 2020 and a longer and more in-depth session at ACTE’s Career Tech VISION conference the first week of December. These practitioner training modules will be available for free on the website in late Fall 2020.

Leadership and Dissemination

The CTE Research Network is regularly updating its resources page with publications of interest to the CTE research field. The most recent is a report of findings from MDRC’s study of P-Tech high schools. The Network’s equity workgroup (a group of researchers from across the Network’s member projects) also published a popular blog this summer on applying an equity lens to CTE research. The Network also posts outside resources such as a REL self-study tool on career readiness and evaluation reports from other researchers.

For more about the CTE Research Network, you can sign up to receive the Network’s quarterly newsletter at the bottom of their website’s home page and follow them on Twitter (@CTEResNetwork) and LinkedIn.

If you are interested in learning more about the CTE Research Network, contact the Director, Kathy Hughes (khughes@air.org).

If you are interested in discussing CTE research opportunities at IES, contact Corinne Alfeld (Corinne.Alfeld@ed.gov).

In 2020, as part of a wider IES investment in resources around cost, IES funded the Cost Analysis in Practice (CAP) Project, a 3-year initiative to support researchers and practitioners who are planning or conducting a cost analysis of educational programs and practices. The CAP Project Help Desk provides free on-demand tools, guidance, and technical assistance, such as support with a cost analysis plan for a grant proposal. After inquiries are submitted to the Help Desk, a member of the CAP Project Team reaches out within two business days. Below is a list of resources that you can access to get more information about cost analysis.

STAGES FOR CONDUCTING A COST ANALYSIS

CAP Project Resources

Cost Analysis Standards and Guidelines 1.0: Practical guidelines for designing and executing cost analyses of educational programs.

IES 2021 RFAs Cost Analysis Requirements: Chart summarizing the CAP Project’s interpretation of the IES 2021 RFAs cost analysis requirements.

Cost Analysis Plan Checklist: Checklist for comprehensive cost analysis plans of educational programs and interventions.

Introduction to Cost Analysis: Video (17 mins).

General Cost Analysis Resources

The Critical Importance of Costs for Education Decisions: Background on cost analysis methods and applications.

Cost Analysis: A Starter Kit: An introduction to cost analysis concepts and steps.

CostOut®: Free IES-funded software to facilitate calculation of costs once you have your ingredients list, includes database of prices.

DecisionMaker®: Free software to facilitate evidence-based decision- making using a cost-utility framework.

Cost-Effectiveness Analysis of Early Reading Programs: A Demonstration With Recommendations for Future Research: Open access journal article.

*More resources available here.

The content for this blog has been adapted from the Cost Analysis in Practice Project informational flyer (CAP Project, 2020) provided by the CAP Project Team. To contact the CAP Help Desk for assistance, please go to https://capproject.org/. You can also find them on Twitter @The_CAP_Project.