Inside IES Research

Notes from NCER & NCSER

Partnering with Researchers Can Help State Leaders Build the Case for CTE

In Massachusetts, Career/Vocational Technical Education Schools (CVTE) are renowned for offering rigorous, high-quality programs of study across a variety of disciplines. While CVTE graduates have always experienced high rates of success academically and in their careers, state leaders in Massachusetts wanted to know whether these outcomes directly result from the CVTE model. In 2017, the Massachusetts Department of Elementary and Secondary Education partnered with Shaun Dougherty (at the time, a researcher at the University of Connecticut), and learned that CVTE students are significantly more likely to graduate from high school and earn an industry-recognized credential than similar students who were not admitted.

Demand for rigorous research on Career Technical Education (CTE) has increased as more policymakers ask questions about the impact on college and career readiness. State CTE Directors may be interested in similar questions as researchers (such as “Does CTE improve educational and career outcomes? Do different programs help different students? What types of programs offer students the highest economic returns?”) but may not think to seek out and collaborate with them or know how to prioritize among the many research requests they receive.

This blog series, a partnership between Advance CTE and the Institute for Education Sciences (IES) seeks to break down the barriers between State CTE Directors and researchers to encourage partnerships that can benefit both.

What Can Research with State Data Tell Us?

Research can be a powerful tool to help State CTE Directors understand what’s working, what isn’t working, and what needs to change. The findings described below provide examples of how strong partnerships between researchers and state policymakers can result in actionable research (click on state name for link to full article).

  • In Arkansas, students with greater exposure to CTE are more likely to graduate from high school, enroll in a two-year college, be employed, and earn higher wages. The study, which was rigorous but not causal, also found that students taking more CTE classes are just as likely to pursue a four-year degree as their peers, and that CTE provides the greatest boost to boys and students from low-income families.
  • Boys who attended CTE high schools in Connecticut experienced higher graduation rates and post-graduation earnings than similar students who did not attend CTE high schools. Further follow-ups using both postsecondary and labor data could provide information about college completion and employment and earnings for different occupational sectors.
  • CTE concentrators in Texas had greater enrollment and persistence in college than their peers. Although rates of CTE concentration decreased, student participation in at least some CTE programming, as well as number of CTE credits earned, increased between the 2008 and 2014 cohorts. Unsurprisingly, the study also found differences by CTE programs of study. Education & Training; Finance; Health Science; and Science, Technology, Engineering & Mathematics (STEM) were most strongly associated with postsecondary enrollment, particularly in baccalaureate programs.

How Can States Use CTE Research to Improve Policy and Practice?

Here are a few things states can do today to start building a CTE research base:

  • Create a codebook of CTE variables in your state’s data system: Include K-12, postsecondary, and labor force variables if you have them. Define the variables clearly – what do they measure, at what level (student, program, district), and for how many years did you collect these variables? Are the measures comparable across years and across datasets?
  • Maximize opportunities to collect longitudinal data: longitudinal databases that span education levels and connect to workforce outcomes permit researchers to conduct rigorous studies on long-term outcomes.
  • Identify universities in your state with strong education, economics, or public policy departments:  Make a list of questions that policymakers in your state most wanted answered, and then approach universities with these proactively. Reach out to the chair(s) of these departments to connect with faculty who may be interested in partnering on answering the questions. Universities can often apply for a research grant that will cover part or all of the funding for state personnel to work on the research project. IES, which provides funding of this nature, opens its next grant competition in summer 2020.
  • Reach out to your Regional Educational Lab (REL) or the REL Career Readiness Research Alliance to inquire about partnering on CTE research: The mission of these IES-funded labs is to provide research and evidence to help educators in the states in their region. For example, REL Central is currently working with four states to replicate the Arkansas study described above (see “Review of Career and Technical Education in Four States”).
  • Stay up to date on the latest research findings in CTE: New research is regularly posted on the CTE Research Network and other websites. This can help you get ideas for what types of research you would like to conduct in your state. Another good source of inspiration is the recommendations of the CTE technical workgroup, which was convened by IES in late 2017 to guide future CTE research directions.
  • Become familiar with how researchers approach CTE research: Learn about why it’s so challenging to understand its impact. The CTE Research Network will hold research trainings for different audiences—including state agency staff— beginning in the summer of 2020. Stay tuned!

Over the next several months, Advance CTE and IES will publish a series of Q&A blog posts with researchers and state CTE leaders talking about how their partnerships developed and what states can do to advance CTE research.

This blog series was co-authored by Corinne Alfeld at IES (corinne.alfeld@ed.gov) and Austin Estes from Advance CTE (aestes@careertech.org), with thanks to Steve Klein of Education Northwest for editorial suggestions. IES began funding research grants in CTE in 2017 and established a CTE Research Network in 2018. IES hopes to encourage more research on CTE in the coming years in order to increase the evidence base and guide program and policy decisions. At the same time, Advance CTE has been providing resources to help states improve their CTE data quality and use data more effectively to improve CTE program quality and equity.

Updates from the CTE Research Network!

“Does Career and Technical Education (CTE) work?” and “For whom does CTE work and how?” are questions on many policymakers’ and education leaders’ minds and ones that the CTE Research Network aims to answer. The mission of the Network, as described in a previous blog post, is to increase the amount of causal evidence in CTE that can inform practice and policy. The Network’s members, who are researchers funded by IES to examine the impact of CTE, have been busy trying to answer all of these questions.

This blog describes three Network updates:

  • Shaun Dougherty, of Vanderbilt University, and his colleagues at the University of Connecticut have been studying the effects of attending a CTE-focused high school among 60,000 students in Connecticut as part of their Network project. They recently reported that:
    • When compared to males attending traditional high schools, males who attended CTE schools were 10 percentage points more likely to graduate from high school and were earning 31 percent more by age 23. The authors noted that the more CTE courses that are available at the regular high school, the less attendance at a CTE high school makes a difference.
    • Analyses of potential mechanisms behind these findings reveal that male students attending a technical high school have higher 9th grade attendance rates and higher 10th grade test scores. However, they are 8 percentage points less likely to attend college (though some evidence indicates that the negative impact on college attendance fades over time).
    • Attending a CTE high school had no impacts on female students. Further, the effects did not differ over student attributes like race and ethnicity, free lunch eligibility or residence in a poor, central city school district.

The study results are being disseminated widely in the media, including via the Brookings Brown Center Chalkboard, The Conversation, and the National Bureau of Economic Research.

  • In other news, the CTE Research Network has welcomed a fourth IES-funded project, led by Julie Edmunds. Edmunds’ team is studying dual enrollment pathways in North Carolina, and one of the pathways focuses on CTE.
  • Finally, the two co-PIs for the Network Lead, Kathy Hughes and Shaun Dougherty, recently participated in a Q&A in Techniques magazine about the purpose of the CTE Network, how the Network will help the field of CTE, and how each of their careers has led them to this work.

The Network Lead has launched a new website where you can get new information about ongoing work and sign up to receive their newsletter.

This post was written by Corinne Alfeld, the NCER-IES program officer responsible for the CTE research topic and the CTE Research Network. Contact her at Corinne.Alfeld@ed.gov with questions.

Career Pathways Research at HHS: Lessons and Opportunities for Education Research

Over the past few years, staff from IES and the Office of Planning, Research, and Evaluation (OPRE) in the Administration for Children and Families (ACF), U.S. Department of Health and Human Services (HHS), have been learning from and supporting one another’s work. Our offices have a shared interest in understanding and improving outcome for adults in postsecondary career pathway programs and for creating a strong evidence base.

We have even funded projects that dovetail nicely. For example, IES funded a development project focusing on Year Up, and OPRE included Year Up programs in their Pathways for Advancing Careers and Education (PACE) Study.

For IES researchers not already aware of OPRE’s research, we would like to highlight three things that may be particularly relevant to the work IES hopes to support.

A Growing Portfolio of Career Pathway Research: For over a decade, OPRE has created a robust research portfolio longitudinal, rigorous experimental career pathways program research.  These career pathway programs provide postsecondary education and training through a series of manageable steps leading to successively higher credentials and employment opportunities. In particular, OPRE has supported the Pathways for Advancing Careers and Education (PACE) Study and the rigorous evaluation of the Health Profession Opportunity Grants (HPOG) Program. You can find reports from these and other self-sufficiency, welfare, and employment activities in the OPRE Resource Library.

Available Data and Funding Opportunity: Later this summer, OPRE will be archiving data from PACE and HPOG at the University of Michigan’s Inter-University Consortium on Political and Social Research (ICPSR) at the Institute for Social Research. These data will be available as restricted-use files for secondary data analysis. To encourage research, OPRE announced a funding opportunity Career Pathways Secondary Data Analysis Grants to support secondary analysis these data. Applications are due August 16, 2019.

Understanding Programs’ Motivations for Participating in Research: Getting education programs (schools, universities, etc.) to join multi-year, randomized controlled trials is difficult. Programs are wary of random assignment or finding null or negative effects. Yet, the programs that participated in the PACE Study were overall quite supportive. A recent report “We Get a Chance to Show Impact”, Program Staff Reflect on Participating in a Rigorous, Multi-site Evaluation documents the hurdles and benefits of participation from a program’s point of view. These programs’ insights are particularly useful for any researcher hoping to form partnerships with education settings.

To learn more about the ongoing career pathways research at OPRE and their findings, please visit https://www.acf.hhs.gov/opre/research/project/career-pathways-research-portfolio.

IES is Expanding the Evidence Base for Career and Technical Education (CTE)

Since February is Career and Technical Education (CTE) Month—let’s look at what is going on in CTE training and research. Formerly known as “vocational education,” CTE generally comprises instruction in the academic, technical, and employability skills and knowledge required to enter into and succeed in specific occupations. CTE can introduce high school students to different career paths and help them build marketable skills or even credentials. For college students, CTE offers an entry point for new and returning students as they gain knowledge and skills in certain occupational fields.

Many policymakers consider CTE to be a key aspect of “college and career readiness.” In 2017, 49 states enacted 241 CTE policies, and 42 states enacted an additional 146 CTE policies in 2018. However, CTE practice and policy are way ahead of research—particularly in terms of research that can more definitively link CTE to specific outcomes and impacts. Over the past few years, IES has made some important strides in this area.

Following a Technical Working Group meeting on the future of CTE research, IES partnered with the Office of Career Technical and Adult Education to launch a new research network called “Expanding the Evidence Base for Career and Technical Education (CTE).” 

 

 

The CTE research network is a five-year grant, to be led by the American Institutes for Research (AIR), with partners at Vanderbilt University, Jobs for the Future, and the Association for Career and Technical Education (ACTE). Currently, three IES research projects have joined the new network, and we hope to eventually include up to six. All of these projects will look at the causal impact of CTE on student outcomes.

Throughout the five-year grant period, the Network Lead will bring together project teams and help to provide vision and support to the research projects as a whole. The Network Lead will also conduct research, provide CTE research training activities, and work to disseminate the Network’s research products so that they can reach the widest possible stakeholder audience. 

The first meeting of the Network members occurred on January 8, 2019 in Washington, DC, where members began to set the priorities for collaborative work across projects. Network members agreed that it is critical for all research projects to provide detailed data broken out by CTE field and by student subgroups, including students with disabilities.

One key priority of the Network is to develop a working definition of CTE for research purposes (i.e., how to define a CTE student and how to measure CTE participation). A related priority is to identify or develop appropriate measures of CTE participation and outcomes that network members, as well as other CTE researchers, can use. Over the course of the grant, network members will have the opportunity to collaborate on a variety of activities.

We will be reporting on the Network’s progress periodically on this blog, but readers are also encouraged to visit the CTE Research Network website, housed by AIR.

 

Blog post by Corinne Alfeld, program officer in the IES National Center for Education Research (NCER)

For more information about the CTE research network, contact corinne.alfeld@ed.gov. Corinne is also the program officer for NCER’s CTE research topic, which will be accepting grant applications for all types of CTE research later in 2019! (Note that funded studies designed to measure the causal impact of CTE programs or policies may be eligible to join the CTE research network in future years). Sign up for the IES Newsflash to be notified when the NCER Requests for Applications are released.

Informing Future Research in Career and Technical Education (CTE)

Career and Technical Education (CTE) has been evolving and expanding at a rapid pace in recent years as industry and education leaders focus on students’ readiness for college and careers. While some studies have shown positive effects of CTE on students, the evidence base is thin. To learn more about the research needs of the CTE field, the National Center for Special Education Research (NCSER) and the National Center for Education Research (NCER) at the Institute of Education Sciences (IES) convened a group of experts in policy, practice and research related to CTE.  The discussion held by the Technical Working Group (TWG) led both NCER and NCSER to increase their investments in CTE for fiscal year 2019: NCSER included a CTE special topic, and NCER changed its CTE topic from a special topic to a standing topic. Applications to both are due August 23, 2018.  Both research centers hope to fund more studies that will help us better understand this growing aspect of education.

The TWG focused on the following four questions:

  1. Who is served by CTE and who is left behind? From national CTE statistics, we know that 82% of all public high schools offer CTE. And, 85% of students earn at least one credit in CTE with the average high school student earning 2.5 CTE credits. However, TWG members noted that research is lacking on specific subpopulations in CTE, such as students from various demographic backgrounds and students with disabilities. Disaggregated data on these dimensions are needed to better understand the CTE experiences of the range of students being served. Such data may help educators improve equity of access to high quality programs for all students.
  2. What do we know―and need to know―about CTE policies, programs, and practices at the secondary and postsecondary levels?  TWG experts discussed the need to know more about industry-recognized credentials and about business and industry engagement in CTE at the secondary level. They argued that we do not know if credentials align with industry requirements, nor do we understand the impact of different types of credentials on student outcomes and wage trajectories. TWG members also noted that the higher the perceived quality or prestige of the CTE program, the more exclusive it becomes, and the more difficult it is for disadvantaged students to obtain access. TWG members also expressed concerns about CTE teacher training, particularly for experts who are recruited from industry without prior teacher preparation. As the experts discussed postsecondary CTE, they suggested that the field would be best served by framing the conversation about secondary to postsecondary pathways as a continuum that enables transparent and sequential transitions from secondary to 2-year and then to 4-year programs or to training or employment, with guidance for students to understand possible sequences.
  3. What are the critical methodological issues in CTE?  TWG members noted that, with a few notable exceptions (e.g.,  a 2008  MDRC study on career academies in New York and a recent study of CTE high schools in Massachusetts), few causal studies on CTE have been conducted. There is an urgent need for more high quality, causal research on CTE policies and programs. In addition, the experts noted that there is almost no research on students with disabilities in CTE. TWG members concluded that the field needs to re-conceptualize CTE research – including better defining CTE students, instructors, programs, and measures – and identify the critical research questions in order to encourage more research in this field.
  4. What is needed to advance CTE research?  State CTE administrators want to know how to identify quality CTE programs so they know how to spend their dollars most effectively on programs that best meet the needs of students. Policymakers also want to know what “works” and what the benefits are of such investments. The TWG members encouraged studies that examine the educational benefits of particular instructional approaches. They also highlighted the importance of collaborative cross-institutional and cross-agency efforts to advance CTE research.

Readers are invited to read the summary of the TWG discussion.

By Corinne Alfeld (NCER program officer) and Kimberley Sprague (former NCSER program officer)