IES Blog

Institute of Education Sciences

Partnering with Researchers Can Help State Leaders Build the Case for CTE

In Massachusetts, Career/Vocational Technical Education Schools (CVTE) are renowned for offering rigorous, high-quality programs of study across a variety of disciplines. While CVTE graduates have always experienced high rates of success academically and in their careers, state leaders in Massachusetts wanted to know whether these outcomes directly result from the CVTE model. In 2017, the Massachusetts Department of Elementary and Secondary Education partnered with Shaun Dougherty (at the time, a researcher at the University of Connecticut), and learned that CVTE students are significantly more likely to graduate from high school and earn an industry-recognized credential than similar students who were not admitted.

Demand for rigorous research on Career Technical Education (CTE) has increased as more policymakers ask questions about the impact on college and career readiness. State CTE Directors may be interested in similar questions as researchers (such as “Does CTE improve educational and career outcomes? Do different programs help different students? What types of programs offer students the highest economic returns?”) but may not think to seek out and collaborate with them or know how to prioritize among the many research requests they receive.

This blog series, a partnership between Advance CTE and the Institute for Education Sciences (IES) seeks to break down the barriers between State CTE Directors and researchers to encourage partnerships that can benefit both.

What Can Research with State Data Tell Us?

Research can be a powerful tool to help State CTE Directors understand what’s working, what isn’t working, and what needs to change. The findings described below provide examples of how strong partnerships between researchers and state policymakers can result in actionable research (click on state name for link to full article).

  • In Arkansas, students with greater exposure to CTE are more likely to graduate from high school, enroll in a two-year college, be employed, and earn higher wages. The study, which was rigorous but not causal, also found that students taking more CTE classes are just as likely to pursue a four-year degree as their peers, and that CTE provides the greatest boost to boys and students from low-income families.
  • Boys who attended CTE high schools in Connecticut experienced higher graduation rates and post-graduation earnings than similar students who did not attend CTE high schools. Further follow-ups using both postsecondary and labor data could provide information about college completion and employment and earnings for different occupational sectors.
  • CTE concentrators in Texas had greater enrollment and persistence in college than their peers. Although rates of CTE concentration decreased, student participation in at least some CTE programming, as well as number of CTE credits earned, increased between the 2008 and 2014 cohorts. Unsurprisingly, the study also found differences by CTE programs of study. Education & Training; Finance; Health Science; and Science, Technology, Engineering & Mathematics (STEM) were most strongly associated with postsecondary enrollment, particularly in baccalaureate programs.

How Can States Use CTE Research to Improve Policy and Practice?

Here are a few things states can do today to start building a CTE research base:

  • Create a codebook of CTE variables in your state’s data system: Include K-12, postsecondary, and labor force variables if you have them. Define the variables clearly – what do they measure, at what level (student, program, district), and for how many years did you collect these variables? Are the measures comparable across years and across datasets?
  • Maximize opportunities to collect longitudinal data: longitudinal databases that span education levels and connect to workforce outcomes permit researchers to conduct rigorous studies on long-term outcomes.
  • Identify universities in your state with strong education, economics, or public policy departments:  Make a list of questions that policymakers in your state most wanted answered, and then approach universities with these proactively. Reach out to the chair(s) of these departments to connect with faculty who may be interested in partnering on answering the questions. Universities can often apply for a research grant that will cover part or all of the funding for state personnel to work on the research project. IES, which provides funding of this nature, opens its next grant competition in summer 2020.
  • Reach out to your Regional Educational Lab (REL) or the REL Career Readiness Research Alliance to inquire about partnering on CTE research: The mission of these IES-funded labs is to provide research and evidence to help educators in the states in their region. For example, REL Central is currently working with four states to replicate the Arkansas study described above (see “Review of Career and Technical Education in Four States”).
  • Stay up to date on the latest research findings in CTE: New research is regularly posted on the CTE Research Network and other websites. This can help you get ideas for what types of research you would like to conduct in your state. Another good source of inspiration is the recommendations of the CTE technical workgroup, which was convened by IES in late 2017 to guide future CTE research directions.
  • Become familiar with how researchers approach CTE research: Learn about why it’s so challenging to understand its impact. The CTE Research Network will hold research trainings for different audiences—including state agency staff— beginning in the summer of 2020. Stay tuned!

Over the next several months, Advance CTE and IES will publish a series of Q&A blog posts with researchers and state CTE leaders talking about how their partnerships developed and what states can do to advance CTE research.

This blog series was co-authored by Corinne Alfeld at IES (corinne.alfeld@ed.gov) and Austin Estes from Advance CTE (aestes@careertech.org), with thanks to Steve Klein of Education Northwest for editorial suggestions. IES began funding research grants in CTE in 2017 and established a CTE Research Network in 2018. IES hopes to encourage more research on CTE in the coming years in order to increase the evidence base and guide program and policy decisions. At the same time, Advance CTE has been providing resources to help states improve their CTE data quality and use data more effectively to improve CTE program quality and equity.

Updates from the CTE Research Network!

“Does Career and Technical Education (CTE) work?” and “For whom does CTE work and how?” are questions on many policymakers’ and education leaders’ minds and ones that the CTE Research Network aims to answer. The mission of the Network, as described in a previous blog post, is to increase the amount of causal evidence in CTE that can inform practice and policy. The Network’s members, who are researchers funded by IES to examine the impact of CTE, have been busy trying to answer all of these questions.

This blog describes three Network updates:

  • Shaun Dougherty, of Vanderbilt University, and his colleagues at the University of Connecticut have been studying the effects of attending a CTE-focused high school among 60,000 students in Connecticut as part of their Network project. They recently reported that:
    • When compared to males attending traditional high schools, males who attended CTE schools were 10 percentage points more likely to graduate from high school and were earning 31 percent more by age 23. The authors noted that the more CTE courses that are available at the regular high school, the less attendance at a CTE high school makes a difference.
    • Analyses of potential mechanisms behind these findings reveal that male students attending a technical high school have higher 9th grade attendance rates and higher 10th grade test scores. However, they are 8 percentage points less likely to attend college (though some evidence indicates that the negative impact on college attendance fades over time).
    • Attending a CTE high school had no impacts on female students. Further, the effects did not differ over student attributes like race and ethnicity, free lunch eligibility or residence in a poor, central city school district.

The study results are being disseminated widely in the media, including via the Brookings Brown Center Chalkboard, The Conversation, and the National Bureau of Economic Research.

  • In other news, the CTE Research Network has welcomed a fourth IES-funded project, led by Julie Edmunds. Edmunds’ team is studying dual enrollment pathways in North Carolina, and one of the pathways focuses on CTE.
  • Finally, the two co-PIs for the Network Lead, Kathy Hughes and Shaun Dougherty, recently participated in a Q&A in Techniques magazine about the purpose of the CTE Network, how the Network will help the field of CTE, and how each of their careers has led them to this work.

The Network Lead has launched a new website where you can get new information about ongoing work and sign up to receive their newsletter.

This post was written by Corinne Alfeld, the NCER-IES program officer responsible for the CTE research topic and the CTE Research Network. Contact her at Corinne.Alfeld@ed.gov with questions.

The Role of RELs in Making WWC Practice Guides Actionable for Educators

Earlier this year, I wrote a short blog about how I envisioned the Regional Educational Laboratories (REL) Program, The What Works Clearinghouse™ (WWC), and the Comprehensive Center Program could work together to take discovery to scale. In it, I promised I would follow-up with more thoughts on a specific—and critically important—example: making WWC Practice Guides actionable for educators. I do so below. At the end of this blog, I pose a few questions on which I welcome comments.

The challenge. The single most important resources the WWC produces are its Practice Guides. Practice Guides evaluate the research on a given topic—say, teaching fractions in elementary and middle school—and boil study findings down to a handful of evidence-based practices for educators. Each practice is given a rating to indicate the WWC’s confidence in the underlying evidence, along with tips for how practices can be implemented in the classroom. In many ways, Practice Guides are IES’s most specific and definitive statements about what works to improve education practice and promote student achievement.

Despite their importance, the amount of effort IES has intentionally dedicated to producing high-quality resources that support educators in implementing Practice Guide recommendations has been uneven. (By most measures, it has been on the decline.) Why? Although we have confidence that the materials we have already produced are high-quality, we cannot prove it. Rigor is part of our DNA, and the absence of systematic efficacy tests demonstrating tools’ contribution to improved teacher practice has made us hesitant to dramatically expand IES-branded resources.

To their credit, several organizations have stepped in to address the “last mile problem” between Practice Guides and classroom practice. Some, like RELs, are IES partners. As a result, we have seen a small number of Practice Guides turned in to professional learning community guides, massively on-line open courses, and other teacher-facing resources. Despite these efforts, similar resources have not been developed for the overwhelming majority of Practice Guides. This means many of our Guides and the dozens of recommendations for evidence-based practice they contain are languishing underused on IES’s virtual bookshelf.

An idea. IES should “back” the systematic transformation of Practice Guide recommendations from words on a page to high-quality materials that support teachers’ use of evidence-based practices in their classrooms. And because we should demonstrate our own practice works, those materials should be tested for efficacy.

From my perspective, RELs are well-suited to this task. This work unambiguously aligns with RELs’ purpose, which is to improve student achievement using scientifically-valid research. It also leverages RELs’ unique value proposition among federal technical assistance providers: the capacity to conduct rigorous research and development activities in partnership with state and local educators. If RELs took on a greater role in supporting Practice Guides in the next REL cycle—which runs from 2022 until 2027—what might it look like in practice?

One model involves RELs collaborating with state and/or district partners to design, pilot, and test a coherent set of resources (a “toolkit”) that help educators bring Practice Guide recommendations to life in the classroom. Potential products might include rubrics to audit current policy or practice, videos of high-quality instructional practice, sample classroom materials, or professional learning community facilitation guides, each linked to one or more Practice Guide recommendations.

Long-time followers of the WWC may recognize the design aspect of this work as similar to the defunct Doing What Works Program. The difference? New resources would not only be developed in collaboration with educators, they must be piloted and tested with them as well. It’s simple, really: if we expect educators to use evidence-based practices in the classroom, we need evidence-based tools to help teachers succeed when implementing them.  

Once vetted, materials must get into the hands of educators who need them. It’s here where the value of the REL-Comprehensive Center partnership becomes clear. With a mission of supporting each state education agency in its school improvement efforts, Regional Comprehensive Centers are in the ideal position to bring resources and implementation supports to state and local education leaders that meet their unique needs. Tools that are developed, piloted, and refined by a REL and educators in a single state can then be disseminated by the national network of Comprehensive Centers to meet other states’ needs.

Extensions. It isn’t hard to imagine other activities that the WWC, RELs, and Comprehensive Centers might take on to maximize this model’s potential effectiveness. Most hinge on building effective feedback loops.

Promoting continuous improvement of Practice Guide resources is an obvious example. RELs could and should be in the business of following Comprehensive Centers as they work with states and districts to implement REL-developed Practice Guide supports, looking for ways to maximize their effectiveness. Similarly, Comprehensive Centers and RELs should be regularly communicating with one another about needs-sensing, identifying areas where support for evidence-based practice is lacking and determining which partners to involve in the solution. When there is a growing body of evidence to support educator best practice, the WWC is in the best position to take the lead and develop a new Practice Guide. When that body of evidence does not exist yet—or when even the practices themselves are underdeveloped—the RELs and other parts of IES, such as the National Centers for Education and Special Education Research, should step in.  

Questions. When the WWC releases a new Practice Guide, its work may be done—at least temporarily. The work of its partners to support take-up of a Guide’s recommendations will, however, have just begun. I’d appreciate your thoughts on how to best accomplish that transition, and offer up the following additional questions for your consideration:

  1. Are we thinking about the problem correctly, and in a helpful way? Are there elements of the problem that should be redefined, and would that lead us to different solutions?

 

  1. What parts of the problem does this proposed solution address well, and where are its shortcomings? Are there other solutions—even solutions that don’t seem to fit squarely within today’s model of the REL Program—that might be more effective?

 

  1. If we proceed under a model like that which is described above:

 

  1. What sort of REL partnership models would be most effective in supporting the conceptualization, design, piloting, and testing of teacher-facing “toolkits” aligned to WWC Practice Guides?

 

  1. What research and evaluation activities—and which outcome measures—should be incorporated into this activity to give IES confidence that the resulting “toolkits” are likely to be associated with changed teacher practice and improved student outcomes?

 

  1. How does the 5-year limit on REL contracts affect the feasibility of this idea, including its scope and cost? What could be accomplished in 5 years, and what might take longer to see to completion?

 

  1. How could RELs leverage existing ED-sponsored content, such as that created by Doing What Works, in service of this new effort?

 

If you have thoughts on these questions or other feedback you would like to share, please e-mail me. I can be reached directly at matthew.soldner@ed.gov. Thanks in advance for the consideration!

by Matthew Soldner, NCEE Commissioner 

Calling All Students to the Mars 2020 “Name the Rover” Contest

On August 27, 2019, NASA launched a national contest for Kindergarten to Grade 12 students to name the Mars 2020 rover, the newest robotic scientist to be sent to Mars.  Scheduled to launch aboard a rocket in July 2020 from Cape Canaveral Air Force Station in Florida and touch down on Mars in February 2021, the to-be-named rover weighs more than 2,300 pounds (1,000 kilograms) and will search for astrobiological signs of past microbial life, characterize the planet’s climate and geology, collect samples for future return to Earth, and pave the way for human exploration of the Red Planet.

By focusing the Mars 2020 “Name the Rover” contest on K to 12 students, NASA seeks to engage U.S. students in the engineering and scientific work that makes Mars exploration possible. The contest also supports national goals to stimulate interest in science, technology, engineering, and mathematics (STEM) and help create the next generation of STEM leaders.

Students can sign up and submit their entries for the competition at https://www.futureengineers.org/nametherover. Entries must include a proposed name for the rover and a short essay of 150 words or less explaining the reasons for the name. NASA will select 156 state winners (one from each state and age group), before narrowing down to the top 9 entries that will be part of a public poll. The grand prize winner who will name the rover will be selected and announced in spring of 2020.

Even if you are not a student you can still participate. US residents over the age of 18 can apply to be judges for the contest to help NASA make their selection.

The Mars 2020 Project at NASA’s Jet Propulsion Laboratory manages rover development for NASA’s Science Mission Directorate. NASA’s Launch Services Program, based at the agency’s Kennedy Space Center in Florida, is responsible for launch management.

NASA Partners with an ED/IES SBIR Awardee to Run the Contest

The education technology firm that NASA selected to help run the competition is Burbank, California-based, Future Engineers.  The “Name the Rover” contest leverages Future Engineers’ online challenge platform, which was developed with the support of a 2017 award from the US Department of Education and Institute of Education Sciences’ Small Business Innovation Research program (ED/IES SBIR).  The platform will receive, manage, display, and judge what is anticipated to be tens of thousands or more student submissions from around the country.

Future Engineers has a history of collaborating on space-themed student challenges. The company previously ran a national competition series in 2018 for the ASME Foundation with technical assistance from NASA, where K-12 students submitted digital designs of useful objects that could be 3D printed on the International Space Station, resulting in the first student-designed 3D print in space.

Future Engineers developed its platform to be an online hub for classrooms and educators to access free, project-based STEM activities, and to provide a portal where students submit and compete in different kinds of maker and innovation challenges across the country. The Mars 2020 “Name the Rover” contest will be the first naming challenge issued on its platform.

We look forward to the results of the competition!

Originally posted on the U.S. Department of Education’s Homeroom blog.


Edward Metz is a research scientist at the Institute of Education Sciences in the US Department of Education.

Bob Collom is an integration lead in the Mars Exploration Program at NASA Headquarters.


About ED/IES SBIR

The U.S. Department of Education’s Small Business Innovation Research program, administered by the Institute of Education Sciences (IES), funds projects to develop education technology products designed to support students, teachers, or administrators in general or special education. The program emphasizes rigorous and relevant research to inform iterative development and to evaluate whether fully-developed products show promise for leading to the intended outcomes. The program also focuses on commercialization once the award period ends so that products can reach students and teachers and be sustained over time. ED/IES SBIR-supported products are currently used by millions of students in thousands of schools around the country.

About NASA’s Mars Exploration Program (MEP)

NASA’s Mars Exploration Program (MEP) in the Planetary Science Division is a science-driven program that seeks to understand whether Mars was, is, or can be, a habitable world. To find out, we need to understand how geologic, climatic, and other processes have worked to shape Mars and its environment over time, as well as how they interact today. To that end, all of our future missions will be driven by rigorous scientific questions that will continuously evolve as we make new discoveries. MEP continues to explore Mars and to provide a continuous flow of scientific information and discovery through a carefully selected series of robotic orbiters, landers and mobile laboratories interconnected by a high-bandwidth Mars/Earth communications network.

IES at the Conference on Computing and Sustainable Societies

Over the summer, researchers, technologists, and policymakers gathered in Accra, Ghana for the Association for Computing Machinery’s Conference on Computing and Sustainable Societies (ACM COMPASS) to discuss the role of information technologies in international development.

Two IES-funded researchers from Carnegie Mellon University’s Program in Interdisciplinary Education Research, Michael Madaio and Dr. Amy Ogan, shared their research on developing voice-based early literacy technologies and evaluating their efficacy with low-literate, bilingual families in the Ivory Coast. 

Their research draws on methods from human-computer interaction, the learning sciences, and information-communication technology for development, to design educational technologies that are culturally and contextually appropriate.

Although the COMPASS conference focused on cross-cultural applications and technology for development, the research presented has implications for U.S. based education researchers, practitioners, and policymakers.

For instance, while research provides evidence for the importance of parental involvement in early literacy, parents with low literacy in the target language – as in many bilingual immigrant communities in the U.S. – may not be able to support their children with the explicit, instrumental help suggested by prior research (for example, letter naming or bookreading). This suggests that there may be opportunities for technology to scaffold low-literate or English Learners (EL) parental support in other ways.

At the conference, researchers described interactive voice-based systems (known as “IVR”) that help low-literate users find out about crop yields, understand local government policies, and engage on social media.  

This body of work has implications for designers of learning technologies in the U.S. Many families may not have a smartphone, but basic feature phones are ubiquitous worldwide, including in low-income, immigrant communities in the U.S. Thus, designers of learning technologies may consider designing SMS- or voice-based (such as IVR) systems, while schools or school districts may consider how to use voice-based systems to engage low-literate or EL families who may not have a smartphone or who may not be able to read SMS information messages.

In a rapidly changing, increasingly globalized world, research at IES may benefit from increased international engagement with international research, both focusing specifically on education, as well as information technology research that has implications for educational research, practice, and policy.

This guest blog was written by Michael Madaio. He is an IES Predoctoral Fellow in the Program in Interdisciplinary Education Research at Carnegie Mellon University. He is placed in the Human-Computer Interaction Institute.