IES Blog

Institute of Education Sciences

Equity: Alignment of Mission and Methods

Editor's Note: The following post was originally posted on the IES-funded CTE Research Network. The grantee has given us permission to post it on the IES blog.

 

 

Funded in 2018 by the Institute of Education Sciences (IES), the Career and Technical Education (CTE) Research Network aims to conduct and promote high-quality casual studies examining the impact of career and technical education. Aligned with the theme of the January 2020 IES Principal Investigators Meeting – Closing the Gaps for All Learners – the Network’s activities include working to deepen the field’s understanding of issues of equity and inequity in CTE research and evaluation.

The importance of understanding equity in CTE research

The Wisconsin Department of Public Instruction defines equity in the following way:

“Every student has access to the educational resources and rigor they need at the right moment in their education across race, gender, ethnicity, language, disability, sexual orientation, family background, and/or family income.”

An explicit focus on equity in CTE is particularly important considering that in the not so distant past, vocational education (a precursor to the term career and technical education, or CTE) often served as the track for youth deemed “unable to learn” or “not college material.” In many cases, vocational education was used to systematically relegate students—many of whom were low-income, Black or African American, Latinx, or American Indian—into low-wage jobs that offered limited opportunities for growth.

Today, the focus of CTE has expanded to include fields in science, technology, engineering and mathematics (STEM) and represents for many young people an opportunity to graduate high school and enter postsecondary education or the labor market with highly valued skills and certifications in numerous fields. As CTE has evolved, participation has become associated with a variety of positive outcomes. For example, researchers have found that CTE course taking is associated with higher high school graduation and postsecondary enrollment rates, higher labor market earnings, and better overall student outcomes.

While these positive CTE outcomes are promising, there is more to understand about the causal outcomes associated with CTE participation, especially among subgroups of students based on race, gender, socioeconomic level, and ability status. IES and the CTE Research Network are committed to deepening the field’s understanding of equity and inequity in CTE studies. Along with acknowledging the pernicious ways in which vocational education has historically been used to discriminate against some students and disaggregating outcome data by student subpopulation (an emphasis in recent Perkins V legislation), the network concludes that at a minimum, engaging in equity-minded research and evaluation requires:

  • Establishing diverse research teams: Research has shown that diversity on teams yields greater innovation, more productivity, and better financial results (Levine, 2020). With these benefits in mind, it is important to be intentional in creating diverse research teams that can bring new perspectives, voices, and approaches to studies that aim to identify, analyze and interpret equity data.
  • Adopting an equity mindset in research and evaluation: To inform the field’s understanding of how CTE may promote or inhibit equitable student outcomes, researchers must commit to recognizing their own biases and examining how those biases may influence their research designs and analyses. An equity mindset also requires capturing and analyzing patterns of inequities that appear in administrative and implementation data.
  • Exploring intersectionality: Adopting an equity mindset—as important for research as is using valid and reliable measures—also requires conducting analyses of CTE outcomes that go beyond merely examining differences between subpopulations. Rather, analyses should also examine intersectionality within subpopulations (for example, by gender and race), which affords the field a more nuanced understanding of how outcomes for members of the same subpopulation may vary by other dimensions of identity (such as gender or ability status). Such analyses can help the field understand what works and for whom—information that can help drive policy and practice.
  • Addressing the systems, policies, and procedures that promote inequities: Inequities do not exist in a vacuum. Thus, it is important to contextualize causal CTE studies, acknowledging how systems, policies, and procedures may create barriers to success for some students. Analyses that take an ecosystems approach—focusing on how the social, economic, and geographic environment shapes outcomes—provide valuable insight into the nature of inequities that exist and how these inequities might be overcome. Equally important is to identify the possible or probable causes of inequities to understand how race, gender, and other variables influence students’ experiences in CTE. Analyses must also extend beyond merely identifying average effect sizes to investigating variation in treatment experiences by subpopulations, an approach that provides valuable insights into how young people in different subpopulations fare relative to their peers in specific contexts. Using data and analysis in this way can provide the evidence needed to support policy recommendations aimed at closing equity gaps and creating the conditions that all students need to transition successfully into adulthood.
  • Engaging the communities that participate in our studies: Because evidence is critical for making data-driven decisions, it is important when designing causal studies to include the participating communities and other stakeholders in the knowledge generation and interpretation processes. These communities and stakeholders can also play an important role in informing researchers’ understanding of the specific causes of inequities identified in study findings. Research should be an inclusive process—the communities being studied and those directly affected by research findings should be included in the planning, implementation, and interpretation of research.
  • Asking what more is needed to promote equity: Embracing equity as a measure of success in education research will take time and will require a significant shift in the way research is conceptualized, designed, and conducted. However, to promote a more just society, it is imperative that researchers keep equity at the center of their work.

Although the CTE Research Network is funded to conduct causal studies, which can play a role in identifying inequities, we realize that other research methods also play a role in deepening the field’s understanding of such inequities. For example, qualitative and implementation research can be used to gain important insight into the contextual factors that shape or reinforce inequities and can also be used to engage stakeholders as informants on the topic. Therefore, building the field’s knowledge of these issues will require employing a range of data collection efforts.

In the meantime, the CTE Research Network is taking the following action steps to continue to advance our equity-minded approach to CTE research:

  • Developing a set of equity questions to consistently consider during network convenings
  • Elevating issues of equity in all network presentations
  • Sharing resources on equity to help network members think critically about how best to bring an equity lens to bear on research and evaluation studies
  • Creating and promoting opportunities to help diversify researchers engaged in causal CTE research

As a network, we believe these research practices will shine a light on (in)equity in CTE. Where inequities exist, we hope our work will inform education policymaking that aims not only to close existing equity gaps but also to prevent the perpetuation of inequities in CTE. We invite other researchers to join us in this effort by taking similar action steps as part of their own research and evaluation endeavors. The following resources can inform researchers’ understanding of equity issues in general and in CTE studies in particular:

 

References

Andrews, K., Parekh, J., & Peckoo, S. (2019). How to embed a racial and ethnic equity perspective in research: Practical guidance for the research process. Washington, DC: Child Trends.

Dougherty, S. M. (2016). Career and technical education in high school: Does it improve student outcomes? Washington, DC: Thomas B. Fordham Institute. https://eric.ed.gov/?id=ED570132

Hemelt, S. W., Lenard, M. A., & Paeplow, C. G. (2017). Building better bridges to life after high school: Experimental evidence on contemporary career academies. Washington, DC: National Center for Analysis of Longitudinal Data in Education Research. https://eric.ed.gov/?id=ED572934

Hodge, E., Dougherty, S., & Burris, C. (2020). Tracking and the future of career and technical education: How efforts to connect school and work can avoid the past mistakes of vocational education. Boulder, CO: National Education Policy Center. Retrieved from http://nepc.colorado.edu/publication/cte

Kemple, J. (2008). Career academies: Long-term impacts on work, education, and transitions to adulthood. New York: MDRC. Retrieved from https://www.mdrc.org/publication/career-academies-long-term-impacts-work-education-and-transitions-adulthood

Rosen, R., & Molina, F. (2019). Practitioner perspectives on equity in career and technical education. New York: MDRC. https://eric.ed.gov/?id=ED596458


Written by Equity in CTE Workgroup, on behalf of the CTE Research Network

This is the fifth in a series of blog posts that stems from the 2020 Annual Principal Investigators Meeting. The theme of the meeting was Closing the Gaps for All Learners and focused on IES’s objective to support research that improves equity in access to education and education outcomes. Other posts in this series include Addressing Persistent Disparities in Education Through IES ResearchWhy I Want to Become an Education ResearcherDiversify Education Sciences? Yes, We Can!, and Closing the Opportunity Gap Through Instructional Alternatives to Exclusionary Discipline.

 

 

IES is Investing $20 Million to Improve Opportunities and Achievement for English Learners in Secondary School Settings

 

IES supports Research and Development Centers (R&D Centers) to conduct focused, scientific research on key education issues that face our nation. One of the most pressing issues currently is a need for a large-scale, coordinated research effort to improve opportunities and achievement for English Learners (ELs) in secondary school settings. These students face numerous barriers and challenges in accessing education opportunities, as they simultaneously develop English language proficiency and subject-matter knowledge. This has resulted in large, persistent differences in academic achievement outcomes, for example, between ELs and non-ELs on the 2017 National Assessment of Educational Progress (NAEP) in reading and math. These differences have consequences for how prepared (or unprepared) students will be to access opportunities in the workforce and postsecondary education.

The National Center for Education Research (NCER), within IES, is funding two national R&D centers on the topic of “Improving Opportunities and Achievements for English Learners in Secondary School Settings” to contribute to the solution of this complex education problem. These R&D centers aim to identify systemic—including policies, rules, and processes—and instructional influences that affect ELs’ access to the general curriculum and its relation to their outcomes. The centers will carry out complementary projects and engage in research, development, evaluation, and national leadership activities. These activities will facilitate meaningful EL participation in the general curriculum and generate insights about levers that have the potential to improve access and education outcomes for ELs in secondary school settings.

Each of the centers is conducting two lines of research, in which they will:

  • Identify and describe the policies and system-level practices that are associated with secondary ELs’ access to the general curriculum and their relation to education outcomes
  • Identify approaches to improving secondary ELs’ access to and ability to learn from instruction in general education courses where English is the language of instruction.

 

The R&D Centers

Transdisciplinary Approaches to Improving Opportunities and Outcomes for English Learners: Using Engagement, Team-Based Learning, and Formative Assessment to Develop Content and Language Proficiency is headed by Dr. David Francis at the University of Houston. This center will undertake a five-year, focused program of research aimed at identifying and removing barriers related to school-level practices influenced by policies that constrain course-taking, as well as developing and testing interventions that leverage transdisciplinary approaches to improve instruction for ELs in grades 6 and 9 science and social studies. These approaches include—

  • Foregrounding content to build language through content instruction
  • Using activities that are engaging and meaningful to students while involving students in the practices of the discipline
  • Organizing learning in heterogeneous teams (Team-Based Learning) to promote collaboration, discussion, and social motivation
  • Integrating formative assessment to improve teachers’ and students’ understanding of students’ development.

The center will also engage in a systematic program of outreach and dissemination at national, state, regional, and local levels, to help connect research findings to policy and practice.

The National Research and Development Center to Improve Education for Secondary English Learners is headed by Aída Walqui at WestEd. The center’s five-year program of research focuses on understanding policies and systems-level practices that are associated with ELs’ access to the general curriculum and how curriculum resources can strengthen the learning opportunities and experiences of both teachers and ELs as they engage in disciplinary practices. The center will—

  • Examine course-taking patterns across multiple states and explore how content-area course access is related to student outcomes
  • Describe models for co-teaching across the country and its implementation in diverse settings
  • Iteratively design and develop curricular materials in English language arts (ELA) and a math summer bridge course for grade 8 students to help teachers shift their pedagogical practices.

In addition, the center will engage in national leadership, capacity-building, and outreach activities to connect research findings to policy and practice and offer policymakers and practitioners compelling and actionable information.

 

Through the Education Research and Development Center Program, NCER’s goal is for the two centers to collectively and individually create innovative solutions, contribute to knowledge and theory, and provide reliable information about how to improve educational opportunities and achievement for ELs in secondary school settings. The body of research conducted by the two centers will identify and address the policies and system-level practices across the US that act as barriers to accessing rigorous courses. Their work will also inform instruction in four critical content areas of secondary instruction—science, social studies, ELA, and math—in several states, including Texas, Massachusetts, New York, and California.

 

For more information about IES’s support on the topic of English Learners, please see the program page; for current funding opportunities for research and training more generally, please see here. Stay tuned for updates on Twitter (@IESResearch) and Facebook as IES continues to support research to improve educational outcomes for English Learners.


Written by Helyn Kim (Helyn.Kim@ed.gov), Program Officer for English Learners at the National Center for Education Research.

 

IES Makes Two New SBIR Awards for the Full-Scale Development of Web-based Tools to Inform Decision Making by Postsecondary Students

The Institute of Education Sciences (IES) has announced two new awards to technology firms to fully develop web-based tools that inform student decision making in postsecondary education. The projects will focus on generating a measure of the return of investment (ROI) for different educational training programs and careers, so that high school and college students have access to data-driven information to guide their decisions.

The awards were made through a special topic offered by the U.S. Department of Education/IES Small Business Innovation Research (known as ED/IES SBIR) program, which funds the research and development of commercially viable education technology. (For information on the 22 awards made through the IES 2020 standard solicitation, read here.)

 

Background and Awards

While websites like College Scorecard and CareerOneStop provide information to explore training programs in colleges and occupations of interest, there is no tool that helps students understand the costs and benefits of individual postsecondary programs in an integrated, customizable, and user-friendly manner.  

A 2019 Phase I SBIR special topic solicitation requested proposals from small businesses to develop new ROI tools that would improve program completion rates, employment and earnings, and satisfaction with college or career, all while minimizing education-related debt. In 2019, two Phase I awards (see here and here) were made to firms to develop prototype ROI tools. The 2020 Phase II SBIR special topic solicitation requested proposals from the two 2019 Phase I awardees with plans for the full-scale development of the ROI tools, as well as an interface to present information to students and users.  Both firms received 2020 Phase II awards through this special topic:

 

  • Illinois-based BrightHive, Inc. is fully developing an open-source toolkit that states and networks can use to calculate metrics that support and inform decision-making by postsecondary students. The Training, Education, and Apprenticeship Program Outcomes Toolkit (TEAPOT) will include several components to support implementation, including data requirements, a field guide with instructional materials and technical tools to help partners to securely and ethically combine wage and education data, a resource hub containing expert guidance and recommendations for deploying the ROI metrics in different contexts and apps, and a reference interface that allows states and networks to present ROI data as part of their career guidance applications.

 

  • Virginia-based Vantage Point Consultants is fully developing the Return on College (ROC) tool to support students, families, and counselors with understanding of the lifetime costs and opportunity tradeoffs associated of different postsecondary degree programs.  In one version of ROC, students will be able to search colleges nationwide based on geographic location and occupational goals, with personalized output on projected debt, loan repayment, disposable income, graduation risk, and the ROI focal points of these decisions. In the other version of ROC, prospective students visiting a college website will receive data driven information on potential careers, salaries, price, and debt expected for degrees offered by a school.

 

After development is complete in early 2022, researchers will analyze whether the tools function as intended and are feasible for students to use. Research will also test if the tools show promise for producing a meaningful and accurate measure of ROI. Both firms will also focus on preparing for the wide-scale distribution of the tools to end users, as well as plans for sustaining the tools over time.

Stay tuned for updates on Twitter (@IESResearch) as IES projects drive innovative forms of technology.


Written by Edward Metz (Edward.Metz@ed.gov), Program Manager, ED/IES SBIR

Investing in Next Generation of Education Technologies to Personalize Learning and Inform Instructional Practice

The Institute of Education Sciences Small Business Innovation Research program (ED/IES SBIR) funds entrepreneurial developers to create the next generation of education technology for students, teachers, and administrators in general and special education. The program emphasizes an iterative research and development process and pilot studies to evaluate the feasibility, usability, and promise of new products to improve educational outcomes. The program also focuses on commercialization after development is complete, so that the products can reach schools and be sustained over time.

In recent years, millions of students in tens of thousands of schools around the country have used technologies developed through ED/IES SBIR. And in the past four months, about one million students and teachers used the technologies for remote teaching and learning, as many ED/IES SBIR-supported developers made their products available at no cost in response to the COVID-19 pandemic and the closure of schools.

 

ED/IES SBIR Announces its 2020 Awards

This week, ED/IES SBIR announced the results of its 2020 award competition. Of the 22 new awards, 16 are for prototype development and 6 are for full-scale development. IES also announced two additional awards through a special topic solicitation in postsecondary education. Read about these awards here.

 

 

Each of the new awards supports a project to develop a product to personalize the student learning experience or generate information that educators can use to guide practice.

Most of the projects are developing a software component (for example, algorithms, artificial intelligence, machine Learning, or natural language processing) that continually adjusts the difficulty of content, provides prompts to support individual students if support is needed, or generates real-time actionable information for educators to track student progress and adjust instruction accordingly. Other projects are developing technologies to promote student learning through self-directed, hands-on, simulated, and immersive experiences. If the future of education includes a blend of in-class and remote learning due to public health crises, or for whatever reasons, technologies such as these will be ready to keep the learning going.

The projects address different ages of students and content areas.

In science, LightHaus is fully developing a virtual reality (VR) intervention for students to explore plant heredity; LightUp is fully developing an augmented reality (AR) app for students to perform hands-on physical science investigations with their own on-device camera; and Myriad Sensors is developing a prototype artificial intelligence formative assessment system that generates feedback in real time as students do hands-on laboratory experiments.

In math, Muzology is creating a prototype for students to create music videos to learn algebra, and Teachley is creating a prototype transmedia kit with videos, comics, and pictures to enhance teaching and learning of hard to learn concepts.

In engineering and computer science, Parametric Studios is fully developing an augmented reality puzzle game for early learners, and Liminal eSports, Makefully, and Beach Day Studios are creating prototype components that each provide feedback to students as they engage in activities to learn to code.

In English Language Arts, Analytic Measures and Hoogalit are each employing natural language processing to develop new prototypes to facilitate speech acquisition, and Learning Ovations is developing a prototype data engine to make recommendations for what individual children should read.

For English learners, KooApps is developing an artificial intelligence prototype to support vocabulary acquisition, and Kings Peak Technologies is employing machine learning to generate passages that blend English and Spanish words together to improve reading comprehension.

For early learners, Cognitive Toybox is fully developing an observation and game-based school readiness assessment.

For postsecondary students, Hats & Ladders is fully developing a social skills game to foster career readiness skills.

In special education, Attainment Company is developing a prototype to support student’s self-management, and Alchemie is developing a prototype of an augmented reality science experience for visually impaired students.

To support school administrators and teachers, LearnPlatform is fully developing a dashboard that generates reports with insights for teachers to implement education technology interventions, and Zuni Learning Tree, Teachley and LiveSchool are developing prototype dashboards to organize and present results on student progress and performance in real time.

 

Stay tuned for updates on Twitter and Facebook as IES continues to support innovative forms of technology.


Written by Edward Metz (Edward.Metz@ed.gov), Program Manager, ED/IES SBIR

CTE Research Network Identifies Four Sites Ready to be Evaluated

 

In 2018, the IES awarded a grant1 to the American Institutes for Research (AIR) to lead a research network focused on career and technical education (CTE), the Expanding the Evidence Base for Career and Technical Education Network (CTE Research Network). The mission of the CTE Research Network is to increase the number of CTE impact studies and strengthen the capacity of the field to conduct and use rigorous CTE research.

One of AIR’s primary tasks as the CTE Network’s Lead is to conduct an evaluability study (also called a feasibility study) to identify CTE models or programs that could be evaluated using a rigorous experimental design. The purpose of the study is to ease the way for other researchers to evaluate CTE by doing the advance work to find suitable sites that may be interested in participating in research. Any interested research team may approach one of these sites to partner in an evaluation. IES and AIR hope that qualified teams will submit an application to the IES Education Research Grants program, under the CTE topic, for grant funding to conduct an evaluation.

In a preliminary report released today, the CTE Network Lead describes the method they used to identify a broad range of programs and models, the vetting criteria, and the reasons for selecting the four sites. For each of the selected sites, the report also describes the scope of the program and student enrollment, the CTE programs offered, the data available, and the willingness of the sites to welcome researchers to evaluate the CTE program. In addition, the report includes the suggested next steps for researchers and possible limitations in carrying out an evaluation of the particular site or program model.

Prior research on CTE over the last half century has mostly been exploratory in nature or, at best, quasi-experimental. One of the primary reasons for the lack of experimental research is that it is difficult to assign students to elective courses. Even quasi-experimental designs are challenging, as it is difficult to statistically control for all the reasons a student might choose to enroll in CTE. See here and here for further discussion of the challenges in conducting CTE research.

The CTE Research Network has another upcoming effort to help increase the CTE evidence base: a free training on causal methods for CTE research. The training will take place online in August 2020; the deadline for applications is June 30, 2020.

News about the CTE Research Network and resources to help CTE researchers can be found on the Network’s website; IES also occasionally blogs about the research findings of Network members. Although most of the CTE Network members are currently studying CTE at the secondary level, we hope that more research will be conducted at the postsecondary level. Researchers interested in applying to IES for a grant to study CTE are welcome to contact Corinne Alfeld (contact information below).


1Using Perkins funds from the Office of Career, Technical, and Adult Education (OCTAE) in partial fulfillment of the legislative requirement for a national research center to carry out scientifically-based research and evaluation for the purpose of developing, improving, and identifying the most successful methods for addressing the education, employment, and training needs of career and technical education (CTE) participants in CTE programs [Sec. 114(d)(4)].

 

Written by Corinne Alfeld (corinne.alfeld@ed.org), IES program officer, and Katherine Hughes (khughes@air.org), principal investigator for the CTE Network Lead at AIR