IES Blog

February is Career and Technical Education (CTE) month! As part of our 20^th anniversary celebration, we want to highlight the great work our CTE Research Network (CTERN) continues to accomplish. The blog below highlights NCER’s conversation with the Equity Working Group of the IES-funded CTE Research Network. 

The Equity Working Group (EWG) of the CTE Research Network (CTERN) has published a new resource for researchers on using an equity lens in developing and conducting CTE research: The Equity Framework for CTE Research. CTERN is hosting a free webinar on February 21st at 3:00 Eastern to provide an overview of the framework and how people can use it. In this blog, members of the Equity Working Group answered questions about the framework and why it is important. 

The framework has a focus on equity, but equity can mean different things to different people. How does the EWG define equity in this framework?

We strongly believe that every student should have the opportunity to engage in quality educational experiences. Students who are interested should have access to CTE programs, regardless of their background characteristics. And school systems should invest in students so that they can succeed in these programs. Ultimately, we find ourselves quoting the Wisconsin Department of Public Instruction’s definition because it neatly captures our position: “Every student has access to the educational resources and rigor they need at the right moment in their education across race, gender, ethnicity, language, disability, sexual orientation, family background, and/or family income.”

Why did the EWG members believe that there was a need for an equity framework for CTE research?

CTE has a long and complicated history, including extensive tracking under its previous incarnation as vocational education. The CTE Equity Working Group was very conscious of this history and wanted to take steps to help ensure that CTE research was helping to ameliorate current inequities. As we say in the framework, “We believe that infusing equity throughout our research is critical to ensuring that research can make a difference in promoting equitable learning experiences and outcomes for all students who participate in CTE.”

We also recognized that many researchers (including ourselves) want to use an equity lens to do their research but lack practical guidance in what that looks like. The working group believed that a framework with concrete examples and tips would help CTE researchers have a clearer picture of what to do and would provide a tool for helping them think differently about their work.

How did the EWG create the framework?

This was a collaborative process that grew out of our first CTE Research Network meeting in 2018 or 2019. A group of us realized that incorporating an equity lens into our work would help us better answer questions that matter to communities. We decided to form a working group, which ended up including around 20 or so researchers, practitioners, and policy staff. We read a lot of good frameworks from different organizations on improving our research practices, so we decided to invest our energy in seeing how it may be applied to a CTE context.

How is the framework structured and what are some key takeaways?

It is important to note what this framework is and is not. This framework is not intended as a methodological primer or a replication of existing research guidance; it is intended to encourage researchers to think about their own work through an equity lens.

The framework starts with a brief history of equity in CTE, a description of the process of creating the framework, a list of vocabulary (we believe having a common language is critical), and a statement of the values that underlie the framework.

The rest of the framework is then organized by six stages of research: 1) project management; 2) research design, 3) measurement and data collection, 4) data analysis, 5) cost and resource equity, and 6) reporting and dissemination. In each section, we include a description of how to implement the stage with an equity-focused lens, with questions for researchers to consider and potential barriers. Throughout, we have included examples from current and future CTE research. We are looking for more examples, so people should feel free to reach out to us at jedmunds@serve.org to share how they are doing this work.

In creating summary products to go along with the framework, we identified six themes that cut across the different stages: ensure transparency, involve the community, develop diverse teams, take a systems approach, acknowledge and attend to bias, and demonstrate respect. These themes are summarized in an infographic.

How do you hope that people will use the framework?

We hope this will help start or further conversations among CTE researchers. We structured the framework around each stage of the research process, so anyone engaging in this work can find elements to incorporate or questions to consider individually and as a team, regardless of where they are in their work right now. For studies just getting off the ground, we did our best to illustrate how researchers can build an equity approach from the start of a project through its completion.

What are some examples of how the framework changed individual EWG members’ research practices?

Julie A. Edmunds (co-facilitator): Working on the framework has crystallized three high-impact equity-focused practices that I now try to infuse throughout my work. First, I pay much more attention to the role of systems in inequities. I try to look at upstream factors that might be causing disparities in educational outcomes as opposed to just documenting gaps that might exist between sub-groups. Second, when presenting those gaps (which we still do because it is useful information), I am much more conscious about how those gaps are displayed. For example, we focus on making sure that “White” is not considered the default category against which all others are compared. Third, we are creating processes to ensure that we share our findings with people who gave us the data. For example, we are sending practitioner-friendly products (such as briefs or infographics) to the school staff we interviewed whose insights formed the basis for some of our findings.

John Sludden (member): The framework has helped us think about our internal processes and keeps us focused on our audience, who we’re doing this for. I’m an analyst on the project, and I’ve been empowered to ask questions, conduct analyses, and present to our research partners at the New York City Department of Education. We’re currently thinking about ways to communicate findings to different audiences. At the moment, we’re working on a plan to share findings with principals of CTE high schools in New York City. Organizationally, we are also working on ways to directly engage students in the city, who know more about the system than we ever will. Similar to Julie, analytically, we have spent a lot of our time and attention on looking at the conditions under which students have not been well-served by the system, and ways that students may be better served by CTE.

This blog was produced by Corinne Alfeld (Corinne.Alfeld@ed.gov), program officer, NCER.

Recently, the Office of the Chief Statistician, within the Office of Management and Budget (OMB), released an initial set of recommended revisions for OMB’s Statistical Policy Directive No. 15 (SPD 15), which provides the statistical standards for collecting and reporting race and ethnicity data across federal agencies. The revisions were proposed by an Interagency Technical Working Group.

This is the next step in a process that began last summer with a simple goal: to ensure that the standards better reflect the diversity of the American people. The initial proposals—developed by federal government staff representing more than 20 agencies—include the following:

• collecting race and ethnicity together with a single question
• adding a response category for Middle Eastern and North African that is separate and distinct from the “White” category
• updating SPD 15’s terminology, definitions, and question wording

These recommendations are preliminary—not final—and they do not represent the positions of OMB or the agencies participating in the Working Group.

The Working Group is committed to a full, transparent revision process and remains on track to reach the goal of completing these important revisions by the summer of 2024.

The Working Group Wants to Hear Directly From the American People

The public’s participation in this process will play a critical role in helping the Working Group improve the way federal agencies safely and accurately collect and use information on the race and ethnicity of our diverse population.

Interested stakeholders can read the full Federal Register Notice and provide comments, participate in one of the Working Group’s bi-monthly virtual listening sessions or upcoming virtual town halls, and schedule a listening session.

Be sure to follow NCES on Twitter, Facebook, LinkedIn, and YouTube and subscribe to the NCES News Flash to receive notifications about the revision process and opportunities to engage with the Working Group.

The Individuals with Disabilities Education Act (IDEA), first enacted in 2004, specifies how public agencies should provide early intervention, special education, and related services to children with disabilities and delineates the rights of families to participate in meetings in which decisions are made on the evaluation, identification, and educational placement of their children. However, students with disabilities transition out of IDEA coverage when they graduate high school or reach age 21. This is a particularly crucial moment for students with intellectual and developmental disabilities (IDD).

Under IDEA, parental decision-making rights transfer to students at age 18 unless parents seek further guardianship. This complex issue requires understanding of the options and what they mean for the student. Alleviating any knowledge gaps is important for better transition outcomes for secondary students. In 2020, the U.S. Department of Education released an updated transition guide, A Transition Guide to Postsecondary Education and Employment for Students and Youth with Disabilities, for high school educators to better equip students on their transition into postsecondary education. The Department also released a brief to better inform students about their rights and the overall transition process as they prepare for postsecondary education and employment.

Allison Hall at the University of Massachusetts, Boston and her research team are exploring the role of special educators in informing youth and parents about transfer of rights and guardianship and its implications for transition outcomes for students with IDD. The project began with a review of the literature, a document review of state-level policies, and interviews with experts in the field of transition for students with IDD. These three initial research activities were followed by interviews with students, parents, and special educators on their experience during discussions of the transition process. We asked her to update us on the project and what they’ve learned so far. (Responses have been edited for brevity and clarity.)

NCSER: Please provide an update of where you are with the project.

Some of the most consequential COVID-19-related decisions for public education were those that modified how much in-person learning students received during the 2020-2021 school year. As part of an IES-funded research project in collaboration with the Virginia Department of Education (VDOE) on COVID’s impact on public education in Virginia, researchers at the University of Virginia (UVA) collected data to determine how much in-person learning students in each grade in each division (what Virginia calls its school districts) were offered over the year. In this guest blog, Erica Sachs, an IES predoctoral fellow at UVA, shares brief insights into this work.

Our Process

COVID-19 has caused uncertainty and disruptions in public education for nearly three years. The purpose of the IES-funded study is to describe how Virginia’s response to COVID-19 may have influenced access to instructional opportunities and equity in student outcomes over multiple time periods. This project is a key source of information for the VDOE and Virginia schools’ recovery efforts. An important first step of this work was to uncover how the decisions divisions made impacted student experiences during the 2020-21 school year. This blog focuses on the processes that were undertaken to identify how much in-person learning students could access.

During 2020-21, students were offered school in three learning modalities: fully remote (no in-person learning), fully in-person (only in-person learning), and hybrid (all students could access some in-person learning). Hybrid learning often occurred when schools split a grade into groups and assigned attendance days to each group. For the purposes of the project, we used the term “attendance rotations” to identify whether and which student group(s) could access in-person school on each day of the week. Each attendance rotation is associated with a learning modality.

Most divisions posted information about learning modality and attendance rotations on their official websites, social media, or board meeting documents. In June and July of 2021, our team painstakingly scoured these sites and collected detailed data on the learning modality and attendance rotations of every grade in every division on every day of the school year. We used these data to create a division-by-grade-by-day dataset.

A More Precise Measure of In-Person Learning

An initial examination of the dataset revealed that the commonly used approach of characterizing student experiences by time in each modality masked potentially important variations in the amount of in-person learning accessible in the hybrid modality. For instance, a division could offer one or four days of in-person learning per week, and both would be considered hybrid. To supplement the modality approach, we created a more precise measure of in-person learning using the existing data on attendance rotations. The new variable counts all in-person learning opportunities across the hybrid and fully in-person modalities, and, therefore, captures the variation obscured in the modality-only approach. To illustrate, when looking only at the time in each modality, just 6.7% of the average student’s school year was in the fully in-person modality. However, using the attendance rotations data revealed that the average student had access to in-person learning for one-third of their school year.

Lessons Learned

One of the biggest lessons I learned working on this project was that we drastically underestimated the scope of the data collection and data management undertaking. I hope that sharing some of the lessons I learned will help others doing similar work.

• Clearly define terminology and keep records of all decisions with examples in a shared file. It will help prevent confusion and resolve disagreements within the team or with partners. Research on COVID-19 in education was relatively new when we started this work. We encountered two terminology-related issues. First, sources used the same term for different concepts, and second, sources used different terms for the same concept. For instance, the VDOE’s definition of the “in-person modality” required four or more days of access to in-person learning weekly, but our team classified four days of access as hybrid because we define “fully in-person modality” as five days of access to in-person learning weekly. Without agreed-upon definitions, people could categorize the same school week under different modalities. Repeated confusion in discussions necessitated a long meeting to hash out definitions, examples, and non-examples of each term and compile them in an organized file.
• Retroactively collecting data from documents can be difficult if divisions have removed information from their web pages. We found several sources especially helpful in our data collection, including the Wayback Machine, a digital archive of the internet, to access archived division web pages, school board records, including the agenda, meeting minutes, or presentation materials, and announcements or letters to families via divisions’ Facebook or Twitter accounts.
• To precisely estimate in-person learning across the year, collect data at the division-by-grade-by-day level. Divisions sometimes changed attendance rotations midweek, and the timing of these changes often differed across grades. Consequently, we found that collecting data at the day level was critical to capture all rotation changes and accurately estimate the amount of in-person learning divisions offered students.

What’s Next?

The research brief summarizing our findings can be downloaded from the EdPolicyWorks website. Our team is currently using the in-person learning data as a key measure of division operations during the reopening year to explore how division operations may have varied depending on division characteristics, such as access to high-speed broadband. Additionally, we will leverage the in-person learning metric to examine COVID’s impact on student and teacher outcomes and assess whether trends differed by the amount of in-person learning divisions offered students.

Erica N. Sachs is an MPP/PhD Student, IES Pre-doctoral Fellow, & Graduate Research Assistant at UVA’s EdPolicyWorks.

This blog was produced by Helyn Kim (Helyn.Kim@ed.gov), Program Officer, NCER.

We hope you enjoyed yesterday’s network lead spotlight! Today, we would like to introduce Dr. Susan Therriault, director, K–12 Systemic Improvement Portfolio at the American Institute for Research. Dr. Therriault’s network, the PreK-12 Research on Education Strategies to Advance Recovery and Turnaround (RESTART) Network, aims to coordinate activities across research teams and provides national leadership on learning acceleration and recovery from pandemic-induced learning loss, sharing findings from the network with education agencies across the United States. Happy reading!

NCER: What are the mission and goals of the PreK–12 RESTART (Research on Education Strategies to Advance Recovery and Turnaround) Network? 

Dr. Therriault: The PreK–12 RESTART Network is an opportunity to develop a coherent and connected research community that speaks directly to the needs of policymakers, leaders, and practitioners. The network focuses on identifying and disseminating evidence-based strategies aligned with the needs of policymakers, leaders, and educators who are serving and supporting accelerated student recovery efforts. This requires the network to identify critical needs of the field and support the research community in developing coherent and coordinated research strategies that build evidence for practices that ensure student recovery—especially among students who have disproportionately struggled in the pandemic context. The PreK–12 RESTART Network will achieve this by need sensing, synthesizing evidence, and building a community that makes meaningful connections between the research community and policymakers, leaders, and educators.

NCER: Why is the PreK–12 RESTART Network important to you? 

Dr. Therriault: The PreK–12 RESTART Network is important to me because, as a researcher, I have watched how the pandemic and subsequent aftershocks of the pandemic have created disruptions to our lives and our public pre-K to 12 education system. The pandemic created fragmentation and division as communities responded and supported individuals in a context marked by social distance and separation. While there are many common challenges across communities, limited social connection affected our ability to share evidence-based solutions and to equitably address the needs of all members of our communities, especially those communities most adversely impacted by COVID-19, including Black and Latinx communities and those marked by poverty and housing and food insecurity.

NCER: How do you think the PreK–12 RESTART Network will impact the pre-K to 12 community?

Dr. Therriault: The PreK–12 RESTART Network is an opportunity to develop a coherent and connected research community that speaks directly to the needs of policymakers, leaders, and practitioners. The key differentiator of the network is that it is purposefully designed to assess needs and engage the research community in providing insight and building evidence for solutions to address those needs.

The network has an important role to play in drawing researchers together to develop measurement solutions and build consensus for approaches to conducting and making meaning of research so that it informs the field. These solutions will be shared with the field to create a more coherent research agenda informed by needs.

The network will ensure that policymakers, leaders, and educators are able to easily access network evidence syntheses and research-team findings through multiple communication formats. Information will be shared through actionable guidance and recommendations purposefully designed for these audiences. A combination of strategies will ensure accessibility. These include digital tools and dashboards that school leaders can easily use to adapt to their circumstances and access to evidence-based strategies by offering recorded webinars, tutorials, videos, and other learning formats.

NCER: What’s one thing you wish more people knew about recovery in pre-K to 12 education? 

Dr. Therriault: The magnitude of the challenge of pandemic recovery in the pre-K to 12 education system is vast and will require new ways of approaching education and support for students. In turn, with the investment of American Rescue Plan funds in schools, this will likely lead to evidence-based innovation and deeper understanding of how to design an education system, district, and school to meet student needs.

The needs of students are varied and highly connected to the experience during the pandemic and after the pandemic; thus, family and community factors are highly relevant and critical to understanding student needs and strategies to address these needs. Recently released NAEP scores provide evidence of the variation and suggest more-significant losses in mathematics and English language arts for students living in low-income households compared to their peers who are not. Further, students living in low-income households were more likely to report not having a place to do work or access to a computer or adequate uninterrupted time compared to their peers. These are critical factors in a remote and even a hybrid learning environment. These differences in experience exacerbate differences in outcomes during the pandemic.

NCER: What are some of the biggest challenges to recovery in pre-K to 12 education? 

Dr. Therriault: The amount of need among students and their families and the fatigued pre-K to 12 education system workforce are the biggest challenges to recovery. Adding to this challenge is the focus on expanding learning time through summer school or longer school days in an effort to accelerate learning. This requires teachers and leaders at a time when, like many of us, they are experiencing burnout.

Finally, we know that most students suffered learning loss and more during the pandemic. Supporting students emotionally as well as academically is necessary for recovery. While many schools have provided emotional support to students prior to the pandemic, the current need is far greater than schools have experienced. This will require extensive outreach to community support and services and additional interventions.

NCER: What are some effective ways to translate education research into practice so that your work will have a direct impact on states, districts, and schools? 

Dr. Therriault: There are several ways we plan to support the translation of research to practice through the PreK–12 RESTART Network. These include:

• Understanding and sharing needs of the field.
  • Conducting needs assessments of the field to examine the evolving needs and share these with the research community.
• Identifying, sharing, and amplifying evidence-based strategies that respond to the needs of the field.
  • Exploring existing and emerging research to identify evidence-based strategies and interventions that align evidence syntheses with the needs of the field.
  • Providing actionable guidance and recommendations that can be easily implemented across different schools and are customized based on need.
  • Creating digital tools that school leaders and educators can use to adapt to their circumstance.
• Building a coherent and coordinated research community focused on pandemic recovery research.
  • Connecting and building consensus among researchers through convenings and solutions working groups that address challenges to conducting research in the pandemic context.
  • Empowering research teams to build studies that align with needs in the field through meaningful connections with policymakers, leaders, educators, and other members of the research community.
  • Supporting engagement and preparation of early-career researchers through trainings and networking opportunities.

NCER: What are some barriers to the uptake of the research outcomes by these organizations?

Dr. Therriault: One of the critical barriers to uptake is timing. States, districts, and schools cannot wait for findings and results—they must act to support the students they have in their classrooms right now. The syntheses and reviews of prior research will help point educators in the direction of interventions and other supports that have a strong evidence base. The researchers participating in the RESTART Network will be supported in rapidly sharing and disseminating findings over the course of their studies to inform decision-making about how best to help students’ academic recovery.

Thank you for reading our conversation with Dr. Susan Therriault! We hope you’ve enjoyed getting to know NCER’s network leads throughout our grantee spotlight series. Let us know your thoughts on the series on Twitter at @IESResearch.