Inside IES Research

Notes from NCER & NCSER

CTE Research Through an Equity Lens

This image depicts six considerations for centering equity in CTE research:  Ensure transparency: Be clear about the why, the what, and the who Involve the community: Obtain feedback from research participants throughout the process Develop diverse teams: Ensure teams represent varied perspectives and are trained in equity-based research perspective Take a systems approach: Be cognizant of historical issues of inequity within vocational education Acknowledge and attend to bias: Consider how bias is present in different parts of research Demonstrate respect: Bring an asset-based perspective

February is Career and Technical Education (CTE) month! As part of our 20th anniversary celebration, we want to highlight the great work our CTE Research Network (CTERN) continues to accomplish. The blog below highlights NCER’s conversation with the Equity Working Group of the IES-funded CTE Research Network

The Equity Working Group (EWG) of the CTE Research Network (CTERN) has published a new resource for researchers on using an equity lens in developing and conducting CTE research: The Equity Framework for CTE Research. CTERN is hosting a free webinar on February 21st at 3:00 Eastern to provide an overview of the framework and how people can use it. In this blog, members of the Equity Working Group answered questions about the framework and why it is important. 

The framework has a focus on equity, but equity can mean different things to different people. How does the EWG define equity in this framework?

We strongly believe that every student should have the opportunity to engage in quality educational experiences. Students who are interested should have access to CTE programs, regardless of their background characteristics. And school systems should invest in students so that they can succeed in these programs. Ultimately, we find ourselves quoting the Wisconsin Department of Public Instruction’s definition because it neatly captures our position: “Every student has access to the educational resources and rigor they need at the right moment in their education across race, gender, ethnicity, language, disability, sexual orientation, family background, and/or family income.”

Why did the EWG members believe that there was a need for an equity framework for CTE research?

CTE has a long and complicated history, including extensive tracking under its previous incarnation as vocational education. The CTE Equity Working Group was very conscious of this history and wanted to take steps to help ensure that CTE research was helping to ameliorate current inequities. As we say in the framework, “We believe that infusing equity throughout our research is critical to ensuring that research can make a difference in promoting equitable learning experiences and outcomes for all students who participate in CTE.”

We also recognized that many researchers (including ourselves) want to use an equity lens to do their research but lack practical guidance in what that looks like. The working group believed that a framework with concrete examples and tips would help CTE researchers have a clearer picture of what to do and would provide a tool for helping them think differently about their work.

How did the EWG create the framework?

This was a collaborative process that grew out of our first CTE Research Network meeting in 2018 or 2019. A group of us realized that incorporating an equity lens into our work would help us better answer questions that matter to communities. We decided to form a working group, which ended up including around 20 or so researchers, practitioners, and policy staff. We read a lot of good frameworks from different organizations on improving our research practices, so we decided to invest our energy in seeing how it may be applied to a CTE context.

How is the framework structured and what are some key takeaways?

It is important to note what this framework is and is not. This framework is not intended as a methodological primer or a replication of existing research guidance; it is intended to encourage researchers to think about their own work through an equity lens.

The framework starts with a brief history of equity in CTE, a description of the process of creating the framework, a list of vocabulary (we believe having a common language is critical), and a statement of the values that underlie the framework.

The rest of the framework is then organized by six stages of research: 1) project management; 2) research design, 3) measurement and data collection, 4) data analysis, 5) cost and resource equity, and 6) reporting and dissemination. In each section, we include a description of how to implement the stage with an equity-focused lens, with questions for researchers to consider and potential barriers. Throughout, we have included examples from current and future CTE research. We are looking for more examples, so people should feel free to reach out to us at jedmunds@serve.org to share how they are doing this work.

In creating summary products to go along with the framework, we identified six themes that cut across the different stages: ensure transparency, involve the community, develop diverse teams, take a systems approach, acknowledge and attend to bias, and demonstrate respect. These themes are summarized in an infographic.

How do you hope that people will use the framework?

We hope this will help start or further conversations among CTE researchers. We structured the framework around each stage of the research process, so anyone engaging in this work can find elements to incorporate or questions to consider individually and as a team, regardless of where they are in their work right now. For studies just getting off the ground, we did our best to illustrate how researchers can build an equity approach from the start of a project through its completion.

What are some examples of how the framework changed individual EWG members’ research practices?

Julie A. Edmunds (co-facilitator): Working on the framework has crystallized three high-impact equity-focused practices that I now try to infuse throughout my work. First, I pay much more attention to the role of systems in inequities. I try to look at upstream factors that might be causing disparities in educational outcomes as opposed to just documenting gaps that might exist between sub-groups. Second, when presenting those gaps (which we still do because it is useful information), I am much more conscious about how those gaps are displayed. For example, we focus on making sure that “White” is not considered the default category against which all others are compared. Third, we are creating processes to ensure that we share our findings with people who gave us the data. For example, we are sending practitioner-friendly products (such as briefs or infographics) to the school staff we interviewed whose insights formed the basis for some of our findings.

John Sludden (member): The framework has helped us think about our internal processes and keeps us focused on our audience, who we’re doing this for. I’m an analyst on the project, and I’ve been empowered to ask questions, conduct analyses, and present to our research partners at the New York City Department of Education. We’re currently thinking about ways to communicate findings to different audiences. At the moment, we’re working on a plan to share findings with principals of CTE high schools in New York City. Organizationally, we are also working on ways to directly engage students in the city, who know more about the system than we ever will. Similar to Julie, analytically, we have spent a lot of our time and attention on looking at the conditions under which students have not been well-served by the system, and ways that students may be better served by CTE.


This blog was produced by Corinne Alfeld (Corinne.Alfeld@ed.gov), program officer, NCER.

Family Access to Knowledge of their Rights in Transition and Guardianship for Students with Disabilities

The Individuals with Disabilities Education Act (IDEA), first enacted in 2004, specifies how public agencies should provide early intervention, special education, and related services to children with disabilities and delineates the rights of families to participate in meetings in which decisions are made on the evaluation, identification, and educational placement of their children. However, students with disabilities transition out of IDEA coverage when they graduate high school or reach age 21. This is a particularly crucial moment for students with intellectual and developmental disabilities (IDD).

Under IDEA, parental decision-making rights transfer to students at age 18 unless parents seek further guardianship. This complex issue requires understanding of the options and what they mean for the student. Alleviating any knowledge gaps is important for better transition outcomes for secondary students. In 2020, the U.S. Department of Education released an updated transition guide, A Transition Guide to Postsecondary Education and Employment for Students and Youth with Disabilities, for high school educators to better equip students on their transition into postsecondary education. The Department also released a brief to better inform students about their rights and the overall transition process as they prepare for postsecondary education and employment.

Allison Hall at the University of Massachusetts, Boston and her research team are exploring the role of special educators in informing youth and parents about transfer of rights and guardianship and its implications for transition outcomes for students with IDD. The project began with a review of the literature, a document review of state-level policies, and interviews with experts in the field of transition for students with IDD. These three initial research activities were followed by interviews with students, parents, and special educators on their experience during discussions of the transition process. We asked her to update us on the project and what they’ve learned so far. (Responses have been edited for brevity and clarity.)

NCSER: Please provide an update of where you are with the project.

Headshot of Allison Hall

Hall: Project staff members recently completed interviews in triads—which included a special educator, parent, and student with IDD—

focused on conversations about the transfer of rights and transition planning process for students after turning 18 through videoconferencing with participants from New York and Massachusetts. The research team, including researchers from Massachusetts Advocates Standing Strong and the Self Advocacy Association of New York State, is now analyzing the data for common themes. Once analyzed, the team will produce briefs and a short video highlighting key findings from the research.

Findings from qualitative data collection with experts in transfer-of-rights and transition planning yielded important information about the factors that influence how the transfer-of-rights conversations happen in special education settings:

  • School-based professionals have limited capacity and knowledge about the long-term impacts of guardianship
  • Schools are operating under an outdated paradigm of ableism—the tendency to intentionally or unintentionally presume incompetence as it relates to decision making for students with IDD
  • There is limited/lack of student engagement in transition planning and decision making
  • Schools are frequently guided by inadequate district and state policies

NCSER: Could you share any resources that may be useful to policymakers or parents?

Hall: Resources on our Institute for Community Inclusion website include an interactive map that describes the transfer-of-rights policies and laws in each state; plain language briefs for students about turning 18, transfer of rights, and alternatives to guardianship; and a brief that supports parents taking advantage of the transfer-of-rights process to position their youth with IDD for better transition outcomes.

NCSER looks forward to seeing the final results of Dr. Hall’s study on understanding how educators provide transfer-of-rights and guardianship information to families and the ways in which this information impacts parent expectations and student self-determination, each of which impacts student outcomes. Findings from this study can inform a future school-based intervention that tests strategies for more robustly incorporating transfer-of-rights discussions into the student-led transition planning process.

This blog was produced by Alysa Conway, NCSER student volunteer and University of Maryland, College Park graduate student with substantive contributions from NCSER program officers Akilah Nelson, Katie Taylor, and Amy Sussman.

Measuring In-Person Learning During the Pandemic

Some of the most consequential COVID-19-related decisions for public education were those that modified how much in-person learning students received during the 2020-2021 school year. As part of an IES-funded research project in collaboration with the Virginia Department of Education (VDOE) on COVID’s impact on public education in Virginia, researchers at the University of Virginia (UVA) collected data to determine how much in-person learning students in each grade in each division (what Virginia calls its school districts) were offered over the year. In this guest blog, Erica Sachs, an IES predoctoral fellow at UVA, shares brief insights into this work.

Our Process

COVID-19 has caused uncertainty and disruptions in public education for nearly three years. The purpose of the IES-funded study is to describe how Virginia’s response to COVID-19 may have influenced access to instructional opportunities and equity in student outcomes over multiple time periods. This project is a key source of information for the VDOE and Virginia schools’ recovery efforts. An important first step of this work was to uncover how the decisions divisions made impacted student experiences during the 2020-21 school year. This blog focuses on the processes that were undertaken to identify how much in-person learning students could access.

During 2020-21, students were offered school in three learning modalities: fully remote (no in-person learning), fully in-person (only in-person learning), and hybrid (all students could access some in-person learning). Hybrid learning often occurred when schools split a grade into groups and assigned attendance days to each group. For the purposes of the project, we used the term “attendance rotations” to identify whether and which student group(s) could access in-person school on each day of the week. Each attendance rotation is associated with a learning modality.

Most divisions posted information about learning modality and attendance rotations on their official websites, social media, or board meeting documents. In June and July of 2021, our team painstakingly scoured these sites and collected detailed data on the learning modality and attendance rotations of every grade in every division on every day of the school year. We used these data to create a division-by-grade-by-day dataset.

A More Precise Measure of In-Person Learning

An initial examination of the dataset revealed that the commonly used approach of characterizing student experiences by time in each modality masked potentially important variations in the amount of in-person learning accessible in the hybrid modality. For instance, a division could offer one or four days of in-person learning per week, and both would be considered hybrid. To supplement the modality approach, we created a more precise measure of in-person learning using the existing data on attendance rotations. The new variable counts all in-person learning opportunities across the hybrid and fully in-person modalities, and, therefore, captures the variation obscured in the modality-only approach. To illustrate, when looking only at the time in each modality, just 6.7% of the average student’s school year was in the fully in-person modality. However, using the attendance rotations data revealed that the average student had access to in-person learning for one-third of their school year.

Lessons Learned

One of the biggest lessons I learned working on this project was that we drastically underestimated the scope of the data collection and data management undertaking. I hope that sharing some of the lessons I learned will help others doing similar work.

  • Clearly define terminology and keep records of all decisions with examples in a shared file. It will help prevent confusion and resolve disagreements within the team or with partners. Research on COVID-19 in education was relatively new when we started this work. We encountered two terminology-related issues. First, sources used the same term for different concepts, and second, sources used different terms for the same concept. For instance, the VDOE’s definition of the “in-person modality” required four or more days of access to in-person learning weekly, but our team classified four days of access as hybrid because we define “fully in-person modality” as five days of access to in-person learning weekly. Without agreed-upon definitions, people could categorize the same school week under different modalities. Repeated confusion in discussions necessitated a long meeting to hash out definitions, examples, and non-examples of each term and compile them in an organized file.
  • Retroactively collecting data from documents can be difficult if divisions have removed information from their web pages. We found several sources especially helpful in our data collection, including the Wayback Machine, a digital archive of the internet, to access archived division web pages, school board records, including the agenda, meeting minutes, or presentation materials, and announcements or letters to families via divisions’ Facebook or Twitter accounts.
  • To precisely estimate in-person learning across the year, collect data at the division-by-grade-by-day level. Divisions sometimes changed attendance rotations midweek, and the timing of these changes often differed across grades. Consequently, we found that collecting data at the day level was critical to capture all rotation changes and accurately estimate the amount of in-person learning divisions offered students.

What’s Next?

The research brief summarizing our findings can be downloaded from the EdPolicyWorks website. Our team is currently using the in-person learning data as a key measure of division operations during the reopening year to explore how division operations may have varied depending on division characteristics, such as access to high-speed broadband. Additionally, we will leverage the in-person learning metric to examine COVID’s impact on student and teacher outcomes and assess whether trends differed by the amount of in-person learning divisions offered students.


Erica N. Sachs is an MPP/PhD Student, IES Pre-doctoral Fellow, & Graduate Research Assistant at UVA’s EdPolicyWorks.

This blog was produced by Helyn Kim (Helyn.Kim@ed.gov), Program Officer, NCER.

NCER’s Investments in Education Research Networks to Accelerate Pandemic Recovery Network Lead Spotlight: Dr. Susan Therriault, RESTART Network

We hope you enjoyed yesterday’s network lead spotlight! Today, we would like to introduce Dr. Susan Therriault, director, K–12 Systemic Improvement Portfolio at the American Institute for Research. Dr. Therriault’s network, the PreK-12 Research on Education Strategies to Advance Recovery and Turnaround (RESTART) Network, aims to coordinate activities across research teams and provides national leadership on learning acceleration and recovery from pandemic-induced learning loss, sharing findings from the network with education agencies across the United States. Happy reading!

 

NCER: What are the mission and goals of the PreK–12 RESTART (Research on Education Strategies to Advance Recovery and Turnaround) Network? 

Dr. Therriault: The PreK–12 RESTART Network is an opportunity to develop a coherent and connected research community that speaks directly to the needs of policymakers, leaders, and practitioners. The network focuses on identifying and disseminating evidence-based strategies aligned with the needs of policymakers, leaders, and educators who are serving and supporting accelerated student recovery efforts. This requires the network to identify critical needs of the field and support the research community in developing coherent and coordinated research strategies that build evidence for practices that ensure student recovery—especially among students who have disproportionately struggled in the pandemic context. The PreK–12 RESTART Network will achieve this by need sensing, synthesizing evidence, and building a community that makes meaningful connections between the research community and policymakers, leaders, and educators.

NCER: Why is the PreK–12 RESTART Network important to you? 

Dr. Therriault: The PreK–12 RESTART Network is important to me because, as a researcher, I have watched how the pandemic and subsequent aftershocks of the pandemic have created disruptions to our lives and our public pre-K to 12 education system. The pandemic created fragmentation and division as communities responded and supported individuals in a context marked by social distance and separation. While there are many common challenges across communities, limited social connection affected our ability to share evidence-based solutions and to equitably address the needs of all members of our communities, especially those communities most adversely impacted by COVID-19, including Black and Latinx communities and those marked by poverty and housing and food insecurity.

NCER: How do you think the PreK–12 RESTART Network will impact the pre-K to 12 community?

Dr. Therriault: The PreK–12 RESTART Network is an opportunity to develop a coherent and connected research community that speaks directly to the needs of policymakers, leaders, and practitioners. The key differentiator of the network is that it is purposefully designed to assess needs and engage the research community in providing insight and building evidence for solutions to address those needs.

The network has an important role to play in drawing researchers together to develop measurement solutions and build consensus for approaches to conducting and making meaning of research so that it informs the field. These solutions will be shared with the field to create a more coherent research agenda informed by needs.

The network will ensure that policymakers, leaders, and educators are able to easily access network evidence syntheses and research-team findings through multiple communication formats. Information will be shared through actionable guidance and recommendations purposefully designed for these audiences. A combination of strategies will ensure accessibility. These include digital tools and dashboards that school leaders can easily use to adapt to their circumstances and access to evidence-based strategies by offering recorded webinars, tutorials, videos, and other learning formats.

NCER: What’s one thing you wish more people knew about recovery in pre-K to 12 education? 

Dr. Therriault: The magnitude of the challenge of pandemic recovery in the pre-K to 12 education system is vast and will require new ways of approaching education and support for students. In turn, with the investment of American Rescue Plan funds in schools, this will likely lead to evidence-based innovation and deeper understanding of how to design an education system, district, and school to meet student needs.

The needs of students are varied and highly connected to the experience during the pandemic and after the pandemic; thus, family and community factors are highly relevant and critical to understanding student needs and strategies to address these needs. Recently released NAEP scores provide evidence of the variation and suggest more-significant losses in mathematics and English language arts for students living in low-income households compared to their peers who are not. Further, students living in low-income households were more likely to report not having a place to do work or access to a computer or adequate uninterrupted time compared to their peers. These are critical factors in a remote and even a hybrid learning environment. These differences in experience exacerbate differences in outcomes during the pandemic.

NCER: What are some of the biggest challenges to recovery in pre-K to 12 education? 

Dr. Therriault: The amount of need among students and their families and the fatigued pre-K to 12 education system workforce are the biggest challenges to recovery. Adding to this challenge is the focus on expanding learning time through summer school or longer school days in an effort to accelerate learning. This requires teachers and leaders at a time when, like many of us, they are experiencing burnout.

Finally, we know that most students suffered learning loss and more during the pandemic. Supporting students emotionally as well as academically is necessary for recovery. While many schools have provided emotional support to students prior to the pandemic, the current need is far greater than schools have experienced. This will require extensive outreach to community support and services and additional interventions.

NCER: What are some effective ways to translate education research into practice so that your work will have a direct impact on states, districts, and schools? 

Dr. Therriault: There are several ways we plan to support the translation of research to practice through the PreK12 RESTART Network. These include:

  • Understanding and sharing needs of the field.
    • Conducting needs assessments of the field to examine the evolving needs and share these with the research community.
  • Identifying, sharing, and amplifying evidence-based strategies that respond to the needs of the field.
    • Exploring existing and emerging research to identify evidence-based strategies and interventions that align evidence syntheses with the needs of the field.
    • Providing actionable guidance and recommendations that can be easily implemented across different schools and are customized based on need.
    • Creating digital tools that school leaders and educators can use to adapt to their circumstance.
  • Building a coherent and coordinated research community focused on pandemic recovery research.
    • Connecting and building consensus among researchers through convenings and solutions working groups that address challenges to conducting research in the pandemic context.
    • Empowering research teams to build studies that align with needs in the field through meaningful connections with policymakers, leaders, educators, and other members of the research community.
    • Supporting engagement and preparation of early-career researchers through trainings and networking opportunities.

NCER: What are some barriers to the uptake of the research outcomes by these organizations?

Dr. Therriault: One of the critical barriers to uptake is timing. States, districts, and schools cannot wait for findings and results—they must act to support the students they have in their classrooms right now. The syntheses and reviews of prior research will help point educators in the direction of interventions and other supports that have a strong evidence base. The researchers participating in the RESTART Network will be supported in rapidly sharing and disseminating findings over the course of their studies to inform decision-making about how best to help students’ academic recovery.


Thank you for reading our conversation with Dr. Susan Therriault! We hope you’ve enjoyed getting to know NCER’s network leads throughout our grantee spotlight series. Let us know your thoughts on the series on Twitter at @IESResearch.

NCER’s Investments in Education Research Networks to Accelerate Pandemic Recovery Network Lead Spotlight: Dr. Thomas Brock, ARCC Network

We hope you enjoyed the first NCER network lead spotlight! Today, we would like to introduce Dr. Thomas Brock, director of the Community College Research Center. Dr. Brock’s network, the Accelerating Recovery in Community Colleges (ARCC) Network, aims to provide timely, actionable research from the pandemic that policymakers and practitioners can use to help community colleges recover from the challenges introduced by the COVID-19 pandemic. Happy reading!

NCER: What are the mission and goals of the Accelerating Recovery in Community Colleges Network?

Dr. Brock: The primary goal of the ARCC Network is to provide timely, actionable research that policymakers and practitioners can use to help community colleges recover from the challenges introduced by the COVID-19 pandemic. These include steep drops in enrollment—particularly for students of color and male students—and learning losses associated with illness, stress, and challenges of online learning.

 

NCER: Why is the ARCC Network important to you?

Dr. Brock: The ARCC Network is important because community colleges are important. They enroll about one-third of all undergraduate college students in the U.S., including many who are from low-income backgrounds and the first in their families to attend college. The nation needs strong community colleges to help students advance educationally and economically. The nation also needs community colleges to prepare workers and support the economy in essential fields such as health care, information technology, construction trades, and manufacturing.

NCER: I understand that you had a central role in establishing the research networks grant program at IES. What is your view of a research network, and how does it differ from a traditional education research project?

Dr. Brock: There is an old adage that the whole is greater than the sum of its parts. With the research networks, IES intends to generate a body of work on a critical education problem or issue that is more impactful than an individual research project is likely to generate. This is because members of a research network come together regularly to discuss their ideas, tackle common methodological challenges, share data collection tools, and make sense of their emerging findings. They think about how to distill, align, and communicate research results from the early stages rather than as an afterthought. This benefits policymakers and practitioners, who look to researchers for insights and guidance. It also benefits the research community by building consensus on what has been learned and what new questions need to be addressed.

NCER: How do you think the ARCC Network will impact our nation’s community colleges?

Dr. Brock: Our hope is that the ARCC Network will help policymakers to be attentive to the needs of community colleges and shed light on the populations and places that need the most help. We also hope that the network will help identify promising policies and practices to promote rapid recovery.

NCER: What are some of the biggest challenges to recovery in community colleges?

Dr. Brock: Community colleges are largely funded based on enrollment. To date, the decline in enrollment has not led to too much reduction in academic programs or services because of the Higher Education Emergency Relief Fund (HEERF), authorized by Congress. HEERF funding ends after the 2022–23 academic year. If enrollments do not rebound quickly, community colleges will have to make significant cuts. This could lead to a downward spiral in which even fewer students enroll or persist, because they do not find the courses or services that they need.

Another challenge to recovery is learning loss. We know from the National Assessment of Educational Progress that there have been declines in reading and math achievement in K–12 schools during the pandemic. As these students mature and enter postsecondary education, they may be less well prepared for college-level work. Community colleges have made significant reforms to developmental education programs in recent years but will need to do more to ensure entering students succeed in college-level courses and make progress toward their academic and career goals.

Finally, we know that the pandemic has taken a severe toll on physical and mental health. Community colleges will need to find ways to reduce stress and promote wellness for everyone in their campus community–students, faculty, and staff.

NCER: What are some effective ways to translate education research into practice so that your work will have a direct impact on states and community college systems? What are some barriers to uptake of research outcomes by these organizations?

Dr. Brock: The ARCC Network will actively disseminate the research findings produced by individual research teams and by our national scan of community college enrollments and recovery practices. We will build a website that functions as an information hub for the most recent enrollment trends and reliable evidence on recovery strategies. We will conduct interactive workshops and webinars for state and local community college leaders and staff who are interested in learning from and adapting research-based practices to support pandemic recovery. We will use our connections with national organizations, like the American Association of Community Colleges and Achieving the Dream, and social media to ensure we reach a broad audience.

NCER: Are there some generalizable tools or lessons learned that are likely to come out of this network project that you think will benefit the education research community as a whole?

Dr. Brock: Yes. One area of focus for ARCC researchers, for example, is how to design and deliver effective online learning. Prior to the pandemic, most research on online learning in community colleges indicated it was not as effective as in-person instruction, but many colleges have upped their game with improved technology and better training and support for faculty who teach online. We have also seen from the pandemic that online learning benefits some students who might not otherwise attend community college, including students who live far from campus (especially in rural areas) or who are juggling demands of work and parenting. We hope to reframe the research debate so that it is less about online versus in-person instruction and more about how to provide online instruction most effectively to students who prefer this modality. We expect the lessons and tools from the ARCC Network will be broadly relevant to community colleges and may be adapted to other education sectors.


Thank you for reading our conversation with Dr. Thomas Brock! Come back tomorrow for our final grantee spotlight!