Inside IES Research

Notes from NCER & NCSER

Career and Technical Education Month®: Improving Outcomes for Secondary Students with Disabilities

February marks Career and Technical Education Month®, a public awareness campaign that highlights and celebrates career and technical education (CTE). CTE programs emphasize career preparation, skill trades, applied sciences, and modern technologies. CTE coursework integrates academic knowledge with post-secondary pathways and careers by directly preparing middle and high school students with coursework related to high-demand industries.

To encourage research to improve career readiness skills and transition outcomes for students with or at risk for disabilities, the National Center for Special Education Research (NCSER) competed a special topic on Career and Technical Education for Students with Disabilities in FY 2019 and FY 2020. A previous blog post focused on the FY 2019 grant. In FY 2020, NCSER awarded two new research grants through this special topic:

CTE Teachers and Long-Term Outcomes for Students with Disabilities

The purpose of this exploratory project is to assess CTE teacher effectiveness for students with disabilities. Principal Investigator Dan Goldhaber at the University of Washington and his colleagues (co-PIs Kristian Holden and Roddy Theobold) will estimate CTE teacher effectiveness using high school attendance, GPA, persistence, and graduation probability as outcomes. They will also consider longer-term outcomes including college enrollment and employment, as well as whether CTE teacher effectiveness varies according to teacher licensure, pathway into teaching, and prior work experiences.

Supported College and Career Readiness for Secondary Students with Emotional and Behavioral Problems

In this study, Principal Investigator Lee Kern at Lehigh University and her colleagues (co-PIs Chris Liang and Jennifer Freeman) will develop and pilot test a multi-component program, Supported College and Career Readiness, that augments typical school-based college and career readiness activities (such as those associated with CTE). The research team aims to further support high school age students with or at risk for emotional and behavioral disorders (EBD), whom research suggests are insufficiently benefiting from college and career readiness activities. As a result, students with or at risk for EBD are frequently unprepared for career or postsecondary education pathways.

We asked each Principal Investigator to share the motivation for studying this research topic and what makes their study unique and impactful.

What inspired you to study this research topic?

Photo of Dan GoldhaberDr. Goldhaber: At CALDER, we have contributed to a large body of research on the impacts of math and ELA teachers on students. This body of evidence suggests that the quality of the teacher workforce is the most important schooling factor influencing students test and non-test outcomes. But we were surprised to find that, despite growing interest in CTE, there is a surprising lack of empirical research on CTE teacher contributions to student learning. We think this is a particularly important issue for students with disabilities for two reasons. First, CTE teachers have different pathways to teaching than academic teachers, and these pathways may provide less pedagogical preparation for the unique needs of students with disabilities. And second, students with disabilities are significantly more likely to be enrolled in these courses relative to students without disabilities. As such, this project lies at the intersection of our prior work on academic teachers, and our recent work on CTE participation for students with disabilities.

Photo of Lee Kern

Dr. Kern: Spending time with high school age students with emotional and behavioral problems brought me to truly understand how little they consider their futures. And, even when they have goals and dreams, they see scarce connections between what they are being asked to do in high school and realizing their goals. It occurred to me that finding ways to strengthen this connection might be an avenue not only to better prepare students for post-high school life, but also to reduce dropout. Indeed, the need to better prepare youth for life after high school has been increasingly embraced, especially in the last decade, evidenced in part by the adoption of career and college readiness (CCR) standards in almost every U.S. state. I witnessed many impressive CCR efforts and programs in high schools, yet it appeared that students with emotional and behavioral problems were failing to access these school-based CCR supports, some of which did not seem well aligned with their needs. With support from a Lehigh University faculty grant, I began to research exactly how this subset of students regarded, accessed, and benefitted from CCR activities and supports. This research underscored the need to supplement important components of CCR to better align with the needs of students with emotional and behavioral problems. Our goal in the current project is to develop and evaluate an intervention package that supplements high school CCR activities to better prepare students with emotional and behavioral problems for community, college, and/or career.

What makes your project unique and exciting? 

Dr. Goldhaber: This project will use unique data from Washington state that allows us to track students with disabilities in high school, postsecondary education, and employment. Access to long-term outcomes is both unique and important given that test scores are not likely to be a very good measure of the contributions that CTE teachers make toward student education. Our project is also closely focused on CTE workforce issues. The staffing of CTE courses is quite challenging—in Washington state, for instance, over half of all CTE teachers have not completed the state’s teacher licensure requirements and hold limited CTE licenses. Thus, we believe the work will garner a lot of attention from policymakers.

Dr. Kern: I am especially enthusiastic about this project because it targets areas in which empirical research tells us that students with emotional and behavioral problems have insufficient CCR skills, perceptions, and knowledge. I am also fortunate to work with my co-PIs, Drs. Freeman and Liang, who bring diverse and unique expertise in the areas of CCR assessment, CCR counseling, racial/ethnic identity development, and more. I am also optimistic about the feasibility of this project because it capitalizes on existing school resources in the form of school-based CCR programs. So, rather than adding programs or interventions, it adapts and expands existing CCR components.

Tell us how your research results could possibly shape CTE-related policy.

Dr. Goldhaber: In broad terms, we do not know much about the impact of CTE teacher quality, so our study has the potential to inform broadly the myriad ways we think about the heterogeneity of CTE teacher effects. But, more specifically and narrowly, we will be looking at the connections between CTE effects and CTE teacher pathways, which should inform policies and practices around CTE preparation and licensure.

Dr. Kern: We hope that our efforts will shape future CCR policy. Our aim is to build the evidence-base in the area of CCR supports that address the needs of students at risk for emotional and behavioral problems. Ultimately, we would like to see federal policies that guarantee at-risk students in all high schools in the U.S. receive comprehensive, evidence-based CCR interventions and supports that fully prepare them for life after high school.

For more information on CTE and students with disabilities, the Association for Career and Technical Education (ACTE) and the National Technical Assistance Center on Transition (NTACT) provide useful resources. In addition, ACTE, NTACT, and Penn State University’s Workforce Education program hosted a five-part webinar series in 2019 about programs, practices, and partnerships that support students with disabilities in CTE. This series can be viewed here.

This blog was authored by Alice Bravo (University of Washington), IES intern through the Virtual Student Federal Service, and Akilah Nelson, Program Officer at NCSER. For more information about the Career and Technical Education for Students with Disabilities topic area, contact Akilah Nelson.

The views expressed by the investigators do not necessarily reflect those of IES.

Supporting Teachers and their English Learners during Online Learning

Under an IES grant, Drs. Leslie Babinski, Steve Amendum, Steve Knotek, and Marta Sánchez are evaluating the impact of the Developing Consultation and Collaboration Skills (DCCS) program. The DCCS program is a year-long professional development intervention designed to support English-as-a-Second Language (ESL) and classroom teachers to develop their skills in collaboration, literacy instruction, and parent outreach and engagement for their Latino students. Unfortunately, COVID-19 disrupted their study, but rather than calling it quits, the research team used it as an opportunity to explore how teachers are responding to a new normal and learned some important lessons along the way.

 

Last spring, we were in the middle of conducting a randomized control trial in elementary schools working with teaching teams of kindergarten, first grade, and ESL teachers when COVID-19 hit, and schools pivoted to online instruction.

As we paused our efficacy trial and adapted our plans, we continued to stay engaged with teachers despite the rapidly shifting circumstances. We realized that the foundational skills central to our intervention—using high-impact instructional strategies for English learners (ELs), building on families’ cultural wealth, promoting collaboration between ESL and classroom teachers, and ongoing, supportive implementation coaching—could be modified and adapted for different contexts and modes of delivery. These ongoing interactions with teachers provided us with an up-close look at their experiences as they connected with families remotely and adapted to this new and unfamiliar way of teaching. We learned about the challenges, successes, and possibilities for providing online instruction for young ELs.

While online teaching during the current school year looks and feels very different from the emergency shift to remote teaching from last spring, we learned important lessons from the experiences of teachers and students.

 

First, we learned that our implementation coaching model could be productive in a virtual format. In fact, teachers were eager to engage in reflective conversations despite the demands on their time and the stress of the pandemic. Each of the school teams continued to meet with our implementation coach. Together, they found new ways to scaffold instruction for ELs during online learning. Teachers supported one another in finding creative ways to use technology to communicate with families and provide support that reached beyond academics. They also created lessons that intentionally adapted instruction to reach ELs in their classrooms. For example, embedded in the videos the teachers created, they continued to use our core instructional strategies, such as previewing academic vocabulary before reading a new text, supplementing student background knowledge related to the content, providing sentence frames for both oral and written participation, and selecting texts related to student cultures and families.

 

Second, it is clear that there are serious inequities in access to online schooling. In our study, even as teachers made the commendable efforts described above, they also reported that more than half of their EL students required a device from the school district, while one-third of the families also needed a Wi-Fi hotspot to access online learning. This rate is considerably higher than a Pew Research Center poll that found that one in five parents reported difficulties with online learning due to lack of a computer or Wi-Fi access, highlighting the fact that many English Learners may have limited access to the technology necessary for online learning.

 

Third, online learning requires a partnership between teachers, families, and other caregivers. Even with a device and internet access, many young children and their families had difficulty logging onto their school accounts and navigating the technical aspects of the learning platforms for online schooling. Teachers reported that about half of the EL students in their classrooms had a parent or guardian who was able to help them with schoolwork at home. About 20% had help from another adult or a sibling. In one case, a teacher described a family in which a kindergarten student received support for online schooling from her brother in second grade. In our study, support at home was critical for student participation in online lessons. Our work with teachers during remote teaching and learning highlighted how parents and other caregivers are important advocates for their children’s education and are eager to partner with teachers and schools to help their children succeed.

 

Looking to the future, it is clear that instruction for ELs will need to focus on equitable access to high-quality instruction, whether online or in-person. Access to devices and Wi-Fi is essential, not only for online teaching and learning, but also for extending learning into the home. Finally, we note that teacher collaboration and coaching can be effectively adapted for an online environment and is an essential component in providing support to teachers for high-quality instruction for ELs.

Although we could not have anticipated the value of continuing to work with teachers during the pivot to remote instruction, we are grateful for the experience and all that we have learned during the process.

 

For more information, see this EdWeek article and this interview on The TakeAway.


Dr. Leslie Babinski is an associate research professor in the Sanford School of Public Policy and the Director of the Center for Child and Family Policy.

Dr. Steve Amendum is a professor at the School of Education at the University of Delaware.

Dr. Steve Knotek is a professor in the School of Education at the University of North Carolina at Chapel Hill.

Dr. Marta Sánchez is an associate professor in the Watson College of Education at the University of North Carolina Wilmington.

This is part of a series of blog posts focusing on conducting education research during COVID-19.

 

New Project Exploring Adult Basic Skills in STEM-Related Postsecondary CTE®

In celebration of CTE® (career and technical education) month, we would like to highlight the launch of an NCER project that aims to help us understand how to best support adults seeking additional CTE education and training.

Through their exploratory project, Literacy, Numeracy, and Problem-Solving Skills in Technology-Rich Environment in the STEM-Related Subbaccalaureate Programs in the United States, researchers will use a mixed-method design to gather information about the distribution of basic skills (literacy, numeracy, and problem solving) for adults in STEM occupations and students enrolled in STEM-related sub-baccalaureate programs at community colleges. Their goal is to help identify the needs of students and the programming practices at community colleges that may promote basic skill development in STEM programs.

The team will be leveraging data from the Program for the International Assessment of Adult Competencies (PIAAC) to survey the distribution of skills and abilities across a nationally representative sample of adults in STEM fields. They will also be collecting primary data from adults and programs in multiple locations, including Indiana, Ohio, and Washington states.

 

 

To help inform the public about their project, the researchers have created a short YouTube video for the public. This project began in July 2020 and may have initial results ready as early as late 2021.

 


Written by Meredith Larson (Meredith.Larson@ed.gov), Program Officer for Postsecondary and Adult Education, NCER.

Representation Matters: Exploring the Role of Gender and Race on Educational Outcomes

This year, Inside IES Research is publishing a series of interviews showcasing a diverse group of IES-funded education researchers that are making significant contributions to education research, policy, and practice.

 

The process of education transmits sociocultural values to learners in addition to information and knowledge. How individuals are represented in curricula and instructional materials can teach students about their place in the world. This can either perpetuate existing systemic inequalities or, conversely, provide a crucial counternarrative to them. With an exploration grant from IES, Anjali Adukia (University of Chicago) and Alex Eble (Teachers College, Columbia University) are exploring how  representation and messages about gender and race in elementary school books may influence student’s education outcomes over time. The researchers will develop and use machine-learning tools that leverage text and image analysis techniques to identify gender- and race-based messages in commonly used elementary-school books.

 

Interview with Anjali Adukia, University of Chicago

Tell us how your research contributes to a better understanding of the importance of diversity, equity, and/or inclusion in education.

In my work, I seek to understand how to reduce inequalities such that children from historically (or contemporaneously) marginalized backgrounds have equal opportunities to fully develop their potential. I examine factors that motivate and shape behavior, preferences, and educational decision-making, with a particular focus on early-life influences. Proceeding from the notion that children are less likely to be able to focus on learning until their basic needs are met, my research uses both econometric methods and qualitative approaches to understand the specific roles different basic needs play in making these decisions. My research, for example, has explored the role of safety and health (sanitation, violence), economic security (road construction, workfare), justice (restorative practices), and representation (children’s books), particularly for marginalized groups.

 

As a woman and a minority, how has your background and experiences shaped your career?

My research is informed and influenced by my own experiences. When I was a child, I never understood why there weren’t more characters that looked like me or when there were, why they had such limited storylines. For me personally, the motivation underlying our IES-funded project was borne out of my lived experience of always searching for content that reflected who I was. I think of representation as a fundamental need: if you don’t see yourself represented in the world around you, it can limit what you see as your potential; and similarly, if you don’t see others represented, it can limit what you see as their potential; and if you only see certain people represented, then this shapes your subconscious defaults.

It was a real watershed moment when I realized that academia allowed me to pursue many of my larger goals in life, in which I hope to meaningfully improve access to opportunities and outcomes for children – especially those from disadvantaged backgrounds. I hope to accomplish this in various ways, paying forward the many kindnesses generously given to me by: (1) producing rigorous policy-relevant evidence that expands our understanding of big questions and opens new avenues for inquiry; (2) translating my research such that it helps inform policymakers and practitioners in the design of school policies and practices; (3) understanding issues with a depth and sophistication that comes from “on-the-ground” insights, knowledge cultivated in multiple disciplines using different methodologies, introspection, humility, and courage; (4) directly working with government agencies, non-profit organizations, and community groups to positively inform policies; (5) contributing to the larger public discourse; and (6) by training and advising students to have the fortitude to ask hard questions, to be able to defend different perspectives on issues, to learn that knowledge brings more questions than answers, and to be willing to take risks and fail (and in the process, I will certainly learn more from them than they will ever be able to learn from me).

 

What has been the biggest challenge you encountered and how did you overcome the challenge?

Life is always filled with challenges, but one challenge starting from when I was young was to feel comfortable in my own skin and to find legitimacy in my own voice. I grew up as an Indian-American daughter of Hindu immigrants in a rural, predominantly white and Christian setting. I was different from the other kids and did not always feel like I fit in. I remember literally trying to erase my skin hoping that it would make it lighter. I found the helpers, as my parents (and Mr. Rogers) would suggest, and tried to focus on the voices that lifted me up – my family, teachers, other mentors, those friends who loved me no matter my differences. My mother always told me to find the kindness, the good, the love in people; to find the common ground and to embrace and learn from the differences. I surrounded myself with love, focusing on what I had and on what I could do rather than what society was telling me I couldn’t do. I turned to concentrating on things that mattered to me, that drove me. I don’t think there is a single challenge in life that I overcame alone. I have been very lucky, and I am deeply grateful for the many gifts in my life, the many loved ones – family, friends, colleagues, mentors, healthcare workers – who have lifted me up, and the opportunities that came my way.

 

How can the broader education research community better support the needs of underrepresented, minority scholars?

The notion of what is considered to be an important question is often driven by the senior scholars in a field, for example, the people considered to be “giants.” Demographically, this small set of leading scholars has historically consisted of people from the most highly represented groups (particularly in economics). And because the field is thus shaped mainly by researchers from a “dominant” group background, the key questions being pursued may not always reflect the experiences or concerns of people from underrepresented backgrounds. Education research has pockets where these different perspectives are being considered, but it can continue to evolve by becoming more open to approaches thought to be less traditional or to questions not typically asked (or asked from a different point of view). Expanding the notion of what is considered important, rigorous research can be difficult and cause growing pains, but it will help expand our knowledge to incorporate more voices.

 

What advice would you give to emerging scholars from underrepresented, minority backgrounds that are pursuing a career in education research?

Keep a journal of questions that arise and topics that pique your curiosity and interest. Soon, you will find questions in the fabric of everyday life, and you will start to articulate the wonder you see in the world around you and what inspires you to action, to understand the universe further. I find that when I return to past writings and journal entries, I am reminded of questions that have ignited my fires and see some of the common themes that emerge over time. Find your voice and know that your voice and views will grow and evolve over time. There are so many interesting and important questions one can pursue. Most importantly, you have to be true to yourself, your own truth. Find circles of trust in which you can be vulnerable. Draw strength from your struggle. There is deep truth and knowledge within you.

 


Dr. Anjali Adukia is an Assistant Professor at the University of Chicago Harris School of Public Policy and the College.

This interview was produced by Christina Chhin (Christina.Chhin@ed.gov), Program Officer, National Center for Education Research.

 

Better Reading Comprehension When You Know That You Don’t Know

The more you already know about a topic, the easier it may be to comprehend and learn from texts about that topic. But knowledge has to start somewhere. So how can we help students learn from texts when they may have low background knowledge?

In their exploratory study, researchers from ETS found that lack of knowledge is not necessarily a barrier to comprehension. Rather, they suggest that students who can identify their lack of background knowledge are more likely to comprehend and learn new information than students who do not acknowledge they lack background knowledge. In other words, knowing that you might not know may lead to better outcomes.

To determine the role of background knowledge, the researchers pretested middle and high school students’ background knowledge through questions related to topics the students may have some but not complete knowledge of, such as ecology, immigration, and wind power. The pretest included an “I don’t know” option, along with correct and incorrect responses.

Students then took a scenario-based assessment in which they read multiple sources about each of the topics. This type of assessment mirrors real-world learning by encouraging readers to build their own interpretations of a topic, which helps researchers determine whether students comprehend what they read.

They found that students who selected “I don’t know” when answering background knowledge questions had better understanding of the content than those who provided wrong answers on these questions. In fact, students who selected “I don’t know” rather than answering incorrectly were nearly three times as likely to learn from sources that provided the correct information than students who had answered the pretest incorrectly. Students who selected “I don’t know” may also learn more than students who had a comparable level of weak background knowledge. The researchers suggest that the “I don’t know” readers may have set different reading goals prior to engaging with the sources than those who guessed incorrectly.

 

Possible Implications for Teaching and Learning

The results from this work support the idea that having and building background knowledge is key. Thus, teachers may want to assess existing knowledge and address knowledge gaps prior to instruction.

Teachers may also want to provide an “I don’t know” option or options that allow students to rate their level of certainty. Doing so may help teachers distinguish between students who recognize their own gaps in knowledge from those who may not be aware that they are wrong or that they simply do not know. This latter group of students may need more help in determining the accuracy of their judgments or may have incorrect knowledge that could interfere with learning.

The researchers further suggest that teachers may want to go beyond the role of background knowledge by teaching students how to set appropriate reading goals and use strategic reading approaches to learn new facts or correct existing misunderstandings.

 


The research reported here was conducted under NCER grant R305A150176: What Types of Knowledge Matters for What Types of Comprehension? Exploring the Role of Background Knowledge on Students' Ability to Learn from Multiple Texts.

This blog was written by Dr. Meredith Larson. Contact her for more information about this project.