Inside IES Research

NCSER celebrates Middle Level Education Month this year by highlighting some of our current research projects aimed at supporting the literacy skills of middle school students with and at risk for disabilities. Data from the National Assessment of Education Progress (NAEP) indicate that in 2019, 68% of eight graders with disabilities scored “Below NAEP Basic” in reading compared to 22% of students without disabilities, a gap that had grown larger over the previous decade. The urgent need to improve the reading skills of middle school students with disabilities has led to some important NCSER-funded research projects.  

Sharon Vaughn and Leticia Martinez at the University of Texas, Austin and Jeanne Wanzek at Vanderbilt University, along with their colleagues, are testing the efficacy of Promoting Adolescents' Comprehension of Text (PACT), a fully developed reading comprehension intervention for middle school students with evidence of efficacy for students without disabilities. The current study will be the first efficacy trial of PACT that focuses specifically on students with disabilities. PACT is a text- and inquiry-based reading comprehension intervention with instructional supports for teachers and material for students in general education classrooms. In this large-scale, multi-site study, the team is using a randomized controlled trial in approximately 80 eighth-grade social studies classrooms, each with at least two students with disabilities. The team will examine the intervention’s impact on student reading and social studies outcomes, whether the impact differs depending on level of teacher fidelity or by student characteristic, and the intervention’s cost-effectiveness over the typical expenditures.

PACT is also being used as the evidence-based literacy intervention in a project designed to support middle school teachers’ knowledge and practices and improve reading and content area knowledge among students with disabilities. Jade Wexler at University of Maryland, College Park and Elizabeth Swanson at the University of Texas, Austin, along with their colleagues, are developing and testing a model for instructional leaders to provide ongoing support to content-area middle school teachers as they implement PACT. More specifically, the team will be developing an intervention package that includes a multi-stage, adaptive intervention coaching model to systematically tailor support to teachers as they implement Tier 1 literacy practices (in this case, the PACT intervention) in their content areas (English language arts, social studies, and science) to improve reading outcomes for students with disabilities. The package will also include a professional development program to train instructional leaders on how to implement the coaching model with teachers effectively.

Deborah Reed at the University of Iowa and her colleagues are also focusing on integrating literacy instruction with content-area instruction, but this team is focused on the Tier 2 level. They are developing and testing an intervention for middle school students with or at risk for reading disabilities who need support in literacy and text-based content in science and social studies. Pairs of students will alternate reading science and social studies texts, with specific academic vocabulary language, on a digital platform. This platform will provide scaffolded support and allow opportunities for individual work in building related reading and writing skills. Passages on each science or social studies topic repeat 85% or more of the unique words but in different contexts to support students’ ability to recognize and read the academic vocabulary. The overall aim of the intervention is to improve student literacy as well as science and social studies performance.

Marcia Barnes and her team at Vanderbilt University are testing the efficacy of a reading comprehension intervention, Connecting Text by Inference and Technology (Connect-IT), with middle school students with or at risk for reading disabilities. Connect-IT, developed with a prior IES grant, was designed to improve inference-making and reading comprehension in this population. In the current study, the research team will examine the impact of the intervention on students in grades 6-8 who did not pass their state English Language Arts test and who have demonstrated difficulties in reading comprehension. They will compare the efficacy of the intervention as implemented in small groups by a teacher, individual implementation of the intervention through computer software with project interventionist supervision, and the school’s business-as-usual classes. The study aims to determine the effect of each version of the intervention on student inference-making abilities and reading comprehension, as well as whether various student skills (such as vocabulary, word reading, attention, and anxiety) may moderate the impact of the interventions. The interventions’ cost-effectiveness will also be evaluated.

Focused on more intensive intervention for middle school students with or at risk for reading  disabilities, Mary Beth Calhoon at the University of Miami is testing the efficacy of a 2-year implementation of the Adolescent Multi-Component Intensive Training Program (AMP-IT-UP), which was previously tested after 1 year of implementation through an IES-funded grant. The intervention uses direct, systematic, explicit instruction (in phonological decoding with comprehension, spelling, and fluency) and cognitive strategy instruction (including use of cues and anchors), combined with reciprocal peer-mediated instruction. Dr. Calhoon’s research team is conducting a randomized controlled trial with middle school students with or at risk for reading disabilities who are still reading at the third-grade level or below. They will examine the impact after 2 years of intervention as well as 1 year after intervention has ended to determine its effect on student word recognition, spelling, fluency, and comprehension skills.

We look forward to reporting on the results of these studies as these teams complete their work in the years to come.

The blog was authored by Amy Sussman (Amy.Sussman@ed.gov), Sarah Brasiel (Sarah.Brasiel@ed.gov), and Katie Taylor (Katherine.Taylor@ed.gov), program officers for NCSER.

Career and Technical Education (CTE) Month® is celebrated every February to raise awareness about the role that CTE has in preparing students for college and career success and the achievements of CTE programs across the country. In recognition of this year’s CTE Month®, we caught up with Dr. Michael Gottfried, University of Pennsylvania, to discuss his CTE research.

Through NCSER’s Career and Technical Education for Students with Disabilities special topic area, Dr. Gottfried was awarded a grant to examine whether participating in STEM CTE courses in high school is related to pursuing and persisting in STEM majors and/or careers for students with learning disabilities (SWLDs), a project featured initially in a March 2020 blog. During our recent conversation, he shared updates with us about his CTE project as well as the policy and practice implications of his research.

When we discussed this project in 2020, you shared your research goals and what you had learned so far. Could you provide us with an update?

Since the last time that we chatted, we have made some great progress on this project. We have had several papers accepted for publication. In some of our work, we were interested in the STEM CTE coursetaking patterns of SWLDs in high school. We found that SWLDs are more likely to participate in CTE courses compared to students without disabilities. Yet, when looking at the specific category of STEM CTE courses, there is no evidence that SWLDs are more likely to participate in high school STEM CTE courses compared to students without disabilities.

We have also looked at specific outcomes for SWLDs in STEM CTE courses. For instance, we examined computer science STEM CTE coursetaking for SWLDs. Participation was associated with growth in STEM self-efficacy and STEM utility (usefulness of what is learned for practical application) for SWLDs, whereas it related to positive development of STEM self-efficacy and STEM identity, but not STEM utility, for students without learning disabilities.

After we discovered that little was known about the association between STEM CTE coursetaking and college STEM persistence for SWLDs, we wanted to explore this area. So far, we have found that SWLDs who earned more units of STEM CTE in high school were more likely to seriously consider and ultimately declare STEM majors in college that are related to high school STEM CTE courses, such as information technology or engineering technology.

You and your colleagues recently published a paper in Education Research based on your NCSER-funded research. Could you summarize the findings in this paper and the implications for policy and practice?  

In our paper, we set out to identify whether there were any observable changes in CTE participation over time. The unique aspect of this study was that it combined national data from the Education Longitudinal Study of 2002 and the High School Longitudinal Study of 2009 with administrative data from the state of Washington. Key findings indicated that CTE participation declined nationally between the graduating class of 2004 and the graduating class of 2009 except in the area of applied science, technology, engineering, mathematics, and medical/health (STEMM) CTE, which includes courses in information technology, engineering technology, and health sciences. Data from Washington tended to be less varied in nature compared to national data, with fewer discernible trends, though in general STEM CTE did appear to have an upward trend for all students.

Our work also has direct relevance to policy. Recent changes in the Carl D. Perkins Career and Technical Education for the 21^st Century Act emphasized the need to focus on increasing access and participation in STEM-related CTE coursework. While there does appear to be an upward trend in participation in these STEM fields, the tradeoff may be coming at the expense of other CTE areas of study. Combined with the increasing pressure for students to complete more and more academic coursework in a push for college readiness, this decrease in non-STEM CTE participation is particularly noteworthy. Finally, our work helps highlight the importance of examining CTE trends at the state and national levels. Different states have different needs and different graduation requirements that may or may not include CTE participation. As such, given the overall call to increase CTE participation for SWLDs, we encourage future research that explores the implications of these trends for this population.

What do you hope that school leaders, CTE teachers, and students will learn from all the research you are conducting?

The research has numerous implications for policy and practice. First, the results will be important for policymakers as they consider new or revised educational policies to support the pursuance and persistence of SWLDs into STEM fields. Education policymakers in particular need to understand the effects of STEM CTE coursetaking for SWLDs at multiple time points (transition into college, during college, and post-college). Understanding these issues more completely will make for well-informed policy decisions that promote short- and long-term success in STEM for SWLDs. This, in turn, has larger social policy implications with respect to upward mobility and lifelong success.

This project also has important implications for practice. By sharing these results, we hope to support education practitioners in making the adjustments necessary to improve the use of educational resources to ensure that SWLDs are prepared for and engaged in fields with high growth potential. For instance, many states have begun to accept STEM CTE courses for graduation requirements, which increases the likelihood students will take these courses. As STEM CTE courses prove important for SWLDs across the pipeline, then states and districts might consider how to best encourage students to take and succeed in these courses.

What additional research is needed to improve CTE policy and practice?

The current work can inform the future development of an intervention, assessment, or decision to evaluate an intervention. Evidence that STEM CTE coursetaking is associated with higher likelihood of college enrollment and the pursuit of STEM pathways for SWLDs supports the need to study interventions that encourage STEM CTE coursetaking for these students. For example, a randomly selected set of SWLDs who do not take traditional STEM could be counseled into taking STEM CTE courses or placement tests could be used to assign students to STEM CTE or traditional STEM courses. In both cases, students could then be followed into college and beyond to compare education and career outcomes using rigorous research designs. The results could provide additional, strong evidence for the value of STEM CTE coursetaking on postsecondary STEM outcomes.

This blog was authored by Akilah Nelson (akilah.nelson@ed.gov), Program Officer at NCSER, and Michael Gottfried (mgottfr2@upenn.edu), Associate Professor at the University of Pennsylvania.

This March, IES Director Mark Schneider released a blog in which he discussed exploring a partnership with the National Science Foundation (NSF) to encourage scientists with expertise in AI and related fields to address the important post-pandemic need for accelerating learning. IES is now excited to announce our resulting participation in NSF’s National Artificial Intelligence (AI) Research Institutes—Accelerating Research, Transforming Society, and Growing the American Workforce solicitation. In this blog, we describe this new funding opportunity, provide examples of existing NCSER-funded research in this area, and highlight the potential for such research to further improve outcomes for learners with disabilities.

 Artificial Intelligence Research Funding Opportunity

With funding from the American Rescue Plan, NCSER plans to support research under Theme 6, Track B: AI-Augmented Learning for Individuals with Disabilities. Proposals must discuss how the work will respond to the needs of learners with or at risk for a disability in an area where the COVID-19 pandemic has further widened existing gaps and/or resulted in decreased access and opportunities for students with disabilities to learn and receive support services. Please review the solicitation, the webinar (November 16), and the frequently asked questions for more information. Interested applicants should note the primary focus of this institute:

“The primary focus of an institute in AI-Augmented Learning includes research and development of AI-driven innovations to radically improve human learning and education. Achievement and opportunity gaps, particularly for learners from disadvantaged or underserved communities, have always been present, but COVID-19 has exacerbated them. Institute plans for this theme should address and measure outcomes with direct education impact, in both the short- and long- term, that have practical significance to educators, parents, or other decision-makers. Plans must also directly address algorithmic bias, model transparency, security and data privacy in the support of learning.”

Current NCSER-Funded Grants Applying Artificial Intelligence and Machine Learning

Prior to the new collaboration between IES and NSF, NCSER funded several grants that apply artificial intelligence and machine learning approaches, including those described below.

With a 2018 NCSER grant, Dr. Maithilee Kunda and her team at Vanderbilt University are building on a technology-based intervention known as Betty’s Brain. This computer-based instructional program for middle school science, designed with the support of a 2006 NCER grant, allows students to teach a computer agent to understand certain concepts, increasing their own knowledge and understanding. Dr. Kunda and her team are developing a new game called Film Detective, which is designed to improve theory of mind (ToM) reasoning in adolescents with autism spectrum disorder (ASD). More information about this project can be found in this IES blog.

With a 2021 NCSER grant, Dr. Patrick Kennedy and his team at University of Oregon are using machine learning to validate a well-known assessment, Dynamic Indicators of Basic Early Literacy Skills, 8^th Edition® (DIBELS 8) as a screener for dyslexia. As of 2020, 47 states require that students be screened for dyslexia in early elementary school and many states use DIBELS for this screening. However, it remains to be validated for this purpose. To address the validity of the DIBELS for screening, this research team is using machine learning approaches to predict and classify scores in relation to a pre-defined target. This will allow the research team to draw conclusions about the validity of the DIBELS 8 for dyslexia screening. These conclusions will be disseminated widely to state and local education agencies and other stakeholders.

The Potential of AI for Improving Outcomes for Learners with Disabilities

In addition to the work that IES is funding, AI has already demonstrated potential for improving outcomes for learners with disabilities in many other ways:

• AI has been used to support children with ASD who have difficulties understanding people’s emotions, with AI-driven apps and robots helping students practice emotion recognition and other social skills.
• AI has informed the development of algorithms that can help those involved in assessment identify disabilities in students, such as ASD, specific learning disabilities (dyslexia, dysgraphia, and dyscalculia), and attention-deficit/hyperactivity disorder (ADHD).
• AI-embedded interventions have included error analysis to inform instruction and personalized feedback in spelling and math for students with disabilities.

Despite these advancements, there appear to be persistent gaps in AI research for students with disabilities, such as AI for students with intellectual and developmental disabilities. This is an especially important area of work because many of these learners have multiple disabilities and/or serious health conditions. For example, children with intellectual and developmental disabilities who also have hearing loss or visual impairment have compounded challenges. Some students with Down syndrome also have hearing loss and other health complications, such as cardiac issues. AI affords an opportunity to integrate health information across different applications to improve the quality of life for these students. These technological solutions can assist in managing information about the students and communicating health information between teachers, physicians, and caregivers.

AI has the potential to transform special education. We hope that this NCSER-NSF partnership will encourage researchers to be creative in planning projects that move the field of AI forward as well as provide innovative solutions to support learners with disabilities.

This blog was co-authored by Sarah Brasiel (Sarah.Brasiel@ed.gov), program officer at NCSER and Bennett Lunn (Bennett.lunn@ed.gov), Truman-Albright Fellow for NCSER and the National Center for Education Research (NCER). IES encourages special education researchers to partners with experts in Artificial Intelligence to submit to this NSF AI Institute solicitation 22-502 to increase the evidence base on use of AI for this population.

Three IES-funded investigators were presented with awards from the Society for Prevention Research (SPR) last week. We recognize and applaud these investigators (pictured above from left to right) Elizabeth Stormshak, Dorothy Espelage, and Patrick Tolan.

Dr. Elizabeth Stormshak, Philip H. Knight Chair and Department Head for Counseling Psychology and Human Services at University of Oregon, received the Translational Science Award from SPR. This award is given to an individual or a team of individuals in recognition for contributions to the field of prevention science through translational research. Dr. Stormshak’s research focuses on understanding risk factors in early and middle childhood associated with the development of problem behavior in late adolescence, including substance use and delinquency. She also studies the process of disseminating evidence-based interventions into real world community settings. She has been the Principal Investigator on multiple IES grants, including a 2018 NCSER-funded project to examine the long-term efficacy of the Kindergarten Family Check-Up (FCU), a school-based, family-centered intervention intended to prevent student social and behavioral problems. This grant is a follow-up and extension to her recently completed randomized controlled trial of Kindergarten FCU, funded by NCER, which found positive impacts on student behavior and academic outcomes during and up to 3 years after the transition to kindergarten. The primary aims of the newer grant are to determine the long-term impact of receiving the original kindergarten intervention and the effects of a middle school booster session of FCU on students' behavior and academic outcomes. Dr. Stormshak is also the director of a NCSER-funded postdoctoral training program focused on the prevention of school-based social and behavioral problems.

Dr. Dorothy Espelage, William C. Friday Distinguished Professor of Education at the University of North Carolina, Chapel Hill, received SPR’s Prevention Science Award. This award is given for the application of scientific methods to develop and test prevention strategies. Dr. Espelage is a leading expert on school safety and has led multiple research studies on school-based violence and bullying. She has been involved in several IES grants. Most recently, she is serving as the Principal Investigator of a 2019 NCSER-funded project to develop and test a professional development program aimed at enhancing elementary school teachers' knowledge and skills for identifying, mitigating, and preventing bullying among students with and without disabilities. Dr. Espelage also led an exploratory study funded by NCER to better understand how teacher practices influence elementary school students’ interpersonal relationships in the classroom and related behavioral outcomes. 

Dr. Patrick H. Tolan, Charles S. Robb Professor of Education at the University of Virginia in the Curry School of Education and in the Department of Psychiatry and Neurobehavioral Sciences in the School of Medicine, and Director Emeritus of the YouthNex Center in the UVA Center to Promote Effective Youth Development, received the Advances in Culture and Diversity in Prevention Science Award as one of three founding members of the Boys of Color Collaborative. This award is given for contributions to the field of prevention science in the area of community and culture. Dr. Tolan’s research career spans 34 years with a focus on program evaluation for promoting positive youth development and preventing youth violence. Dr. Tolan is currently the Principal Investigator of a 2019 Follow-Up study of an IES efficacy study of the integration of two prevention programs, Good Behavior Game and My Teaching Partner (GBG+MTP), for teachers who have recently entered the teaching profession.

Congratulations to the award recipients!

This blog was co-authored by Jackie Buckley (NCSER), Katie Taylor (NCSER), Emily Doolittle (NCER), and Amy Sussman (NCSER).

Classroom teachers of young children face a seemingly never-ending challenge – how to manage disruptive behavior while simultaneously teaching effectively and supporting the needs of every student in the classroom. Researchers at Virginia Commonwealth University and the University of Florida have received five IES research grants over the past decade – three through the National Center for Special Education Research (NCSER) and two from the National Center for Education Research (NCER) – to develop and test a model of training and professional development, including coaching, for early childhood and early elementary school teachers on how best to support children who engage in disruptive and otherwise challenging classroom behaviors.

With their first IES grant in 2008, Drs. Maureen Conroy and Kevin Sutherland developed the original BEST in CLASS model for early childhood teachers. The goal of BEST in CLASS - PK is to increase the quantity and quality of specific instructional practices with young children (ages 3-5 years old) who engage in high rates of challenging behaviors with the ultimate goal of preventing and reducing problem behavior. Professional development consists of a six-hour workshop that uses didactic and interactive learning activities supported by video examples and practice opportunities. Following the workshop, teachers receive a training manual and 14 weeks of practice-based coaching in the classroom. 

The results of this promising development work led to a 2011 IES Efficacy study to test the impact of BEST in CLASS - PK on teacher practices and child outcomes. Based on positive findings from that Efficacy study the team was awarded two additional Development and Innovation grants – one in 2016 to develop a web-based version of BEST in CLASS – PK to increase accessibility and scalability and another in 2015 to adapt BEST in CLASS – PK for early elementary school classrooms (BEST in CLASS – Elementary). Drs. Sutherland and Conroy are currently in the second year of an Efficacy study to test the impact of BEST in CLASS - Elementary to determine if the positive effects of BEST in CLASS in preschool settings are replicated in early elementary classrooms.

Written by Emily Doolittle, NCER Team Lead for Social Behavioral Research, and Jacquelyn Buckley, NCSER Team Lead for Disability Research