IES Blog

In honor of American Education Week, IES recognizes the many school-based educators and staff who work together to support student learning and growth. This is particularly true for teams working to improve outcomes for students with disabilities. Providing special education requires a team approach with collaboration among a variety of professionals. To this end, school-based teams—teachers, administrators, special education and behavior specialists, and other support professionals—at the elementary level are in a constant process of problem solving. Student needs, ever evolving, are best met using targeted data and evidence-based practices. But how do school teams ensure that they are defining student needs accurately and applying the most effective interventions? In the busy school environment, how can team members best use their meeting time to serve students?

NCSER-funded researchers have been working to measure and support school team efforts through the development of decision-making models and observation tools refined and expanded over the course of more than 15 years. One approach dedicated to training team members and facilitating successful problem-solving meetings has been demonstrated to improve the decisions made by school teams and is now being integrated with student data systems and supported by online tools for staff.

Dr. Rob Horner (University of Oregon) and his team recognized the need for school problem-solving teams to access to student academic and behavioral data and to have a protocol for the effective use of these data. Based on an observational tool, Decision Observation Recording and Analysis (DORA), and a decision-making process developed with a previous NCSER grant, they evaluated the efficacy of Team-Initiated Problem Solving (TIPS). Focused on the school-based team meeting procedures, TIPS helps train school staff to use data to define student problems and develop targeted solutions that draw from existing research but are specific to each student’s unique circumstances and needs. This randomized controlled trial tested the TIPS model with school teams trained in schoolwide Positive Behavioral Interventions and Supports (PBIS), a systems-level framework that involves implementing multi-tiered, evidence-based practices to improve student social/behavioral and academic outcomes. Results indicated that the teams already had fairly strong foundational meeting procedures (such as use of agenda, minutes, and assigned roles) following the general PBIS training, but after exposure to TIPS training and coaching, school teams were better able to identify precise academic and behavioral problems in the students they observed. The solutions they generated were more targeted and, notably, researchers saw a shift from solutions that focused on changing the student to those that aimed to alter the student’s environment. In addition, teams that participated in TIPS were more likely to implement solutions they developed and their schools had fewer out-of-school suspensions than the schools that had teams in the control group.

Although there were positive effects of the TIPS model, it is important to note that drawing together a team of school staff with diverse specialties and relationships with the student remains a challenge. Together with Dr. Horner, Dr. Erin Chaparro (University of Oregon) is leading an IES-funded project to develop a set of technology tools to facilitate the use of TIPS with problem-solving teams. The project includes both online professional development modules tailored to team members’ needs and an app to assist with meeting protocols and easy access to meeting history and student data. These programs, collectively called the TIPS EdTech tools, are intended to improve team functioning and, by extension, student outcomes. The researchers are currently completing a pilot study to help determine the fidelity of implementing these tools and the promise for positive impacts on team functioning and student outcomes.  

TIPS is now being used in additional research. Dr. Wayne Sailor (University of Kansas) and his research team are focused on school teams’ ability to effectively leverage data to integrate student behavioral and academic supports. This NCSER-funded grant aims to improve school teams’ use of an integrated multi-tiered systems of support, which works to combine behavior and academic services, through the development of a decision support system (DSS). The DSS consists of two parts, one of which is an adaptation of the TIPS model for problem-solving team meetings termed “the meeting engine.” The second component consists of an existing digital system called DataWall, an integrated data system to link education databases, chart data, and build summary reports at various levels (such as school, grade, or student). This research team is currently enhancing DataWall while integrating with TIPS procedures.

Serving students with disabilities requires the skills and collaboration of many different education professionals, such as teachers and special education teachers, administrators, service providers, and paraeducators. The challenge of coordinating the efforts of school-based teams calls for ongoing innovation by both researchers and practitioners. TIPS and its iterations are one evidence-based way of helping to facilitate school staff supports for diverse student needs.

Written by Julianne Kasper, Virtual Student Federal Service Intern at IES and graduate student in Education Policy & Leadership at American University.

Looking at data by race and ethnicity can provide a better understanding of education performance and outcomes than examining statistics that describe all students. In observation of Native American Heritage Month, this blog presents NCES findings on the learning experiences of American Indian/Alaska Native (AI/AN) students throughout their education careers.

Early Childhood Education

• In 2019, 45 percent of AI/AN 3- to 4-year-olds and 83 percent of AI/AN 5-year-olds were enrolled in school.
   

K–12 Education

• The 2019 National Indian Education Study (NIES) surveyed students, teachers, and school principals about the experiences of AI/AN students in 4th and 8th grades.
   
  • How much do AI/AN students know about their culture?
    • Most 4th-grade AI/AN students reported having at least “a little” knowledge of their AI/AN tribe or group, with 17 percent reporting knowing “nothing.” About 19 to 23 percent reported having “a lot” of cultural knowledge across school types. (For more information, see NIES 2019, p. 11.)
       
  • Where do AI/AN students learn about their culture?
    • Family members were identified as the people who taught students the most about AI/AN history, with 45 percent of 4th-grade students and 60 percent of 8th-grade students so reporting. Teachers were the second most commonly identified group of people important for educating students on AI/AN cultural topics. (For more information, see NIES 2019, p. 12.)
       
  • How do teachers contribute to AI/AN student cultural knowledge?
    • A majority of AI/AN students had teachers who integrated AI/AN culture or history into reading lessons: overall, 89 percent of 4th-grade students and 76 percent of 8th-grade students had teachers who reported using these concepts in reading lessons “at least once a year.” (For more information, see NIES 2019, p. 16.)
       
  • What are AI/AN student trends on assessments in mathematics and reading?
    • Nationally, mathematics scores for AI/AN students from 2015 to 2019 remained unchanged for 4th-graders and declined for 8th-graders. Most states saw no change. (For more information, see NIES 2019, p. 46.)
       
• In 2019, 52 percent of AI/AN 4th-grade students had access to a computer at home. (For more information, see NIES 2019, p. 45.)
   
• There were 505,000 AI/AN students enrolled in public schools in 1995, compared with 490,000 AI/AN students in fall 2018 (the last year of data available).
   
• In fall 2018, less than half of AI/AN students (40 percent) attended schools where minority students comprised at least 75 percent of the student population.
   
• There are approximately 45,000 American Indian/Alaska Native students served by approximately 180 Bureau of Indian Education (BIE) schools located on 64 reservations in 23 states.
   
• In school year 2018–19, the adjusted cohort graduation rate (ACGR) was 74 percent for AI/AN public school students. The ACGRs for AI/AN students ranged from 51 percent in Minnesota to 94 percent in Alabama and were higher than the U.S. average in eight states (Texas, Virginia, Louisiana, Tennessee, Connecticut, New Jersey, Alabama, and Kentucky).
   
• In 2020, 95 percent of 25- to 29-year-olds who were AI/AN had completed at least high school.
  
  

Postsecondary Education

• In academic year 2018–19, 14 percent of bachelor’s degrees conferred to AI/AN graduates were in a STEM field.
   
• About 41 percent of AI/AN students who began seeking a bachelor’s degree full-time at a 4-year institution in fall 2013 completed that degree at the same institution within 6 years.

By Mandy Dean, AIR

In 2014, IES funded one of two R&D Centers on Knowledge Utilization (or Knowledge/Evidence Use) to explore how 1) education researchers can make their work more relevant and useful to practitioners located in state and local education agencies and in individual schools, 2) the work of practitioners can inform research efforts, and 3) practitioners can make decisions based on research evidence. The National Center for Research in Policy and Practice (NCRPP) at the University of Colorado at Boulder recently completed its grant. Corinne Alfeld, a Program Officer in IES NCER, talked with Principal Investigator Bill Penuel about the Center’s findings and recommendations. The full list of Center staff and collaborators can be found on NCRPP’s website.

What were the outcomes of your IES-funded KU R&D Center? 

Our center studied how school and district leaders accessed and used research through a nationally representative survey and through case studies of school district decision making, research-practice partnerships, and a professional association of state leaders in science. Across all studies, we found that leaders highly valued research. At the same time, we found some things that might be surprising to many researchers:

• Most leaders accessed research through their relationships and networks rather than through web sites, journal articles, or resources like the What Works Clearinghouse. The most common ways leaders accessed research was through their own professional associations, conferences, and colleagues in education settings. In some cases, these networks provided leaders with access to high-quality research. In our study of the professional association of science leaders, leaders cited National Academies of Sciences, Engineering, and Medicine consensus study reports as the most shared and used among members.
• Leaders used research for a variety of purposes and for a range of decision-making activities not only to make decisions about what programs to adopt. Many leaders, for example, design professional development for educators for which there may be no program to adopt. In our case studies of district research use, we found that leaders did rely on research—conceptually—to inform their design activities. Research use was embedded within the ongoing decision-making routines for designing, implementing, and evaluating professional development activities.
• Leaders did not turn to impact studies of individual interventions when they looked for research. Instead, they more commonly turned to books and other kinds of publications that provided syntheses or summaries of research. A common thread was that these sources of research provided frameworks for action, broken down into clear steps they could follow.

How can researchers use your findings to improve their dissemination efforts?

Despite the value leaders placed on research, there is clearly room for improvement with respect to dissemination. More than half of respondents to our survey said that by the time research was published, it was no longer valuable to them. Here are two strategies that researchers might find useful.

1. Long-term research-practice partnerships (RPPs) were sites where leaders found research to be both timely and relevant to them. These partnerships come in different shapes and sizes and have different goals. However, all engage educators in helping define the very questions that will be addressed in the research, and some also engage in co-design and testing of interventions to address persistent problems of practice and to work toward visions of more equitable systems of education.
2. Embedding researchers within leaders’ professional organizations can help disseminate research in a timely manner. These members present regularly at association meetings and conferences. They also participate in committees, where they develop tools to inform ongoing leadership activities.

What these strategies have in common is that dissemination is not an afterthought. In fact, dissemination is not a good word for what these researchers are doing. A better word is engagement. Researchers are engaging educators throughout the process of research and development, not just at the end.

How can your findings be used to improve practitioner access to and use or consideration of research findings?

Many of the strategies for engagement involve educators engaging with various aspects of research directly. Years ago, Weiss and colleagues described this as its own form of research use, called process use. Engaging educators in the actual research process does something that is important for supporting research use, namely giving them time to make sense of research and its implications for their work.

We found evidence that involvement in RPPs for educators was helpful to their own policymaking and practice. More than three-quarters, for example, said that their external partners shaped the design of professional development, and many also said that their partnership helped to integrate newly developed practices in the partnership.

What are your plans for future work in this area?

At present, NCRPP is involved in two exciting new projects. The first is a project funded by the William T. Grant Foundation, which focuses on developing and validating measures to assess the effectiveness of RPPs. We’re using a framework developed by Erin Henrick and colleagues to evaluate RPPs, and we’ve gathered survey and interview data from more than 60 RPPs. Our goal is to develop formative measures to help RPPs evaluate progress on each of the five dimensions of the framework.

The second project is funded by the Wallace Foundation to study and support equity-centered leadership and districts in forming partnerships with researchers as they develop and test strategies for creating equity-centered leadership pipelines. Both projects are being undertaken in collaboration with the National Network of Education Research-Practice Partnerships.

What do you see as the next steps for this field?

While it is tempting to suggest “more research is needed,” what is needed is an “evidence-informed” approach to evidence use—the application of what we already know about evidence use when it comes to policy and practice. That requires us to shift focus away from imagining that better, plain-language research briefs will help us improve research use. Instead, we need to encourage researchers to engage in more substantive ways with practice throughout the research process, to improve its relevance and timeliness.

We also need to embrace a broader conception of the kinds of evidence and information that can inform decision making, one that reflects the range of information that leaders currently use and could turn to. Of particular importance is considering the experiences of those students, families, and communities to whom we owe a great education debt as important sources for decision making. If we take a broader view of evidence, a new question emerges: How can we consider and integrate different sources of evidence in a way that is informed by values such as equity and justice into decision making? This is the sort of question I hope the field can pursue in the future.

Findings from the 2015 IES-funded Center for Research Use in Education (CRUE) at the University of Delaware will be highlighted in a blog in 2022. Stay tuned! If you have further questions, please contact Corinne.Alfeld@ed.gov.

This March, IES Director Mark Schneider released a blog in which he discussed exploring a partnership with the National Science Foundation (NSF) to encourage scientists with expertise in AI and related fields to address the important post-pandemic need for accelerating learning. IES is now excited to announce our resulting participation in NSF’s National Artificial Intelligence (AI) Research Institutes—Accelerating Research, Transforming Society, and Growing the American Workforce solicitation. In this blog, we describe this new funding opportunity, provide examples of existing NCSER-funded research in this area, and highlight the potential for such research to further improve outcomes for learners with disabilities.

 Artificial Intelligence Research Funding Opportunity

With funding from the American Rescue Plan, NCSER plans to support research under Theme 6, Track B: AI-Augmented Learning for Individuals with Disabilities. Proposals must discuss how the work will respond to the needs of learners with or at risk for a disability in an area where the COVID-19 pandemic has further widened existing gaps and/or resulted in decreased access and opportunities for students with disabilities to learn and receive support services. Please review the solicitation, the webinar (November 16), and the frequently asked questions for more information. Interested applicants should note the primary focus of this institute:

“The primary focus of an institute in AI-Augmented Learning includes research and development of AI-driven innovations to radically improve human learning and education. Achievement and opportunity gaps, particularly for learners from disadvantaged or underserved communities, have always been present, but COVID-19 has exacerbated them. Institute plans for this theme should address and measure outcomes with direct education impact, in both the short- and long- term, that have practical significance to educators, parents, or other decision-makers. Plans must also directly address algorithmic bias, model transparency, security and data privacy in the support of learning.”

Current NCSER-Funded Grants Applying Artificial Intelligence and Machine Learning

Prior to the new collaboration between IES and NSF, NCSER funded several grants that apply artificial intelligence and machine learning approaches, including those described below.

With a 2018 NCSER grant, Dr. Maithilee Kunda and her team at Vanderbilt University are building on a technology-based intervention known as Betty’s Brain. This computer-based instructional program for middle school science, designed with the support of a 2006 NCER grant, allows students to teach a computer agent to understand certain concepts, increasing their own knowledge and understanding. Dr. Kunda and her team are developing a new game called Film Detective, which is designed to improve theory of mind (ToM) reasoning in adolescents with autism spectrum disorder (ASD). More information about this project can be found in this IES blog.

With a 2021 NCSER grant, Dr. Patrick Kennedy and his team at University of Oregon are using machine learning to validate a well-known assessment, Dynamic Indicators of Basic Early Literacy Skills, 8^th Edition® (DIBELS 8) as a screener for dyslexia. As of 2020, 47 states require that students be screened for dyslexia in early elementary school and many states use DIBELS for this screening. However, it remains to be validated for this purpose. To address the validity of the DIBELS for screening, this research team is using machine learning approaches to predict and classify scores in relation to a pre-defined target. This will allow the research team to draw conclusions about the validity of the DIBELS 8 for dyslexia screening. These conclusions will be disseminated widely to state and local education agencies and other stakeholders.

The Potential of AI for Improving Outcomes for Learners with Disabilities

In addition to the work that IES is funding, AI has already demonstrated potential for improving outcomes for learners with disabilities in many other ways:

• AI has been used to support children with ASD who have difficulties understanding people’s emotions, with AI-driven apps and robots helping students practice emotion recognition and other social skills.
• AI has informed the development of algorithms that can help those involved in assessment identify disabilities in students, such as ASD, specific learning disabilities (dyslexia, dysgraphia, and dyscalculia), and attention-deficit/hyperactivity disorder (ADHD).
• AI-embedded interventions have included error analysis to inform instruction and personalized feedback in spelling and math for students with disabilities.

Despite these advancements, there appear to be persistent gaps in AI research for students with disabilities, such as AI for students with intellectual and developmental disabilities. This is an especially important area of work because many of these learners have multiple disabilities and/or serious health conditions. For example, children with intellectual and developmental disabilities who also have hearing loss or visual impairment have compounded challenges. Some students with Down syndrome also have hearing loss and other health complications, such as cardiac issues. AI affords an opportunity to integrate health information across different applications to improve the quality of life for these students. These technological solutions can assist in managing information about the students and communicating health information between teachers, physicians, and caregivers.

AI has the potential to transform special education. We hope that this NCSER-NSF partnership will encourage researchers to be creative in planning projects that move the field of AI forward as well as provide innovative solutions to support learners with disabilities.

This blog was co-authored by Sarah Brasiel (Sarah.Brasiel@ed.gov), program officer at NCSER and Bennett Lunn (Bennett.lunn@ed.gov), Truman-Albright Fellow for NCSER and the National Center for Education Research (NCER). IES encourages special education researchers to partners with experts in Artificial Intelligence to submit to this NSF AI Institute solicitation 22-502 to increase the evidence base on use of AI for this population.

The college pipeline from start to finish is an extraordinarily complex process with numerous decision points, options, and obstacles. Students from advantaged social backgrounds are more likely than those from disadvantaged backgrounds to attend schools and colleges staffed with advisers and support staff that have time and resources to assist them. They may also draw on relationships with family, adults in their communities, or knowledgeable peers for assistance in navigation decisions. For students without such supports, the sequence of choices may become so overwhelming that they respond by delaying decisions or making poor choices that lead to sizable delays in their degree progression. With these challenges in mind, IES has funded three information and advising initiatives that draw on insights from researchers, practitioners, and the research literature.

Technical Working Group Meeting

In July 2019, NCER convened a technical working group of 14 researchers and practitioners for a set of conversations structured around three intervention strategies that have garnered substantial attention over the last 5 years: nudges and other light-touch informational campaigns; intensive, proactive coaching and advising; and comprehensive approaches that comprise advising and other supports such as technology and financial incentives. Researchers and practitioners shared perceptions about the effectiveness of each strategy, its relevance to targeted student populations, and conditions for implementation. At the end of the day, working group members provided recommendations (see the Technical Working Group Meeting Summary for a full list), including the following:  

• Institutions should help determine what strategies get tested, apply for research grants, and participate in the research as it progresses.
• Research is needed that addresses the large amount of information that students face and that identifies the types of information that students respond to and act on.
• Replication studies should be designed to measure the effectiveness of promising intervention strategies for specific student groups, with the goal of enhancing effectiveness and cost-effectiveness.

Practice Guide on Effective Advising for Postsecondary Students

In October 2021, the What Works Clearinghouse (WWC) released a Practice Guide on Effective Advising for Postsecondary Students. The practice guide includes four evidence-based recommendations designed for an audience of administrators and staff at community colleges, 4-year institutions, and other public or private technical colleges who are responsible for designing and/or delivering advising to students:

• Intentionally design and deliver comprehensive, integrated advising that incorporates academic and non-academic supports to empower students to reach their educational goals.
• Transform advising to focus on the development of sustained, personalized relationships with individual students throughout their college career.
• Use mentoring and coaching to enhance comprehensive, integrated advising in ways that support students’ achievement and progression.
• Embed positive incentives in intentionally designed advising structures to encourage student participation and continued engagement.

Gap Analysis of Information and Advising Research and Practice

In March 2020, the Lead Team of the College Completion Network began a project aimed at identifying gaps in the research evidence base for information and advising strategies. The project is organized into three parts:

1. A systematic review of the research literature, documenting evidence of the effect of information and advising policies, practices, and programs on student outcomes
2. A scan of information and advising policies, practices, and programs that colleges use to improve student outcomes
3. A gap analysis to compare the findings from the scan to the findings from the systematic review to look for effective practices that are not widely implemented and promising practices in the field that have not been evaluated

The team plans to report its full set of findings by December 2021. College Completion Network study descriptions are available here: https://collegecompletionnetwork.org/studies.

This blog is the second in a blog series on Effective Postsecondary Interventions that highlights interventions with evidence of effectiveness generated through IES-funded research. For the first blog in the series, please see here.

Written by James Benson (James.Benson@ed.gov), a Program Officer for Postsecondary Education within NCER’s Policy and Systems Division, and Felicia Sanders (Felicia.Sanders@ed.gov), a Program Officer for the What Works Clearinghouse within NCEE’s Knowledge Use Division.