Inside IES Research

Notes from NCER & NCSER

Honoring Native American Language and Culture: Supporting Native American Students in Our Schools

As we celebrate Native American Heritage Month, we want to recognize the rich and diverse traditions, linguistic backgrounds, and cultural heritages that American Indian and Alaska Native (AI/AN) students bring to classrooms. Indigenous knowledge can enrich perspectives of all students and educators. Despite their many strengths, AI/AN students tend to lag behind their peers on academic assessments. For instance, as reported in the Condition of Education 2020, on the 2019 National Assessment of Educational Progress (NAEP), AI/AN students consistently score lower in reading and math and have lower graduation rates and the highest high school drop-out rates relative to their peers.

To close this achievement gap and support AI/AN students in their academic success, the AI/AN community has recommended integrating Native Language and Culture (NLC) into instruction. However, some studies have found a negative association between use of NLC and AI/AN student outcomes. In this guest blog, Dr. Claudia Vincent discusses her IES-funded study, which aims to obtain nuanced understanding of the construct of NLC and its relation to AI/AN student learning and achievement in school. 

With IES funding from 2014 to 2017, our team of researchers at the University of Oregon worked with data from the National Indian Education Study (NIES) to operationalize Native Language and Culture (NLC) in schools from different viewpoints (student, teacher, school administrator) and examine the relationship between use of NLC and student academic and behavioral outcomes. Here’s what we learned from our study.

NLC is multi-dimensional and means different things to students, teachers, and administrators.

For students, NLC meant direct contact with AI/AN people as well as access to instructional material providing information about AI/AN traditions, languages, and history. For teachers, most of whom are not of AI/AN descent, NLC meant use of AI/AN traditions, history, and issues in academic instruction and access to materials and resources reflecting those traditions, history, and issues. For administrators, NLC meant involvement of local AI/AN people in the school, the school’s ability to provide instruction in AI/AN culture, and the school’s ability to provide instruction in AI/AN languages.

The multi-faceted nature of NLC suggests that different NLC practices likely benefit different students differently in different contexts. Our exploration of the relationship between the use of NLC as defined by the NIES data and student academic outcomes as measured by NAEP data provided insight into the contextual variables affecting the benefits of NLC. First, implementation of recommended NLC practices is rare overall. AI/AN teachers speaking Native language(s) and teaching in classrooms with high AI/AN enrollment located in schools employing AI/AN teachers and staff implement the recommended practices more often. Second, NLC benefitted math achievement most for those AI/AN students whose families identified strongly with AI/AN traditions and customs and who attended schools with high AI/AN enrollment.

These findings suggest that alignment between school and home cultures can promote the achievement of AI/AN students, but that NLC might be less beneficial, or even detrimental, for students who do not have a strong AI/AN identity, or who attend schools with low AI/AN enrollment. In the latter context, NLC in the classroom might be associated with stereotype threat, meaning that AI/AN students might perform lower when negative biases about their ethnic backgrounds are more prominent.

While our data analyses provided important insights into the many dimensions of NLC and its relation to AI/AN student success, the lived experiences of our advisory board members brought our findings to life. In addition to our research team, our study was guided by an advisory board consisting of AI/AN scholars and community members representing the Cherokee Nation, the Choctaw Nation, the Yakama Nation, the Kiowa Tribe of Oklahoma, and Oglala Lakota descendancy. Their contributions were instrumental in providing context to our findings. For example, the board suggested that providing students with access to AI/AN people might be most beneficial if teachers create an inclusive and welcoming environment where visitors and their contributions to educational experiences are clearly honored. Similarly, a classroom visit from an AI/AN guest should be linked to broader instructional goals to prevent tokenization of AI/AN culture. Teachers should feel comfortable and supported in challenging the dominant cultural narrative in their school by questioning content of textbooks in order to encourage their students to think critically about the cultural context of their education. 

AI/AN students represent a highly diverse group who bring critical perspectives to our classrooms. Promoting learning environments where they can succeed would benefit not only AI/AN students but enrich the educational experiences of all students.  


This post is part of our Native American Heritage Month blog series, In the first post, the National Center for Education Statistics (NCES) shared NCES findings on the learning experiences of AI/AN students throughout their education careers.

Dr. Claudia Vincent is a Research Associate in the Center for Equity Promotion, College of Education at the University of Oregon. Her research focuses on identifying and developing solutions for persistent racial/ethnic disparities in discipline and academic achievement.

This guest blog was produced by Katina Stapleton (Katina.Stapleton@ed.gov), co-Chair of the IES Diversity and Inclusion Council and Emily Doolittle (Emily.Doolittle@ed.gov), NCER Team Lead for Social Behavioral Research.

Recognizing School-Based Teams for American Education Week: Team-Initiated Problem Solving (TIPS)

In honor of American Education Week, IES recognizes the many school-based educators and staff who work together to support student learning and growth. This is particularly true for teams working to improve outcomes for students with disabilities. Providing special education requires a team approach with collaboration among a variety of professionals. To this end, school-based teams—teachers, administrators, special education and behavior specialists, and other support professionals—at the elementary level are in a constant process of problem solving. Student needs, ever evolving, are best met using targeted data and evidence-based practices. But how do school teams ensure that they are defining student needs accurately and applying the most effective interventions? In the busy school environment, how can team members best use their meeting time to serve students?

NCSER-funded researchers have been working to measure and support school team efforts through the development of decision-making models and observation tools refined and expanded over the course of more than 15 years. One approach dedicated to training team members and facilitating successful problem-solving meetings has been demonstrated to improve the decisions made by school teams and is now being integrated with student data systems and supported by online tools for staff.

Photo of Dr. Rob Horner

Dr. Rob Horner (University of Oregon) and his team recognized the need for school problem-solving teams to access to student academic and behavioral data and to have a protocol for the effective use of these data. Based on an observational tool, Decision Observation Recording and Analysis (DORA), and a decision-making process developed with a previous NCSER grant, they evaluated the efficacy of Team-Initiated Problem Solving (TIPS). Focused on the school-based team meeting procedures, TIPS helps train school staff to use data to define student problems and develop targeted solutions that draw from existing research but are specific to each student’s unique circumstances and needs. This randomized controlled trial tested the TIPS model with school teams trained in schoolwide Positive Behavioral Interventions and Supports (PBIS), a systems-level framework that involves implementing multi-tiered, evidence-based practices to improve student social/behavioral and academic outcomes. Results indicated that the teams already had fairly strong foundational meeting procedures (such as use of agenda, minutes, and assigned roles) following the general PBIS training, but after exposure to TIPS training and coaching, school teams were better able to identify precise academic and behavioral problems in the students they observed. The solutions they generated were more targeted and, notably, researchers saw a shift from solutions that focused on changing the student to those that aimed to alter the student’s environment. In addition, teams that participated in TIPS were more likely to implement solutions they developed and their schools had fewer out-of-school suspensions than the schools that had teams in the control group.

Although there were positive effects of the TIPS model, it is important to note that drawing together a team of school staff with diverse specialties and relationships with the student remains a challenge. Together with Dr. Horner, Dr. Erin Chaparro (University of Oregon) is leading an IES-funded project to develop a set of technology tools to facilitate the use of TIPS with problem-solving teams. The project includes both online professional development modules tailored to team members’ needs and an app to assist with meeting protocols and easy access to meeting history and student data. These programs, collectively called the TIPS EdTech tools, are intended to improve team functioning and, by extension, student outcomes. The researchers are currently completing a pilot study to help determine the fidelity of implementing these tools and the promise for positive impacts on team functioning and student outcomes.  

TIPS is now being used in additional research. Dr. Wayne Sailor (University of Kansas) and his research team are focused on school teams’ ability to effectively leverage data to integrate student behavioral and academic supports. This NCSER-funded grant aims to improve school teams’ use of an integrated multi-tiered systems of support, which works to combine behavior and academic services, through the development of a decision support system (DSS). The DSS consists of two parts, one of which is an adaptation of the TIPS model for problem-solving team meetings termed “the meeting engine.” The second component consists of an existing digital system called DataWall, an integrated data system to link education databases, chart data, and build summary reports at various levels (such as school, grade, or student). This research team is currently enhancing DataWall while integrating with TIPS procedures.

Serving students with disabilities requires the skills and collaboration of many different education professionals, such as teachers and special education teachers, administrators, service providers, and paraeducators. The challenge of coordinating the efforts of school-based teams calls for ongoing innovation by both researchers and practitioners. TIPS and its iterations are one evidence-based way of helping to facilitate school staff supports for diverse student needs.

Written by Julianne Kasper, Virtual Student Federal Service Intern at IES and graduate student in Education Policy & Leadership at American University.

How Do Education Leaders Access and Use Research Evidence?

In 2014, IES funded one of two R&D Centers on Knowledge Utilization (or Knowledge/Evidence Use) to explore how 1) education researchers can make their work more relevant and useful to practitioners located in state and local education agencies and in individual schools, 2) the work of practitioners can inform research efforts, and 3) practitioners can make decisions based on research evidence. The National Center for Research in Policy and Practice (NCRPP) at the University of Colorado at Boulder recently completed its grant. Corinne Alfeld, a Program Officer in IES NCER, talked with Principal Investigator Bill Penuel about the Center’s findings and recommendations. The full list of Center staff and collaborators can be found on NCRPP’s website.

What were the outcomes of your IES-funded KU R&D Center? 

Our center studied how school and district leaders accessed and used research through a nationally representative survey and through case studies of school district decision making, research-practice partnerships, and a professional association of state leaders in science. Across all studies, we found that leaders highly valued research. At the same time, we found some things that might be surprising to many researchers:

  • Most leaders accessed research through their relationships and networks rather than through web sites, journal articles, or resources like the What Works Clearinghouse. The most common ways leaders accessed research was through their own professional associations, conferences, and colleagues in education settings. In some cases, these networks provided leaders with access to high-quality research. In our study of the professional association of science leaders, leaders cited National Academies of Sciences, Engineering, and Medicine consensus study reports as the most shared and used among members.
  • Leaders used research for a variety of purposes and for a range of decision-making activities not only to make decisions about what programs to adopt. Many leaders, for example, design professional development for educators for which there may be no program to adopt. In our case studies of district research use, we found that leaders did rely on research—conceptually—to inform their design activities. Research use was embedded within the ongoing decision-making routines for designing, implementing, and evaluating professional development activities.
  • Leaders did not turn to impact studies of individual interventions when they looked for research. Instead, they more commonly turned to books and other kinds of publications that provided syntheses or summaries of research. A common thread was that these sources of research provided frameworks for action, broken down into clear steps they could follow.

How can researchers use your findings to improve their dissemination efforts?

Despite the value leaders placed on research, there is clearly room for improvement with respect to dissemination. More than half of respondents to our survey said that by the time research was published, it was no longer valuable to them. Here are two strategies that researchers might find useful.

  1. Long-term research-practice partnerships (RPPs) were sites where leaders found research to be both timely and relevant to them. These partnerships come in different shapes and sizes and have different goals. However, all engage educators in helping define the very questions that will be addressed in the research, and some also engage in co-design and testing of interventions to address persistent problems of practice and to work toward visions of more equitable systems of education.
  2. Embedding researchers within leaders’ professional organizations can help disseminate research in a timely manner. These members present regularly at association meetings and conferences. They also participate in committees, where they develop tools to inform ongoing leadership activities.

What these strategies have in common is that dissemination is not an afterthought. In fact, dissemination is not a good word for what these researchers are doing. A better word is engagement. Researchers are engaging educators throughout the process of research and development, not just at the end.

How can your findings be used to improve practitioner access to and use or consideration of research findings?

Many of the strategies for engagement involve educators engaging with various aspects of research directly. Years ago, Weiss and colleagues described this as its own form of research use, called process use. Engaging educators in the actual research process does something that is important for supporting research use, namely giving them time to make sense of research and its implications for their work.

We found evidence that involvement in RPPs for educators was helpful to their own policymaking and practice. More than three-quarters, for example, said that their external partners shaped the design of professional development, and many also said that their partnership helped to integrate newly developed practices in the partnership.

What are your plans for future work in this area?

At present, NCRPP is involved in two exciting new projects. The first is a project funded by the William T. Grant Foundation, which focuses on developing and validating measures to assess the effectiveness of RPPs. We’re using a framework developed by Erin Henrick and colleagues to evaluate RPPs, and we’ve gathered survey and interview data from more than 60 RPPs. Our goal is to develop formative measures to help RPPs evaluate progress on each of the five dimensions of the framework.

The second project is funded by the Wallace Foundation to study and support equity-centered leadership and districts in forming partnerships with researchers as they develop and test strategies for creating equity-centered leadership pipelines. Both projects are being undertaken in collaboration with the National Network of Education Research-Practice Partnerships.

What do you see as the next steps for this field?

While it is tempting to suggest “more research is needed,” what is needed is an “evidence-informed” approach to evidence use—the application of what we already know about evidence use when it comes to policy and practice. That requires us to shift focus away from imagining that better, plain-language research briefs will help us improve research use. Instead, we need to encourage researchers to engage in more substantive ways with practice throughout the research process, to improve its relevance and timeliness.

We also need to embrace a broader conception of the kinds of evidence and information that can inform decision making, one that reflects the range of information that leaders currently use and could turn to. Of particular importance is considering the experiences of those students, families, and communities to whom we owe a great education debt as important sources for decision making. If we take a broader view of evidence, a new question emerges: How can we consider and integrate different sources of evidence in a way that is informed by values such as equity and justice into decision making? This is the sort of question I hope the field can pursue in the future.


Findings from the 2015 IES-funded Center for Research Use in Education (CRUE) at the University of Delaware will be highlighted in a blog in 2022. Stay tuned! If you have further questions, please contact Corinne.Alfeld@ed.gov.

 

AI-Augmented Learning for Individuals with Disabilities: New Funding Opportunity, Current Research, and the Potential for Improving Student Outcomes

This March, IES Director Mark Schneider released a blog in which he discussed exploring a partnership with the National Science Foundation (NSF) to encourage scientists with expertise in AI and related fields to address the important post-pandemic need for accelerating learning. IES is now excited to announce our resulting participation in NSF’s National Artificial Intelligence (AI) Research Institutes—Accelerating Research, Transforming Society, and Growing the American Workforce solicitation. In this blog, we describe this new funding opportunity, provide examples of existing NCSER-funded research in this area, and highlight the potential for such research to further improve outcomes for learners with disabilities.

 Artificial Intelligence Research Funding Opportunity

With funding from the American Rescue Plan, NCSER plans to support research under Theme 6, Track B: AI-Augmented Learning for Individuals with Disabilities. Proposals must discuss how the work will respond to the needs of learners with or at risk for a disability in an area where the COVID-19 pandemic has further widened existing gaps and/or resulted in decreased access and opportunities for students with disabilities to learn and receive support services. Please review the solicitation, the webinar (November 16), and the frequently asked questions for more information. Interested applicants should note the primary focus of this institute:

The primary focus of an institute in AI-Augmented Learning includes research and development of AI-driven innovations to radically improve human learning and education. Achievement and opportunity gaps, particularly for learners from disadvantaged or underserved communities, have always been present, but COVID-19 has exacerbated them. Institute plans for this theme should address and measure outcomes with direct education impact, in both the short- and long- term, that have practical significance to educators, parents, or other decision-makers. Plans must also directly address algorithmic bias, model transparency, security and data privacy in the support of learning.”

Current NCSER-Funded Grants Applying Artificial Intelligence and Machine Learning

Prior to the new collaboration between IES and NSF, NCSER funded several grants that apply artificial intelligence and machine learning approaches, including those described below.

With a 2018 NCSER grant, Dr. Maithilee Kunda and her team at Vanderbilt University are building on a technology-based intervention known as Betty’s Brain. This computer-based instructional program for middle school science, designed with the support of a 2006 NCER grant, allows students to teach a computer agent to understand certain concepts, increasing their own knowledge and understanding. Dr. Kunda and her team are developing a new game called Film Detective, which is designed to improve theory of mind (ToM) reasoning in adolescents with autism spectrum disorder (ASD). More information about this project can be found in this IES blog.

With a 2021 NCSER grant, Dr. Patrick Kennedy and his team at University of Oregon are using machine learning to validate a well-known assessment, Dynamic Indicators of Basic Early Literacy Skills, 8th Edition® (DIBELS 8) as a screener for dyslexia. As of 2020, 47 states require that students be screened for dyslexia in early elementary school and many states use DIBELS for this screening. However, it remains to be validated for this purpose. To address the validity of the DIBELS for screening, this research team is using machine learning approaches to predict and classify scores in relation to a pre-defined target. This will allow the research team to draw conclusions about the validity of the DIBELS 8 for dyslexia screening. These conclusions will be disseminated widely to state and local education agencies and other stakeholders.

The Potential of AI for Improving Outcomes for Learners with Disabilities

In addition to the work that IES is funding, AI has already demonstrated potential for improving outcomes for learners with disabilities in many other ways:

  • AI has been used to support children with ASD who have difficulties understanding people’s emotions, with AI-driven apps and robots helping students practice emotion recognition and other social skills.
  • AI has informed the development of algorithms that can help those involved in assessment identify disabilities in students, such as ASD, specific learning disabilities (dyslexia, dysgraphia, and dyscalculia), and attention-deficit/hyperactivity disorder (ADHD).
  • AI-embedded interventions have included error analysis to inform instruction and personalized feedback in spelling and math for students with disabilities.

Despite these advancements, there appear to be persistent gaps in AI research for students with disabilities, such as AI for students with intellectual and developmental disabilities. This is an especially important area of work because many of these learners have multiple disabilities and/or serious health conditions. For example, children with intellectual and developmental disabilities who also have hearing loss or visual impairment have compounded challenges. Some students with Down syndrome also have hearing loss and other health complications, such as cardiac issues. AI affords an opportunity to integrate health information across different applications to improve the quality of life for these students. These technological solutions can assist in managing information about the students and communicating health information between teachers, physicians, and caregivers.

AI has the potential to transform special education. We hope that this NCSER-NSF partnership will encourage researchers to be creative in planning projects that move the field of AI forward as well as provide innovative solutions to support learners with disabilities.

This blog was co-authored by Sarah Brasiel (Sarah.Brasiel@ed.gov), program officer at NCSER and Bennett Lunn (Bennett.lunn@ed.gov), Truman-Albright Fellow for NCSER and the National Center for Education Research (NCER). IES encourages special education researchers to partners with experts in Artificial Intelligence to submit to this NSF AI Institute solicitation 22-502 to increase the evidence base on use of AI for this population.

Information and Advising: Identifying Effective Strategies that Help Students Navigate Postsecondary Education

The college pipeline from start to finish is an extraordinarily complex process with numerous decision points, options, and obstacles. Students from advantaged social backgrounds are more likely than those from disadvantaged backgrounds to attend schools and colleges staffed with advisers and support staff that have time and resources to assist them. They may also draw on relationships with family, adults in their communities, or knowledgeable peers for assistance in navigation decisions. For students without such supports, the sequence of choices may become so overwhelming that they respond by delaying decisions or making poor choices that lead to sizable delays in their degree progression. With these challenges in mind, IES has funded three information and advising initiatives that draw on insights from researchers, practitioners, and the research literature.

Technical Working Group Meeting

In July 2019, NCER convened a technical working group of 14 researchers and practitioners for a set of conversations structured around three intervention strategies that have garnered substantial attention over the last 5 years: nudges and other light-touch informational campaigns; intensive, proactive coaching and advising; and comprehensive approaches that comprise advising and other supports such as technology and financial incentives. Researchers and practitioners shared perceptions about the effectiveness of each strategy, its relevance to targeted student populations, and conditions for implementation. At the end of the day, working group members provided recommendations (see the Technical Working Group Meeting Summary for a full list), including the following:  

  • Institutions should help determine what strategies get tested, apply for research grants, and participate in the research as it progresses.
  • Research is needed that addresses the large amount of information that students face and that identifies the types of information that students respond to and act on.
  • Replication studies should be designed to measure the effectiveness of promising intervention strategies for specific student groups, with the goal of enhancing effectiveness and cost-effectiveness.

Practice Guide on Effective Advising for Postsecondary Students

In October 2021, the What Works Clearinghouse (WWC) released a Practice Guide on Effective Advising for Postsecondary Students. The practice guide includes four evidence-based recommendations designed for an audience of administrators and staff at community colleges, 4-year institutions, and other public or private technical colleges who are responsible for designing and/or delivering advising to students:

  • Intentionally design and deliver comprehensive, integrated advising that incorporates academic and non-academic supports to empower students to reach their educational goals.
  • Transform advising to focus on the development of sustained, personalized relationships with individual students throughout their college career.
  • Use mentoring and coaching to enhance comprehensive, integrated advising in ways that support students’ achievement and progression.
  • Embed positive incentives in intentionally designed advising structures to encourage student participation and continued engagement.

Gap Analysis of Information and Advising Research and Practice

In March 2020, the Lead Team of the College Completion Network began a project aimed at identifying gaps in the research evidence base for information and advising strategies. The project is organized into three parts:

  1. A systematic review of the research literature, documenting evidence of the effect of information and advising policies, practices, and programs on student outcomes
  2. A scan of information and advising policies, practices, and programs that colleges use to improve student outcomes
  3. A gap analysis to compare the findings from the scan to the findings from the systematic review to look for effective practices that are not widely implemented and promising practices in the field that have not been evaluated

The team plans to report its full set of findings by December 2021. College Completion Network study descriptions are available here: https://collegecompletionnetwork.org/studies.


This blog is the second in a blog series on Effective Postsecondary Interventions that highlights interventions with evidence of effectiveness generated through IES-funded research. For the first blog in the series, please see here.

Written by James Benson (James.Benson@ed.gov), a Program Officer for Postsecondary Education within NCER’s Policy and Systems Division, and Felicia Sanders (Felicia.Sanders@ed.gov), a Program Officer for the What Works Clearinghouse within NCEE’s Knowledge Use Division.