Inside IES Research

In honor of American Education Week, IES recognizes the many school-based educators and staff who work together to support student learning and growth. This is particularly true for teams working to improve outcomes for students with disabilities. Providing special education requires a team approach with collaboration among a variety of professionals. To this end, school-based teams—teachers, administrators, special education and behavior specialists, and other support professionals—at the elementary level are in a constant process of problem solving. Student needs, ever evolving, are best met using targeted data and evidence-based practices. But how do school teams ensure that they are defining student needs accurately and applying the most effective interventions? In the busy school environment, how can team members best use their meeting time to serve students?

NCSER-funded researchers have been working to measure and support school team efforts through the development of decision-making models and observation tools refined and expanded over the course of more than 15 years. One approach dedicated to training team members and facilitating successful problem-solving meetings has been demonstrated to improve the decisions made by school teams and is now being integrated with student data systems and supported by online tools for staff.

Dr. Rob Horner (University of Oregon) and his team recognized the need for school problem-solving teams to access to student academic and behavioral data and to have a protocol for the effective use of these data. Based on an observational tool, Decision Observation Recording and Analysis (DORA), and a decision-making process developed with a previous NCSER grant, they evaluated the efficacy of Team-Initiated Problem Solving (TIPS). Focused on the school-based team meeting procedures, TIPS helps train school staff to use data to define student problems and develop targeted solutions that draw from existing research but are specific to each student’s unique circumstances and needs. This randomized controlled trial tested the TIPS model with school teams trained in schoolwide Positive Behavioral Interventions and Supports (PBIS), a systems-level framework that involves implementing multi-tiered, evidence-based practices to improve student social/behavioral and academic outcomes. Results indicated that the teams already had fairly strong foundational meeting procedures (such as use of agenda, minutes, and assigned roles) following the general PBIS training, but after exposure to TIPS training and coaching, school teams were better able to identify precise academic and behavioral problems in the students they observed. The solutions they generated were more targeted and, notably, researchers saw a shift from solutions that focused on changing the student to those that aimed to alter the student’s environment. In addition, teams that participated in TIPS were more likely to implement solutions they developed and their schools had fewer out-of-school suspensions than the schools that had teams in the control group.

Although there were positive effects of the TIPS model, it is important to note that drawing together a team of school staff with diverse specialties and relationships with the student remains a challenge. Together with Dr. Horner, Dr. Erin Chaparro (University of Oregon) is leading an IES-funded project to develop a set of technology tools to facilitate the use of TIPS with problem-solving teams. The project includes both online professional development modules tailored to team members’ needs and an app to assist with meeting protocols and easy access to meeting history and student data. These programs, collectively called the TIPS EdTech tools, are intended to improve team functioning and, by extension, student outcomes. The researchers are currently completing a pilot study to help determine the fidelity of implementing these tools and the promise for positive impacts on team functioning and student outcomes.  

TIPS is now being used in additional research. Dr. Wayne Sailor (University of Kansas) and his research team are focused on school teams’ ability to effectively leverage data to integrate student behavioral and academic supports. This NCSER-funded grant aims to improve school teams’ use of an integrated multi-tiered systems of support, which works to combine behavior and academic services, through the development of a decision support system (DSS). The DSS consists of two parts, one of which is an adaptation of the TIPS model for problem-solving team meetings termed “the meeting engine.” The second component consists of an existing digital system called DataWall, an integrated data system to link education databases, chart data, and build summary reports at various levels (such as school, grade, or student). This research team is currently enhancing DataWall while integrating with TIPS procedures.

Serving students with disabilities requires the skills and collaboration of many different education professionals, such as teachers and special education teachers, administrators, service providers, and paraeducators. The challenge of coordinating the efforts of school-based teams calls for ongoing innovation by both researchers and practitioners. TIPS and its iterations are one evidence-based way of helping to facilitate school staff supports for diverse student needs.

Written by Julianne Kasper, Virtual Student Federal Service Intern at IES and graduate student in Education Policy & Leadership at American University.

In 2014, IES funded one of two R&D Centers on Knowledge Utilization (or Knowledge/Evidence Use) to explore how 1) education researchers can make their work more relevant and useful to practitioners located in state and local education agencies and in individual schools, 2) the work of practitioners can inform research efforts, and 3) practitioners can make decisions based on research evidence. The National Center for Research in Policy and Practice (NCRPP) at the University of Colorado at Boulder recently completed its grant. Corinne Alfeld, a Program Officer in IES NCER, talked with Principal Investigator Bill Penuel about the Center’s findings and recommendations. The full list of Center staff and collaborators can be found on NCRPP’s website.

What were the outcomes of your IES-funded KU R&D Center? 

Our center studied how school and district leaders accessed and used research through a nationally representative survey and through case studies of school district decision making, research-practice partnerships, and a professional association of state leaders in science. Across all studies, we found that leaders highly valued research. At the same time, we found some things that might be surprising to many researchers:

• Most leaders accessed research through their relationships and networks rather than through web sites, journal articles, or resources like the What Works Clearinghouse. The most common ways leaders accessed research was through their own professional associations, conferences, and colleagues in education settings. In some cases, these networks provided leaders with access to high-quality research. In our study of the professional association of science leaders, leaders cited National Academies of Sciences, Engineering, and Medicine consensus study reports as the most shared and used among members.
• Leaders used research for a variety of purposes and for a range of decision-making activities not only to make decisions about what programs to adopt. Many leaders, for example, design professional development for educators for which there may be no program to adopt. In our case studies of district research use, we found that leaders did rely on research—conceptually—to inform their design activities. Research use was embedded within the ongoing decision-making routines for designing, implementing, and evaluating professional development activities.
• Leaders did not turn to impact studies of individual interventions when they looked for research. Instead, they more commonly turned to books and other kinds of publications that provided syntheses or summaries of research. A common thread was that these sources of research provided frameworks for action, broken down into clear steps they could follow.

How can researchers use your findings to improve their dissemination efforts?

Despite the value leaders placed on research, there is clearly room for improvement with respect to dissemination. More than half of respondents to our survey said that by the time research was published, it was no longer valuable to them. Here are two strategies that researchers might find useful.

1. Long-term research-practice partnerships (RPPs) were sites where leaders found research to be both timely and relevant to them. These partnerships come in different shapes and sizes and have different goals. However, all engage educators in helping define the very questions that will be addressed in the research, and some also engage in co-design and testing of interventions to address persistent problems of practice and to work toward visions of more equitable systems of education.
2. Embedding researchers within leaders’ professional organizations can help disseminate research in a timely manner. These members present regularly at association meetings and conferences. They also participate in committees, where they develop tools to inform ongoing leadership activities.

What these strategies have in common is that dissemination is not an afterthought. In fact, dissemination is not a good word for what these researchers are doing. A better word is engagement. Researchers are engaging educators throughout the process of research and development, not just at the end.

How can your findings be used to improve practitioner access to and use or consideration of research findings?

Many of the strategies for engagement involve educators engaging with various aspects of research directly. Years ago, Weiss and colleagues described this as its own form of research use, called process use. Engaging educators in the actual research process does something that is important for supporting research use, namely giving them time to make sense of research and its implications for their work.

We found evidence that involvement in RPPs for educators was helpful to their own policymaking and practice. More than three-quarters, for example, said that their external partners shaped the design of professional development, and many also said that their partnership helped to integrate newly developed practices in the partnership.

What are your plans for future work in this area?

At present, NCRPP is involved in two exciting new projects. The first is a project funded by the William T. Grant Foundation, which focuses on developing and validating measures to assess the effectiveness of RPPs. We’re using a framework developed by Erin Henrick and colleagues to evaluate RPPs, and we’ve gathered survey and interview data from more than 60 RPPs. Our goal is to develop formative measures to help RPPs evaluate progress on each of the five dimensions of the framework.

The second project is funded by the Wallace Foundation to study and support equity-centered leadership and districts in forming partnerships with researchers as they develop and test strategies for creating equity-centered leadership pipelines. Both projects are being undertaken in collaboration with the National Network of Education Research-Practice Partnerships.

What do you see as the next steps for this field?

While it is tempting to suggest “more research is needed,” what is needed is an “evidence-informed” approach to evidence use—the application of what we already know about evidence use when it comes to policy and practice. That requires us to shift focus away from imagining that better, plain-language research briefs will help us improve research use. Instead, we need to encourage researchers to engage in more substantive ways with practice throughout the research process, to improve its relevance and timeliness.

We also need to embrace a broader conception of the kinds of evidence and information that can inform decision making, one that reflects the range of information that leaders currently use and could turn to. Of particular importance is considering the experiences of those students, families, and communities to whom we owe a great education debt as important sources for decision making. If we take a broader view of evidence, a new question emerges: How can we consider and integrate different sources of evidence in a way that is informed by values such as equity and justice into decision making? This is the sort of question I hope the field can pursue in the future.

Findings from the 2015 IES-funded Center for Research Use in Education (CRUE) at the University of Delaware will be highlighted in a blog in 2022. Stay tuned! If you have further questions, please contact Corinne.Alfeld@ed.gov.

This March, IES Director Mark Schneider released a blog in which he discussed exploring a partnership with the National Science Foundation (NSF) to encourage scientists with expertise in AI and related fields to address the important post-pandemic need for accelerating learning. IES is now excited to announce our resulting participation in NSF’s National Artificial Intelligence (AI) Research Institutes—Accelerating Research, Transforming Society, and Growing the American Workforce solicitation. In this blog, we describe this new funding opportunity, provide examples of existing NCSER-funded research in this area, and highlight the potential for such research to further improve outcomes for learners with disabilities.

 Artificial Intelligence Research Funding Opportunity

With funding from the American Rescue Plan, NCSER plans to support research under Theme 6, Track B: AI-Augmented Learning for Individuals with Disabilities. Proposals must discuss how the work will respond to the needs of learners with or at risk for a disability in an area where the COVID-19 pandemic has further widened existing gaps and/or resulted in decreased access and opportunities for students with disabilities to learn and receive support services. Please review the solicitation, the webinar (November 16), and the frequently asked questions for more information. Interested applicants should note the primary focus of this institute:

“The primary focus of an institute in AI-Augmented Learning includes research and development of AI-driven innovations to radically improve human learning and education. Achievement and opportunity gaps, particularly for learners from disadvantaged or underserved communities, have always been present, but COVID-19 has exacerbated them. Institute plans for this theme should address and measure outcomes with direct education impact, in both the short- and long- term, that have practical significance to educators, parents, or other decision-makers. Plans must also directly address algorithmic bias, model transparency, security and data privacy in the support of learning.”

Current NCSER-Funded Grants Applying Artificial Intelligence and Machine Learning

Prior to the new collaboration between IES and NSF, NCSER funded several grants that apply artificial intelligence and machine learning approaches, including those described below.

With a 2018 NCSER grant, Dr. Maithilee Kunda and her team at Vanderbilt University are building on a technology-based intervention known as Betty’s Brain. This computer-based instructional program for middle school science, designed with the support of a 2006 NCER grant, allows students to teach a computer agent to understand certain concepts, increasing their own knowledge and understanding. Dr. Kunda and her team are developing a new game called Film Detective, which is designed to improve theory of mind (ToM) reasoning in adolescents with autism spectrum disorder (ASD). More information about this project can be found in this IES blog.

With a 2021 NCSER grant, Dr. Patrick Kennedy and his team at University of Oregon are using machine learning to validate a well-known assessment, Dynamic Indicators of Basic Early Literacy Skills, 8^th Edition® (DIBELS 8) as a screener for dyslexia. As of 2020, 47 states require that students be screened for dyslexia in early elementary school and many states use DIBELS for this screening. However, it remains to be validated for this purpose. To address the validity of the DIBELS for screening, this research team is using machine learning approaches to predict and classify scores in relation to a pre-defined target. This will allow the research team to draw conclusions about the validity of the DIBELS 8 for dyslexia screening. These conclusions will be disseminated widely to state and local education agencies and other stakeholders.

The Potential of AI for Improving Outcomes for Learners with Disabilities

In addition to the work that IES is funding, AI has already demonstrated potential for improving outcomes for learners with disabilities in many other ways:

• AI has been used to support children with ASD who have difficulties understanding people’s emotions, with AI-driven apps and robots helping students practice emotion recognition and other social skills.
• AI has informed the development of algorithms that can help those involved in assessment identify disabilities in students, such as ASD, specific learning disabilities (dyslexia, dysgraphia, and dyscalculia), and attention-deficit/hyperactivity disorder (ADHD).
• AI-embedded interventions have included error analysis to inform instruction and personalized feedback in spelling and math for students with disabilities.

Despite these advancements, there appear to be persistent gaps in AI research for students with disabilities, such as AI for students with intellectual and developmental disabilities. This is an especially important area of work because many of these learners have multiple disabilities and/or serious health conditions. For example, children with intellectual and developmental disabilities who also have hearing loss or visual impairment have compounded challenges. Some students with Down syndrome also have hearing loss and other health complications, such as cardiac issues. AI affords an opportunity to integrate health information across different applications to improve the quality of life for these students. These technological solutions can assist in managing information about the students and communicating health information between teachers, physicians, and caregivers.

AI has the potential to transform special education. We hope that this NCSER-NSF partnership will encourage researchers to be creative in planning projects that move the field of AI forward as well as provide innovative solutions to support learners with disabilities.

This blog was co-authored by Sarah Brasiel (Sarah.Brasiel@ed.gov), program officer at NCSER and Bennett Lunn (Bennett.lunn@ed.gov), Truman-Albright Fellow for NCSER and the National Center for Education Research (NCER). IES encourages special education researchers to partners with experts in Artificial Intelligence to submit to this NSF AI Institute solicitation 22-502 to increase the evidence base on use of AI for this population.

The college pipeline from start to finish is an extraordinarily complex process with numerous decision points, options, and obstacles. Students from advantaged social backgrounds are more likely than those from disadvantaged backgrounds to attend schools and colleges staffed with advisers and support staff that have time and resources to assist them. They may also draw on relationships with family, adults in their communities, or knowledgeable peers for assistance in navigation decisions. For students without such supports, the sequence of choices may become so overwhelming that they respond by delaying decisions or making poor choices that lead to sizable delays in their degree progression. With these challenges in mind, IES has funded three information and advising initiatives that draw on insights from researchers, practitioners, and the research literature.

Technical Working Group Meeting

In July 2019, NCER convened a technical working group of 14 researchers and practitioners for a set of conversations structured around three intervention strategies that have garnered substantial attention over the last 5 years: nudges and other light-touch informational campaigns; intensive, proactive coaching and advising; and comprehensive approaches that comprise advising and other supports such as technology and financial incentives. Researchers and practitioners shared perceptions about the effectiveness of each strategy, its relevance to targeted student populations, and conditions for implementation. At the end of the day, working group members provided recommendations (see the Technical Working Group Meeting Summary for a full list), including the following:  

• Institutions should help determine what strategies get tested, apply for research grants, and participate in the research as it progresses.
• Research is needed that addresses the large amount of information that students face and that identifies the types of information that students respond to and act on.
• Replication studies should be designed to measure the effectiveness of promising intervention strategies for specific student groups, with the goal of enhancing effectiveness and cost-effectiveness.

Practice Guide on Effective Advising for Postsecondary Students

In October 2021, the What Works Clearinghouse (WWC) released a Practice Guide on Effective Advising for Postsecondary Students. The practice guide includes four evidence-based recommendations designed for an audience of administrators and staff at community colleges, 4-year institutions, and other public or private technical colleges who are responsible for designing and/or delivering advising to students:

• Intentionally design and deliver comprehensive, integrated advising that incorporates academic and non-academic supports to empower students to reach their educational goals.
• Transform advising to focus on the development of sustained, personalized relationships with individual students throughout their college career.
• Use mentoring and coaching to enhance comprehensive, integrated advising in ways that support students’ achievement and progression.
• Embed positive incentives in intentionally designed advising structures to encourage student participation and continued engagement.

Gap Analysis of Information and Advising Research and Practice

In March 2020, the Lead Team of the College Completion Network began a project aimed at identifying gaps in the research evidence base for information and advising strategies. The project is organized into three parts:

1. A systematic review of the research literature, documenting evidence of the effect of information and advising policies, practices, and programs on student outcomes
2. A scan of information and advising policies, practices, and programs that colleges use to improve student outcomes
3. A gap analysis to compare the findings from the scan to the findings from the systematic review to look for effective practices that are not widely implemented and promising practices in the field that have not been evaluated

The team plans to report its full set of findings by December 2021. College Completion Network study descriptions are available here: https://collegecompletionnetwork.org/studies.

This blog is the second in a blog series on Effective Postsecondary Interventions that highlights interventions with evidence of effectiveness generated through IES-funded research. For the first blog in the series, please see here.

Written by James Benson (James.Benson@ed.gov), a Program Officer for Postsecondary Education within NCER’s Policy and Systems Division, and Felicia Sanders (Felicia.Sanders@ed.gov), a Program Officer for the What Works Clearinghouse within NCEE’s Knowledge Use Division.

IES has funded scholars that push for equitable educational experiences. Dr. Lillian Durán is one researcher who stands out in this area. Her work has focused on improving instructional and assessment practices with preschool-aged dual language learners (DLLs). Dr. Durán recently was funded to expand the Individual Growth and Development Indicators (IGDI) suite of psychometrically robust measures for Spanish-speaking DLLs by developing and validating measures for 3-year-olds.  As a continuation of our Hispanic Heritage Month Series, we asked Dr. Durán to discuss her research with Hispanic student populations.

Lorena Aceves, a Society for Research Child Development Federal Postdoctoral Policy Fellow at the Administration for Children and Families’ Office of Head Start on detail with IES, asked Dr. Durán about her work and her experiences. See her responses below.

How have your background and experiences shaped your scholarship and career?

I am the first generation born in the United States. My mother was born in Rüstungen, Germany in 1931. This was in central Germany that was divided after WWII and became East Germany. She escaped as a young woman and made her way to the United States. My father was born in Nochistlán, Mexico in 1911, and his family migrated to the California when he was six years old because his father worked on building the railroads. In my home, we spoke German, Spanish and English, but English was my primary language. My personal experience in my family has fostered my interest in multilingual homes, and children who are growing up in first generation families.

Professionally, I became an early childhood special education teacher in 1998 and worked for 9 years both in Prince George’s County, Maryland and later in rural southwestern Minnesota. When I moved to Minnesota, I served three counties where Spanish-speaking children were about 25% of the population. I was the only teacher in nine school districts that spoke any Spanish, and I realized the incredible need in the field to support families who speak languages other than English, especially since there are so few teachers and specialists who are multilingual. In Minnesota, I was motivated to pursue a doctorate to fully immerse myself in understanding evidence-based solutions to serving multilingual children and their families.

What got you interested in a career in education science?

When I was a teacher, I had so many questions about best approaches to working with multilingual children and their families. I found myself looking for extra reading and trainings, but there was little information available to help me. At that time, I was a lead teacher and had signed up for my district to participate in a research project with Dr. Mary McEvoy out of the University of Minnesota. She was instrumental in encouraging me to apply to the doctoral program and agreed to be my advisor. In the end, she tragically passed away in an airplane accident, as many reading this will know, and Dr. Scott McConnell stepped in and took me on as an advisee. I tell this story because I think it is important to remember how important mentorship is to women of color out in the field and the incredible impact providing opportunities and encouragement can have. Without Mary pointing out my potential and giving me the confidence to even consider a doctorate, I might never have applied to a program.

In your area of work, what do you see as the greatest research needs or recommendations to address diversity and equity and improve the relevance of education research for diverse communities of students and families?

This is a complex question because the truth is there are many competing priorities. However, I believe an important priority at this point is to develop more effective bilingual language and literacy interventions that support meaningful improved outcomes reflecting community priorities and values. The interventions need to move beyond a singular focus on English language and literacy development to include culturally and linguistically sustaining practices in intervention design. We need to think much more deeply about the outcomes we are working to achieve and conduct more longitudinal research that can document change and performance over time. There is significant evidence that multilingual learners, in particular, need time to progress and that short-term studies cannot adequately capture more meaningful academic and life outcomes. Our current IES-funded project is looking to develop IGDIs for 3-year-olds to help educators make data-based decisions to improve children’s language and early literacy performance in Spanish, as well as to track growth in their development over time. I also think we need to conduct more research with a broader range of understudied populations including more cultures and languages to better understand their needs as the United States increases in diversity. In order to improve equity, we need to move beyond treating all multilingual students as one uniform group and begin to more systematically explore within group differences to effectively differentiate educational approaches to maximize outcomes.

What has been the biggest challenge you have encountered, and how did you overcome the challenge?

Quite honestly, the biggest challenge I have had to overcome in my life was my childhood. My parents had many challenges and struggles, and I had to care for my own needs and learn how to survive on my own from a very early age. I know this is personal, but I think this experience will resonate with many as we often do not address how many of us who go into education have experienced adverse early experiences ourselves and have had to draw on our inner phoenixes to get to where we are. Once I survived the first 18 years and was able to maintain my sense of self-worth, self-efficacy, and joy, there is not much else the world can throw at me that I can’t survive.

What advice would you give to emerging scholars from underrepresented, minoritized groups that are pursuing a career in education research?

The best advice I can give is to be true to yourself and have confidence in your intelligence and your contribution to the field. Change is difficult for many people, and there are many entrenched ideologies and practices in academic settings that might inhibit your creativity and ingenuity, but don’t let them! During my doctoral program, I had ideas about a Spanish version of the IGDIs. Initial reactions to the idea included, “Why do we need to measure kids in Spanish if we are teaching them in English?” I did not let that discourage me from reading and understanding what it would take to develop a measure in Spanish. After a decade of IES funding, it is clear there is a need for Spanish early language and literacy measures, and there is, in fact, currently a clear mandate to do a much better job of measuring children in their home languages to accurately capture their ability levels and reduce the likelihood that they will be underestimated reinforcing deficit-based stereotypes.

How does your research contribute to a better understanding of the importance of diversity, equity, and inclusion in education?

A critical but often overlooked part of education is assessment. Without accurate assessment, it is difficult to know whether what we are doing is working. I have had the great fortune to spend the last 10 years dedicated to Spanish assessment development. Having available high quality and psychometrically sound measures in Spanish that programs can use with confidence is critical to promoting equity in educational practices. It is important that measures developed in languages other than English are not simply translations of English measures, but rather true reflections of the cultural and linguistic characteristics of the population of interest. Technical manuals and evidence of the validity of the measure should be readily available just like they are for the English versions. Too often, measures developed in Spanish have undergone a less rigorous development process, and this does not support the accurate measurement of the ability levels of Spanish-speaking students. Therefore, my team’s assessment work has created a roadmap for embedding equity into measurement design, and I hope that our work leads to more strength-based approaches to assessment and intervention with young Spanish-speaking children that honors their home language and culture.

How can the broader education research community better support the careers and scholarship of researchers from underrepresented groups?

I think we need to create more accessible early career funding mechanisms for scholars of color and other underrepresented groups. Securing IES or NIH funding is a daunting process that realistically only pays off for very few of us. Smaller grants that can launch pilot work in emerging fields should be available to seed promising research careers and lines of research. This approach would support innovation and create space for more diverse scholarship and representation. We need to democratize the funding streams and think of new ways that scholars can enter the field with adequate support to launch their work.

Dr. Lillian Durán is an associate professor and associate dean for academic affairs at the College of Education at the University of Oregon.

This interview was produced and edited by Lorena Aceves, a Society for Research Child Development Federal Postdoctoral Policy Fellow at the Administration for Children and Families’ Office of Head Start on detail with National Center for Education Research, IES.