Inside IES Research

Research identifies benefits of access to same-race/ethnicity teachers for Black and Hispanic students. However, the teacher workforce is overwhelmingly White, and little is known about the system-level strategies that are successful at diversifying the profession. In recognition of Black History Month, we asked researcher Dr. David Blazar to discuss his recently awarded IES project that aims to advance the literature base on how school systems can recruit more Black teachers. This is what he shared.

What does existing research say about the need for more Black teachers?

Building on a longstanding theoretical and qualitative literature base from scholars including Gloria Ladson-Billings, Geneva Gay, Richard Milner, and many others, researchers have gathered causal evidence to support the claim of the benefit of Black teachers to Black students. Analyzing test score data from Tennessee's Project STAR experiment, Dee (2004) found that assignment to a Black teacher significantly increased the math and reading achievement of Black students.

Fast forward 18 years, and the research findings largely remain the same while the evidence base has grown substantially (see one meta-analysis, and a research synthesis). In the second experiment on this topic after Dee, my own recent analyses currently available in a working paper not only replicate the earlier test-score impacts, but also show that

• Test-scores effects (roughly 0.2 SD) persist at very similar magnitudes 6 years later when students are in high school, a rare pattern in education research
• Black and other underrepresented teachers of color have even larger effects (upwards of 0.45 SD) on the social-emotional development of their students of color and their White students
• Black and other teachers of color are much more likely than White teachers to hold mindsets and engage in classroom practices aligned to “culturally responsive teaching,” which in turn benefits a range of student outcomes

In short: The effects of Black teachers on the outcomes of Black students are larger than those of most other interventions as documented in the broader education research literature (generally no higher than 0.1 SD).

I pair these hugely meaningful findings with three more sobering facts:

• Black teachers are underrepresented in the teacher workforce. Roughly 7% of teachers nationally are Black, compared to roughly 15% of students. These patterns have not shifted much over the last several decades, even though calls to diversify the teacher workforce started over 30 years ago.
• The mismatch between student and teacher demographics may be due to “leaks” at multiple stages of the school-to-career pipeline, including lower rates of high school graduation amongst Black students relative to their White peers, similar gaps in college graduation rates, less interest in teaching as a career, and greater financial barriers and opportunity costs even when the interest is there.
• Despite impressive work by educators, scholars, and policymakers to design multiple strategies for recruiting Black individuals into teaching, the bulk of these remain “promising practices” rather than evidence-based best practices.

How will your IES-funded study address the need for more Black teachers?

Because the underrepresentation of Black teachers in U.S. schools is notable and longstanding, researchers and school systems must work together—and quickly—to consider multiple strategies. Stating that we need to diversify the teacher workforce is neither new nor novel. The imperative was posed several decades ago, and it is time that we figure out how best to do it.

To address this challenge head on, I am collaborating with Ramon Goings, Seth Gershenson, and other scholars, as well as with state agencies and policy actors in Maryland to explore several recruitment strategies aimed at diversifying the teacher workforce, implemented at different stages of the school-to-career pipeline.

Aligned to the theoretical literature, a core feature of our study is that we focus on strategies that look locally for prospective teaching talent and are therefore known as “grow-your-own” programs. These approaches aim to align the demographics of incoming teachers with the demographics of current student populations and ensure that those incoming teachers are familiar with the local area. We further designed our study to explore multiple components of and potential solutions to the policy problem, given that recruitment is unlikely to be addressed with a one-size-fits-all approach. Even though the partnership and data come from Maryland, the recruitment strategies and our study are relevant to the recruitment strategies used in states across the country.

The three strategies are—

• Early exposure to teaching in high school through the Teacher Academy of Maryland—a career and technical education program of study—that provides high school students with an opportunity to learn about teaching as a career, gain teaching experience in a real-world classroom, and earn an associate’s degree in teaching alongside their high school diploma.
• Financial support and incentives for college students, including the recently implemented Teaching Fellows for Maryland Scholarship. Scholarships aim to decrease financial barriers and opportunity costs that may prevent Black individuals from becoming teachers.
• Career-changer programs, such as alternative-route teacher certification and residency programs that both decrease barriers to entry into the profession and focus on recruiting locally.

Our analyses will provide some of the first quantitative data linking the rollout of varied recruitment strategies and the workforce decisions of prospective Black teachers. Beyond analyses of each individual program, our findings will provide important guidance not only about how best to intervene but also when to do so. We look forward to sharing what we find and to building an evidence base alongside other scholars and funding agencies tackling this important issue.

David Blazar is an Assistant Professor at the University of Maryland College Park (UMCP) in the Education Policy and Leadership program. He also is the Faculty Director of the Maryland Equity Project, a UMCP initiative to improve educational outcomes and close achievement gaps through research.

This interview blog is part of a larger IES blog series on diversity, equity, inclusion and accessibility (DEIA) in the education sciences. It was produced by Katina Stapleton (Katina.Stapleton@ed.gov), co-Chair of the IES Diversity and Inclusion Council, and Wai-Ying Chow (Wai-Ying.Chow@ed.gov), the Effective Instruction program officer within the National Center for Education Research.

This is the second of a two-part blog series from an IES-funded partnership project. The first part described how the process of cost-effectiveness analysis (CEA) provided useful information that led to changes in practice for a school nurse program and restorative practices at Jefferson County Public Schools (JCPS) in Louisville, KY. In this guest blog, the team discusses how the process of conducting value-added analysis provided useful program information over and above the information they obtained via CEA or academic return on investment (AROI).

Since we know you loved the last one, it’s time for another fun thought experiment! Imagine that you have just spent more than a year gathering, cleaning, assembling, and analyzing a dataset of school investments for what you hope will be an innovative approach to program evaluation. Now imagine the only thing your results tell you is that your proposed new application of value-added analysis (VAA) is not well-suited for these particular data. What would you do? Well, sit back and enjoy another round of schadenfreude at our expense. Once again, our team of practitioners from JCPS and researchers from Teachers College, Columbia University and American University found itself in a very unenviable position.

We had initially planned to use the rigorous VAA (and CEA) to evaluate the validity of a practical measure of academic return on investment for improving school budget decisions on existing school- and district-level investments. Although the three methods—VAA, CEA, and AROI—vary in rigor and address slightly different research questions, we expected that their results would be both complementary and comparable for informing decisions to reinvest, discontinue, expand/contract, or make other implementation changes to an investment. To that end, we set out to test our hypothesis by comparing results from each method across a broad spectrum of investments. Fortunately, as with CEA, the process of conducting VAA provided additional, useful program information that we would not have otherwise obtained via CEA or AROI. This unexpected information, combined with what we’d learned about implementation from our CEAs, led to even more changes in practice at JCPS.

Data Collection for VAA Unearthed Inadequate Record-keeping, Mission Drift, and More

Our AROI approach uses existing student and budget data from JCPS’s online Investment Tracking System (ITS) to compute comparative metrics for informing budget decisions. Budget request proposals submitted by JCPS administrators through ITS include information on target populations, goals, measures, and the budget cycle (1-5 years) needed to achieve the goals. For VAA, we needed similar, but more precise, data to estimate the relative effects of specific interventions on student outcomes, which required us to contact schools and district departments to gather the necessary information. Our colleagues provided us with sufficient data to conduct VAA. However, during this process, we discovered instances of missing or inadequate participant rosters; mission drift in how requested funds were actually spent; and mismatches between goals, activities, and budget cycles. We suspect that JCPS is not alone in this challenge, so we hope that what follows might be helpful to other districts facing similar scenarios.

More Changes in Practice 

The lessons learned during the school nursing and restorative practice CEAs discussed in the first blog, and the data gaps identified through the VAA process, informed two key developments at JCPS. First, we formalized our existing end-of-cycle investment review process by including summary cards for each end-of-cycle investment item (each program or personnel position in which district funds were invested) indicating where insufficient data (for example, incomplete budget requests or unavailable participation rosters) precluded AROI calculations. We asked specific questions about missing data to elicit additional information and to encourage more diligent documentation in future budget requests. 

Second, we created the Investment Tracking System 2.0 (ITS 2.0), which now requires budget requesters to complete a basic logic model. The resources (inputs) and outcomes in the logic model are auto-populated from information entered earlier in the request process, but requesters must manually enter activities and progress monitoring (outputs). Our goal is to encourage and facilitate development of an explicit theory of change at the outset and continuous evidence-based adjustments throughout the implementation. Mandatory entry fields now prevent requesters from submitting incomplete budget requests. The new system was immediately put into action to track all school-level Elementary and Secondary School Emergency Relief (ESSER)-related budget requests.

Process and Partnership, Redux

Although we agree with the IES Director’s insistence that partnerships between researchers and practitioners should be a means to (eventually) improving student outcomes, our experience shows that change happens slowly in a large district. Yet, we have seen substantial changes as a direct result of our partnership. Perhaps the most important change is the drastic increase in the number of programs, investments, and other initiatives that will be evaluable as a result of formalizing the end-of-cycle review process and creating ITS 2.0. We firmly believe these changes could not have happened apart from our partnership and the freedom our funding afforded us to experiment with new approaches to addressing the challenges we face.   

Stephen M. Leach is a Program Analysis Coordinator at JCPS and PhD Candidate in Educational Psychology Measurement and Evaluation at the University of Louisville.

Dr. Robert Shand is an Assistant Professor at American University.

Dr. Bo Yan is a Research and Evaluation Specialist at JCPS.

Dr. Fiona Hollands is a Senior Researcher at Teachers College, Columbia University.

If you have any questions, please contact Corinne Alfeld (Corinne.Alfeld@ed.gov), IES-NCER Grant Program Officer.

This is the first of a two-part guest blog series from an IES-funded partnership project between Teachers College, Columbia University, American University, and Jefferson County Public Schools in Kentucky. The purpose of the project is to explore academic return on investment (AROI) as a metric for improving decision-making around education programs that lead to improvements in student education outcomes. In this guest blog entry, the team showcases cost analysis as an integral part of education program evaluation.

Here’s a fun thought experiment (well, at least fun for researcher-types). Imagine you just discovered that two of your district partner’s firmly entrenched initiatives are not cost-effective. What would you do? 

Now, would your answer change if we told you that the findings came amidst a global pandemic and widespread social unrest over justice reform, and that those two key initiatives were a school nurse program and restorative practices? That’s the exact situation we faced last year in Jefferson County Public Schools (JCPS) in Louisville, KY. Fortunately, the process of conducting rigorous cost analyses of these programs unearthed critical evidence to help explain mostly null impact findings and inform very real changes in practice at JCPS.

Cost-Effectiveness Analysis Revealed Missing Program Components

Our team of researchers from Teachers College, Columbia University and American University, and practitioners from JCPS had originally planned to use cost-effectiveness analysis (CEA) to evaluate the validity of a practical measure of academic return on investment for improving school budget decisions. With the gracious support of JCPS program personnel in executing our CEAs, we obtained a treasure trove of additional quantitative and qualitative cost and implementation data, which proved to be invaluable.

Specifically, for the district’s school nurse program, the lack of an explicit theory of change, of standardized evidence-based practices across schools, and of a monitoring plan were identified as potential explanations for our null impact results. In one of our restorative practices cost interviews, we discovered that a key element of the program, restorative conferences, was not being implemented at all due to time constraints and staffing challenges, which may help explain the disappointing impact results.

Changes in Practice

In theory, our CEA findings indicated that JCPS should find more cost-effective alternatives to school nursing and restorative practices. In reality, however, both programs were greatly expanded; school nursing in response to COVID and restorative practices because JCPS leadership has committed to moving away from traditional disciplinary practices. Our findings regarding implementation, however, lead us to believe that key changes can lead to improved student outcomes for both.

In response to recommendations from the team, JCPS is developing a training manual for new nurses, a logic model illustrating how specific nursing activities can lead to better outcomes, and a monitoring plan. For restorative practices, while we still have a ways to go, the JCPS team is continuing to work with program personnel to improve implementation.

One encouraging finding from our CEA was that, despite imperfect implementation, suspension rates for Black students were lower in schools that had implemented restorative practices for two years compared to Black students in schools implementing the program for one year. Our hope is that further research will identify the aspects of restorative practices most critical for equitably improving school discipline and climate.

Process and Partnership

Our experience highlights unexpected benefits that can result when researchers and practitioners collaborate on all aspects of cost-effectiveness analysis, from collecting data to applying findings to practice. In fact, we are convinced that the ongoing improvements discussed here would not have been possible apart from the synergistic nature of our partnership. While the JCPS team included seasoned evaluators and brought front-line knowledge of program implementation, information systems, data availability, and district priorities, our research partners brought additional research capacity, methodological expertise, and a critical outsider’s perspective.

Together, we discovered that the process of conducting cost-effectiveness analysis can provide valuable information normally associated with fidelity of implementation studies. Knowledge gained during the cost analysis process helped to explain our less-than-stellar impact results and led to key changes in practice. In the second blog of this series, we’ll share how the process of conducting CEA and value-added analysis led to changes in practice extending well beyond the specific programs we investigated.

Stephen M. Leach is a Program Analysis Coordinator at JCPS and PhD Candidate in Educational Psychology Measurement and Evaluation at the University of Louisville.

Dr. Fiona Hollands is a Senior Researcher at Teachers College, Columbia University.

Dr. Bo Yan is a Research and Evaluation Specialist at JCPS.

Dr. Robert Shand is an Assistant Professor at American University.

If you have any questions, please contact Corinne Alfeld (Corinne.Alfeld@ed.gov), IES-NCER Grant Program Officer.

School environments are places in which students, particularly students of color, are exposed to implicit bias and discrimination that can negatively impact their academic outcomes. In this interview blog, we asked prevention scientist Dr. Chynna McCall to discuss how her career journey and her experiences working with children and families from diverse populations inspired her research on creating equitable school environments.   

How did you begin your career journey as a prevention scientist?

Perhaps my most valued professional experience is serving as a licensed school psychologist in public schools in Colorado, working with children and families from racially, culturally, and linguistically diverse populations. This experience inspired me to join the Missouri Prevention Science Institute in 2018 as an Institute of Education Sciences postdoctoral fellow, where I studied how to use research to solve real-world problems. More specifically, I learned how to use prevention science to develop and evaluate evidence-based practices and interventions that prevent negative social and emotional impacts before they happen. After my fellowship, I was hired and promoted to a senior research associate position at the Missouri Prevention Science Institute. In this role, I have operational responsibilities for various federally funded grants and conduct my own grant-funded research. Presently, I am working on the development and testing of an equity-focused social-emotional learning curriculum for 3rd through 5th grade students.

What challenges did you observe as a school psychologist?

As a school psychologist, I worked in two vastly different school districts. In one, most students came from low-income families, spoke English as a second language, and the school's performance on standardized tests was significantly below average. Most of the challenges I tackled during my time there could be categorized as social-emotional; most students had unbalanced home lives, and many suffered emotional or physical trauma. Because the school district pressured teachers to improve test scores, focus on behavior and classroom management unilaterally shifted towards scholastics. The unfortunate outcome was neglecting to acknowledge the role that student behavior and the root causes of those behaviors play in affecting academic outcomes. While the second district I worked for was a high-performing one with generally high socioeconomic status, I chose to work for the school designated for those children in the district who have serious emotional disabilities.

Even though there are stark differences between the two districts, I consistently encountered a need for students to develop better relationships with their teachers, peers, and parents, develop a better sense of self, and for teachers, other school personnel, students, and parents to have a better understanding of how their practices and interactions are impacting student social-emotional and academic outcomes.

How does your background as a school psychologist influence your research?

My experience as a school psychologist has reinforced my understanding of what is needed to improve public education and what research questions are of utmost importance. Through my time as a school psychologist, it helped me define the goals of my research, which include 1) understanding the influence of prejudice and discrimination on student internal and external behaviors and outcomes, 2) understanding how school personnel expression of prejudice and discrimination influence student internal and external behaviors and outcomes, and 3) determining how to most effectively develop an equitable school environment that positively influences marginalized and minoritized youth outcomes.

My research examines how school environment—including the prejudicial and discriminatory thoughts and behaviors of school staff, students, and guardians—influences identity development, identity expression (for example, racial identity, gender identity, sexuality, and intersectionality) and internal and external behaviors. The objective is to use this knowledge to create a school environment that facilitates prosocial student identity development. My research hinges on my observations and experiences as a practicing school psychologist to focus on how to shift differential outcomes observed in public education due to experiences of discrimination both in and out of the school setting.

In your area of research, what do you see as the greatest research needs or recommendations to address diversity and equity and improve the relevance of education research for diverse communities of students and families?

I believe schools at every level of education are microcosms for the greater society. How students traverse through the school system dictates how they will navigate through the macrocosm of society. How students navigate the school system can be improved if school systems are equipped with the tools that allow staff to prepare the students better academically, socially, and emotionally. These tools are essential for students who are having a difficult time because of cultural, linguistic, psychological, or physical differences from their peers. It is crucial for the research community to continually advocate for positive change in our education system, work towards better understanding student needs, and develop effective and efficient tools that better promote student growth and outcomes.

I also believe that researchers who study school environments must explicitly study bias. We have to look at whether and how school professionals are becoming aware of and challenging their implicit biases, as well as how students are becoming aware of bias and how they deal with it—either by internalizing it or challenging it. We also must look into how challenging or accepting bias affects students emotionally, behaviorally, and academically.

What advice would you give to emerging scholars from underrepresented, minoritized groups that are pursuing a career in education research?

See your perspective and experience as assets. Your perspective is underrepresented and is needed in making necessary changes to education and education outcomes. When you view your perspective as something of value, you are better able to determine what unaddressed research questions need to be asked and to move education research in a direction that is more inclusive of every student.

This year, Inside IES Research is publishing a series of interviews (see here, here, and here) showcasing a diverse group of IES-funded education researchers and fellows that are making significant contributions to education research, policy, and practice. As part of our Black History Month blog series, we are focusing on African American/Black researchers and fellows as well as researchers who focus on the education of Black students.

Dr. Chynna McCall is a Senior Research Associate with the Missouri Prevention Science Institute at the University of Missouri. Prior to this position, she was an IES postdoctoral fellow in the Missouri Interdisciplinary Postdoctoral Research and Training Program training program.

Produced by Corinne Alfeld (Corinne.Alfeld@ed.gov), postdoctoral training program officer, and Katina Stapleton (Katina.Stapleton@ed.gov), co-Chair of the IES Diversity and Inclusion Council and predoctoral training program officer.

Since 2004, the Institute of Education Sciences has funded predoctoral training programs to increase the number of well-trained PhD students who are prepared to conduct rigorous and relevant education research. In addition to providing training to doctoral students, all IES-funded predoctoral programs are encouraged to help broaden participation in the education sciences as part of their leadership activities. In this guest blog post, the leadership team of the University of Virginia predoctoral training program discusses their continuing efforts to create diverse pathways for students interested in education research.

In 2008, the IES-funded Virginia Education Science Training (VEST) Program began the Summer Undergraduate Research Program (SURP) with the goal of recruiting more students from traditionally marginalized groups into education science research. Each year, 8–10 students from around the United States traveled to receive faculty mentorship in independent research at the University of Virginia. In doing so, they experienced facilitated opportunities to develop new research skills and reflect about their own identities as scholars and students of color, first generation college students and/or students from families with low income. They became active members of research groups, visited IES program officers in Washington, DC, and presented their own research at the Leadership Alliance National Symposium.

Quite fortuitously, at an IES principal investigator meeting, we connected with the leadership of the IES-funded Research Institute for Scholars of Equity (RISE) program taking place at North Carolina Central University (NCCU). As a result, for four years, we collaborated with RISE leadership to host two-day RISE fellow visits to UVA. During these visits RISE fellows shared their projects and ideas with VEST fellows and faculty. The RISE and SURP fellows also mingled and attended workshops on graduate school admissions.

We had three goals for these efforts:

• Provide IES pre-doctoral fellows with the opportunity to apply leadership skills to working with undergraduates
• Increase the diversity of education scientists
• Increase the diversity of our IES-sponsored PhD program

Enter COVID. In 2020, bringing students to UVA for the summer wasn’t feasible or wise. Instead, we reflected on our past successful experiences with NCCU and realized we could improve the quality of student experiences if we also worked closely with faculty at other universities. To start, we engaged with Virginia State University (VSU) and Norfolk State University (NSU), two Virginia HBCUs, to create the Open Doors Program.

Initially, eight faculty and administrators from NSU and VSU met with the UVA team, which included a post-doctoral fellow and a PhD student who coordinated discussions, helped design the curriculum, and built an Open Doors handbook. The design team built a program in which 12 rising juniors at NSU and VSU would:

• Engage in the research and writing process that will lead to a research product and presentation that reflects their strengths, interests, and goals
• Gain a deeper understanding of the opportunities available to them in graduate school
• Have the opportunity to examine the complexities and multiple layers of their intersectional identities, identify assets and cultural wealth, and identify academic strengths and areas of growth
• Build relationships with faculty and graduate student mentors

Due to the pandemic, the program was offered virtually over four weeks with a combination of seminars and mentoring sessions. The program exceeded our expectations. The students all indicated that Open Doors was a useful learning experience for them and provided them with a better understanding of the opportunities available in graduate school. The faculty valued the opportunity to work with each other. We will be offering Open Doors 2.0 next June with another cohort of 12 students from NSU and VSU. We learned a lot from our first year and have planned several modifications to the program. For example, this year, we anticipate that students and some NSU and VSU faculty will be on campus at UVA for two of the four weeks; the other two weeks will be virtual.

These efforts have been true learning experiences for UVA faculty and VEST fellows. We have several recommendations for other programs eager to create pathways programs.

• Clarify your goals and organize the program around the key outcomes that you are trying to achieve. For SURP and Open Doors, we focused in on four outcomes: preparation to conduct education research, preparation for graduate school, expansion of networks, and providing access to new mentoring relationships.
• Teach skills as well as knowledge. Our evaluation of SURP points to the importance of teaching skills so students can formulate research questions, recognize research designs, analyze and interpret data, and write about research. Students reported gaining skills in these areas which are critical to success in graduate school in education research.
• Identify ways to enhance cultural capital. Students benefit from knowledge, familiarity, and comfort with university life. In Open Doors, we wanted to build an authentic collaboration that allowed faculty, graduate students, and undergraduate students at the HBCUs and UVA to learn from each other, extending the cultural capital of all participants.

Our efforts have been exciting yet humbling. Above all, we enjoy listening to and learning from the SURP and Open Doors students. In Open Doors, we also enjoyed building relationships with faculty at other institutions. We have increasingly become aware of the challenges we face in efforts to increase the diversity of our programs. Recruitment is just a first step. Creating graduate school experiences that are conducive to learning and engagement for students from diverse group is an important second step. And a third critical step is to transform life at our universities so that students (and faculty) from traditionally marginalized groups can thrive and flourish. In doing so, we expect that universities will be better able to meet a full range of new challenges that lie ahead in education science.

Sara Rimm-Kaufman is the Commonwealth Professor of Education in the Educational Psychology-Applied Developmental Science program at the University of Virginia School of Education and Human Development.

Jim Wyckoff is the Memorial Professor of Education and Public Policy in the Education Policy program and directs the Center on Education Policy and Workforce Competitiveness at the University of Virginia.

Jamie Inlow is the Coordinator for the VEST Predoctoral Training Program in the University of Virginia School of Education and Human Development.

This blog post is part of an ongoing series featuring IES training programs as well as our blog series on diversity, equity, inclusion, and accessibility (DEIA) within IES grant programs.

Produced by Katina Stapleton (Katina.Stapleton@ed.gov), co-Chair of the IES Diversity and Inclusion Council and predoctoral training program officer.