This guest blog was contributed by Dr. Holly Miller, who currently serves as a STEM Next Opportunity Fund Fellow at the Institute of Education Science’s National Center for Education Evaluation.
Since August 2022, I’ve been serving as the STEM Next Opportunity Afterschool and Summer Learning Fellow at the U.S. Department of Education (ED). More specifically, I work within the Department’s Institute of Education Sciences (IES).
Upon arriving at IES, I was charged with a specific challenge: amplify how evidence-based practice in out-of-school time (OST) can support student learning and development. This mission was made all the more relevant by the need for states and districts to respond to the consequences of the COVID pandemic which, at the time, remained an official national emergency.
Perhaps naively, I hoped to walk in on Day One and find “The Official Compendium of Evidence-based Practices in Global Pandemics and Related Crises” that I could pull off the shelf and hand to educators. Unfortunately, I quickly discovered no such tome existed. And I began to realize that one of the biggest challenges I’d face in my new role was getting to know ED itself! To an outsider, the Department can seem like a huge machine. Getting to know it, though, can pay incredible dividends. As I came to learn, there are tons of great resources—if only you know where to look.
One of OST educators’ first stops in getting to know ED should be IES. For the uninitiated, IES is the Department’s statistics, research, and evaluation arm. The mission of IES is to provide scientific evidence on which to ground education practice and policy and to share this information in formats that are useful and accessible to educators, parents, policymakers, researchers, and the public. It is independent and non-partisan.
Across its four centers—the National Centers for Education Statistics, Education Evaluation, Education Research, and Special Education Research—IES conducts six broad types of work (http://ies.ed.gov):
1. Providing data to describe the “condition of education,” including students’ academic proficiency.
2. Conducting surveys and sponsoring research projects to understand where education needs improvement and how these improvements might be made.
3. Funding development and rigorous testing of new approaches for improving education outcomes for all students.
4. Conducting large-scale evaluations of federal education programs and policies.
5. Providing resources to increase the use of data and research in education decision-making, including independent reviews of research on “what works” in education through the What Works Clearinghouse.
6. Supporting the advancement of statistics and research through specialized training and development of methods and measures.
I could see that this work had the potential to benefit a variety of stakeholders—teachers, administrators, students, researchers, and policymakers. Still, I had so many unanswered questions. As a middle school teacher, I frequently told students, “The only dumb question is the one you don’t ask.” Therefore, as I surveyed the education research landscape at IES, I asked lots and lots of questions. My presence at IES was akin to a toddler at the zoo for the first time: “What are those? Why is that so big? Why don’t we have more of these? When do we eat?” Months of asking and I find my queries have been distilled into two essential questions:
- What has been the impact of the COVID pandemic on students and educators; and
- How can education research, like that conducted or sponsored by IES, help us understand—and address—those impacts?
What has been the impact of the COVID-19 pandemic?
The pandemic disrupted nearly every aspect of daily life in the United States, including the education system. One of the most alarming impacts of the pandemic on education has been the widening of pre-existing gaps in student achievement and the resources that students need to be successful.
We all know the statistics … students have lost tons of learning. The "Report on the Condition of Education" is a congressionally mandated annual report from the National Center for Education Statistics (NCES). Using the most recent data available from NCES and other sources, the report contains key indicators on the condition of education in the United States at all levels, from prekindergarten through postsecondary, as well as labor force outcomes and international comparisons. For example, the report on the condition of education 2023 recently released shares that on both the 4th- and 8th-grade NAEP mathematics assessments, higher percentages of students performed below NAEP Basic in 2022 than in 2019 (Irwin et al., 2023). This has been particularly bad among students who have historically been underserved. The average NAEP mathematics scores in 2022 were generally lower for English Learners (EL) students than for non-EL students; lower for those identified as students with disabilities than for their peers without disabilities; and higher for students in low-poverty schools than for students in high-poverty schools. These patterns were similar to those observed for reading (Irwin et al., 2023).
This is surely due, at least in part, to differences in the resources students have access to. Even before the pandemic, huge gaps in resources existed. The pandemic only made matters worse. According to a report by the U.S. Department of Education’s Office for Civil Rights (OCR) (2021), low-income students and students of color have been disproportionately negatively impacted by school shutdowns and remote learning practices. These students often lack access to reliable technology and internet resources, making it difficult for them to participate fully in online classes and complete assignments. Additionally, many students rely on meals provided by schools, so the closure of physical school buildings has led to food insecurity for some.
Also of note: the dramatic effect on student wellbeing. During the pandemic, mental health concerns such as fear, anxiety, and depression were common among the general public, especially children and older adults (Brooks et al., 2020; Pfefferbaum & North, 2020). Research on the pandemic’s impact on mental health among students finds that “they showed increased fear, stress, and decreased happiness, and these were associated with their learning quality change.” (Hu et al., 2022).
Furthermore, the impact of COVID on educators is increasingly well-known. Educators had to make changes in short order, often with limited resources. This had consequences. Educators faced increased stress levels due to the shift to remote instruction, and many reported struggling to maintain a work-life balance while working from home. Findings indicate teachers reported greater mental health concerns than those in many other professions, and that remote teachers reported significantly higher levels of distress than those teaching in person (Kush et al., 2021). For some, it was too much, and they made the decision to leave the profession. Forty percent of public schools hiring for open teaching positions in special education in 2020–21 reported having difficulties filling the opening, compared with 17 percent in 2011–12 (Irwin et al., 2023) Not only were teachers leaving the workforce, but potential teachers were second-guessing their career choice. The number of persons enrolled in traditional teacher preparation programs decreased by 30 percent between 2012–13 and 2019–20, and the number of persons completing such programs decreased by 28 percent between 2012–13 and 2019–20 (Irwin et al., 2023).
All of us are looking for solutions to all these problems. Given that I entered IES during the pandemic, I wanted to know how I could leverage its resources to help.
How can education research help?
First, I had to understand how IES, as a science agency, was structured to do the work of education research. My college textbook on education research (Newby, 2010) asserted that it should have three objectives: to explore issues and find answers to questions, to collect and disseminate information that shapes policy and decision-making, and to improve practice for practitioners.
It’s easy to see how the six broad areas of work at IES I listed above fit within those three objectives. For example, in normal (that is, pre-COVID) times, it’s the job of the National Center for Education Statistics (NCES) to collect and disseminate education-related statistics and information about student achievement to inform the work of researchers, policymakers, and other education decision-makers. IES’ two research Centers, the National Centers for Education Research (NCER) and Special Education Research (NCSER) support researchers’ exploration of a wide range of education topics and their use of high-quality methods to answer important questions of policy and practice. Finally, the National Center for Education Evaluation and Regional Assistance (NCEE) conducts its own rigorous evaluations of federal policies and programs; supports states and districts in the use of data, evidence, and applied research to improve local practice; and disseminates information about “what works” through its What Works Clearinghouse (WWC). In the wake of the pandemic, IES had to quickly focus its activities and resources to meet new demands across the education system. Here are just a few of the new questions that IES had to address amid the pandemic.
- What’s happening in schools, and who is learning in-person versus virtually or in hybrid settings? In late 2021, NCES leveraged work being done as part of the National Assessment of Educational Progress (NAEP) to meet an immediate need to better understand schools’ policies about learning mode, masking, and social distancing. In the weeks that followed, the School Pulse Panel was created (https://ies.ed.gov/schoolsurvey/spp/). Initially, the School Pulse focused on collecting monthly information on the impact of the COVID-19 pandemic from a national sample of elementary, middle, high, and combined-grade public schools. Over time, its focus has broadened. While some survey questions are asked repeatedly to observe trends over time, others are unique each month. IES is now able to provide regular and near-real-time snapshots into “what’s happening” in the nation’s schools on a wide range of topics that matter to educators, policymakers, and families.
- How can educators and caregivers support student learning in online, hybrid, and at-home settings? With schools closed and remote learning becoming the norm, educators and caregivers had to adapt their teaching methods and find new ways to engage students. As part of a mandate to provide assistance about “what works” in education, NCEE supported a series of efforts to bring together information for teachers navigating online and hybrid teaching environments and for caregivers who were providing instruction at home. NCEE commissioned work leading to the development of the “Best Practice in K-12 Online Teaching” minicourse (here), freely available from North Carolina State University, to support teachers new to online education in their transition to the medium. (The literature review on which the mini-course is based can be found here). NCEE’s Regional Educational Laboratories developed nearly 200 pandemic-related resources. Notable examples include “Supporting Your Child’s Reading at Home” (https://ies.ed.gov/ncee/rel/Products/Region/southeast/Resource/100679), which focuses on the development of early literacy skills, and “Teaching Math to Young Children for Families and Caregivers” (https://ies.ed.gov/ncee/rel/Products/Region/central/Resource/100652).
Since its inception in 2002, IES and its Centers have supported decision-makers—be they federal, state, or local—and educators in making use of high-quality evidence in their practice. The pandemic showed just important IES, its resources, and its infrastructure, can be.
In the pandemic’s wake, though, it seems to me that building even more evidence about “what works” is vital. The American Rescue Plan (ARP) provided historic levels of resources to expand educational opportunities and to ensure that education is better able to address the wide-ranging needs of students and their families – especially those who were disproportionately impacted by the pandemic. Many ARP investments, including those related to OST, have the requirement that programs be rooted in evidence-based practices. Because there are still things to learn about what makes strong programs, we can strengthen the field by building evidence that can address key problems of practice.
Conclusion
When I came to ED and IES, searching for information on how to use evidence-based practices to support COVID recovery within the context of OST, I was lost. As I’ve come to better understand the organization, I’ve learned that vast resources are available. Half of the battle was just figuring out “what lives where” within the Department! I hope this blog has given OST practitioners a bit of a roadmap to make their own process of discovery easier.
In Part Two of this series, I will explore how OST learning fits into ED, education research, and the post-pandemic education system. The latter has been profoundly affected, creating an opportunity for innovation and transformation in the delivery of education. The value of research cannot be underestimated in this context. As a result, my next blog will pose two questions. First, I’ll ask what the role of OST in learning recovery can be in the years ahead. Then I’ll consider what evidence needs to be built to make the most of what OST can offer. I hope you’ll read it!
I’d love to hear your thoughts on this blog. Send them my way at holly.miller@ed.gov.
Citations
Brooks S.K., Webster R.K., Smith L.E., Woodland L., Wessely S., Greenberg N., Rubin G.J. (2020). The psychological impact of quarantine and how to reduce it: rapid review of the evidence. Lancet North Am. Ed. 395(10227):912–920.
Education in a Pandemic: The Disparate Impacts of COVID-19 on America's Students 2021 U.S. Department of Education’s Office for Civil Rights (OCR).
Hu K, Godfrey K, Ren Q, Wang S, Yang X, Li Q. (2022). The impact of the COVID-19 pandemic on college students in USA: Two years later. Psychiatry Res. Sep; 315:114685.
Huck, C., & Zhang, J. (2021). Effects of the COVID-19 Pandemic on K-12 Education: A Systematic Literature Review. New Waves-Educational Research and Development Journal, 24(1), 53-84.
Irwin, V., Wang, K., Tezil, T., Zhang, J., Filbey, A., Jung, J., ... & Parker, S. (2023). Report on the Condition of Education 2023. NCES 2023-144. National Center for Education Statistics.
Kush, J. M., Badillo-Goicoechea, E., Musci, R. J., & Stuart, E. A. (2021). Teacher mental health during the COVID-19 pandemic: informing policies to support teacher well-being and effective teaching practices.
Newby, P. (2010). Research Methods for Education. Pearson Education.
Pfefferbaum B., North C.S. (2020). Mental health and the Covid-19 pandemic. N. Engl. J. Med.;383(6):510–512.