IES Blog

This guest blog was contributed by Dr. Holly Miller, who currently serves as a STEM Next Opportunity Fund Fellow at the Institute of Education Science’s National Center for Education Evaluation.

Since August 2022, I’ve been serving as the STEM Next Opportunity Afterschool and Summer Learning Fellow at the U.S. Department of Education (ED). More specifically, I work within the Department’s Institute of Education Sciences (IES).

Upon arriving at IES, I was charged with a specific challenge: amplify how evidence-based practice in out-of-school time (OST) can support student learning and development. This mission was made all the more relevant by the need for states and districts to respond to the consequences of the COVID pandemic which, at the time, remained an official national emergency.

Perhaps naively, I hoped to walk in on Day One and find “The Official Compendium of Evidence-based Practices in Global Pandemics and Related Crises” that I could pull off the shelf and hand to educators. Unfortunately, I quickly discovered no such tome existed. And I began to realize that one of the biggest challenges I’d face in my new role was getting to know ED itself! To an outsider, the Department can seem like a huge machine. Getting to know it, though, can pay incredible dividends. As I came to learn, there are tons of great resources—if only you know where to look.

One of OST educators’ first stops in getting to know ED should be IES. For the uninitiated, IES is the Department’s statistics, research, and evaluation arm. The mission of IES is to provide scientific evidence on which to ground education practice and policy and to share this information in formats that are useful and accessible to educators, parents, policymakers, researchers, and the public. It is independent and non-partisan.

Across its four centers—the National Centers for Education Statistics, Education Evaluation, Education Research, and Special Education Research—IES conducts six broad types of work (http://ies.ed.gov):

1. Providing data to describe the “condition of education,” including students’ academic proficiency.

2. Conducting surveys and sponsoring research projects to understand where education needs improvement and how these improvements might be made.

3. Funding development and rigorous testing of new approaches for improving education outcomes for all students.

4. Conducting large-scale evaluations of federal education programs and policies.

5. Providing resources to increase the use of data and research in education decision-making, including independent reviews of research on “what works” in education through the What Works Clearinghouse.

6. Supporting the advancement of statistics and research through specialized training and development of methods and measures.

I could see that this work had the potential to benefit a variety of stakeholders—teachers, administrators, students, researchers, and policymakers. Still, I had so many unanswered questions. As a middle school teacher, I frequently told students, “The only dumb question is the one you don’t ask.” Therefore, as I surveyed the education research landscape at IES, I asked lots and lots of questions. My presence at IES was akin to a toddler at the zoo for the first time: “What are those? Why is that so big? Why don’t we have more of these? When do we eat?” Months of asking and I find my queries have been distilled into two essential questions:

1. What has been the impact of the COVID pandemic on students and educators; and

2. How can education research, like that conducted or sponsored by IES, help us understand—and address—those impacts?

What has been the impact of the COVID-19 pandemic?

The pandemic disrupted nearly every aspect of daily life in the United States, including the education system. One of the most alarming impacts of the pandemic on education has been the widening of pre-existing gaps in student achievement and the resources that students need to be successful.

We all know the statistics … students have lost tons of learning. The "Report on the Condition of Education" is a congressionally mandated annual report from the National Center for Education Statistics (NCES). Using the most recent data available from NCES and other sources, the report contains key indicators on the condition of education in the United States at all levels, from prekindergarten through postsecondary, as well as labor force outcomes and international comparisons. For example, the report on the condition of education 2023 recently released shares that on both the 4th- and 8th-grade NAEP mathematics assessments, higher percentages of students performed below NAEP Basic in 2022 than in 2019 (Irwin et al., 2023).  This has been particularly bad among students who have historically been underserved. The average NAEP mathematics scores in 2022 were generally lower for English Learners (EL) students than for non-EL students; lower for those identified as students with disabilities than for their peers without disabilities; and higher for students in low-poverty schools than for students in high-poverty schools. These patterns were similar to those observed for reading (Irwin et al., 2023).

This is surely due, at least in part, to differences in the resources students have access to. Even before the pandemic, huge gaps in resources existed. The pandemic only made matters worse. According to a report by the U.S. Department of Education’s Office for Civil Rights (OCR) (2021), low-income students and students of color have been disproportionately negatively impacted by school shutdowns and remote learning practices. These students often lack access to reliable technology and internet resources, making it difficult for them to participate fully in online classes and complete assignments. Additionally, many students rely on meals provided by schools, so the closure of physical school buildings has led to food insecurity for some.

Also of note: the dramatic effect on student wellbeing. During the pandemic, mental health concerns such as fear, anxiety, and depression were common among the general public, especially children and older adults (Brooks et al., 2020; Pfefferbaum & North, 2020).  Research on the pandemic’s impact on mental health among students finds that “they showed increased fear, stress, and decreased happiness, and these were associated with their learning quality change.” (Hu et al., 2022).

Furthermore, the impact of COVID on educators is increasingly well-known. Educators had to make changes in short order, often with limited resources. This had consequences. Educators faced increased stress levels due to the shift to remote instruction, and many reported struggling to maintain a work-life balance while working from home. Findings indicate teachers reported greater mental health concerns than those in many other professions, and that remote teachers reported significantly higher levels of distress than those teaching in person (Kush et al., 2021). For some, it was too much, and they made the decision to leave the profession. Forty percent of public schools hiring for open teaching positions in special education in 2020–21 reported having difficulties filling the opening, compared with 17 percent in 2011–12 (Irwin et al., 2023) Not only were teachers leaving the workforce, but potential teachers were second-guessing their career choice. The number of persons enrolled in traditional teacher preparation programs decreased by 30 percent between 2012–13 and 2019–20, and the number of persons completing such programs decreased by 28 percent between 2012–13 and 2019–20 (Irwin et al., 2023).  

All of us are looking for solutions to all these problems. Given that I entered IES during the pandemic, I wanted to know how I could leverage its resources to help.

How can education research help?

First, I had to understand how IES, as a science agency, was structured to do the work of education research. My college textbook on education research (Newby, 2010) asserted that it should have three objectives: to explore issues and find answers to questions, to collect and disseminate information that shapes policy and decision-making, and to improve practice for practitioners.

It’s easy to see how the six broad areas of work at IES I listed above fit within those three objectives. For example, in normal (that is, pre-COVID) times, it’s the job of the National Center for Education Statistics (NCES) to collect and disseminate education-related statistics and information about student achievement to inform the work of researchers, policymakers, and other education decision-makers. IES’ two research Centers, the National Centers for Education Research (NCER) and Special Education Research (NCSER) support researchers’ exploration of a wide range of education topics and their use of high-quality methods to answer important questions of policy and practice. Finally, the National Center for Education Evaluation and Regional Assistance (NCEE) conducts its own rigorous evaluations of federal policies and programs; supports states and districts in the use of data, evidence, and applied research to improve local practice; and disseminates information about “what works” through its What Works Clearinghouse (WWC). In the wake of the pandemic, IES had to quickly focus its activities and resources to meet new demands across the education system. Here are just a few of the new questions that IES had to address amid the pandemic.

• What’s happening in schools, and who is learning in-person versus virtually or in hybrid settings? In late 2021, NCES leveraged work being done as part of the National Assessment of Educational Progress (NAEP) to meet an immediate need to better understand schools’ policies about learning mode, masking, and social distancing. In the weeks that followed, the School Pulse Panel was created (https://ies.ed.gov/schoolsurvey/spp/). Initially, the School Pulse focused on collecting monthly information on the impact of the COVID-19 pandemic from a national sample of elementary, middle, high, and combined-grade public schools. Over time, its focus has broadened. While some survey questions are asked repeatedly to observe trends over time, others are unique each month. IES is now able to provide regular and near-real-time snapshots into “what’s happening” in the nation’s schools on a wide range of topics that matter to educators, policymakers, and families.

• How can learning be accelerated in the wake of the pandemic? In addition to dozens of individual researcher-initiated projects on accelerating student learning, NCER and NCSER have funded three recovery-focused research networks. They include the Leveraging Evidence to Accelerate Recovery Nationwide (LEARN) Network (https://ies.ed.gov/funding/grantsearch/program.asp?ID=2123), the PreK-12 Research on Education Strategies to Advance Recovery and Turnaround (RESTART) Network (https://ies.ed.gov/funding/grantsearch/program.asp?ID=2124) and the Accelerating Recovery in Community College (ARCC) Network (https://ies.ed.gov/funding/grantsearch/program.asp?ID=2125). Additional investments were made specifically focused on meeting the needs of students with or at risk of disabilities, which were disproportionately affected by the pandemic. One, the Artificial Intelligence Institute for Transforming Education for Children with Speech and Language Processing Challenges, was cosponsored between IES and the National Science Foundation (https://ies.ed.gov/funding/grantsearch/details.asp?ID=5840).

• How can educators and caregivers support student learning in online, hybrid, and at-home settings? With schools closed and remote learning becoming the norm, educators and caregivers had to adapt their teaching methods and find new ways to engage students. As part of a mandate to provide assistance about “what works” in education, NCEE supported a series of efforts to bring together information for teachers navigating online and hybrid teaching environments and for caregivers who were providing instruction at home. NCEE commissioned work leading to the development of the “Best Practice in K-12 Online Teaching” minicourse (here), freely available from North Carolina State University, to support teachers new to online education in their transition to the medium. (The literature review on which the mini-course is based can be found here). NCEE’s Regional Educational Laboratories developed nearly 200 pandemic-related resources. Notable examples include “Supporting Your Child’s Reading at Home” (https://ies.ed.gov/ncee/rel/Products/Region/southeast/Resource/100679), which focuses on the development of early literacy skills, and “Teaching Math to Young Children for Families and Caregivers” (https://ies.ed.gov/ncee/rel/Products/Region/central/Resource/100652).

Since its inception in 2002, IES and its Centers have supported decision-makers—be they federal, state, or local—and educators in making use of high-quality evidence in their practice. The pandemic showed just important IES, its resources, and its infrastructure, can be.

In the pandemic’s wake, though, it seems to me that building even more evidence about “what works” is vital. The American Rescue Plan (ARP) provided historic levels of resources to expand educational opportunities and to ensure that education is better able to address the wide-ranging needs of students and their families – especially those who were disproportionately impacted by the pandemic. Many ARP investments, including those related to OST, have the requirement that programs be rooted in evidence-based practices. Because there are still things to learn about what makes strong programs, we can strengthen the field by building evidence that can address key problems of practice.

Conclusion

When I came to ED and IES, searching for information on how to use evidence-based practices to support COVID recovery within the context of OST, I was lost. As I’ve come to better understand the organization, I’ve learned that vast resources are available. Half of the battle was just figuring out “what lives where” within the Department! I hope this blog has given OST practitioners a bit of a roadmap to make their own process of discovery easier.

In Part Two of this series, I will explore how OST learning fits into ED, education research, and the post-pandemic education system. The latter has been profoundly affected, creating an opportunity for innovation and transformation in the delivery of education. The value of research cannot be underestimated in this context. As a result, my next blog will pose two questions. First, I’ll ask what the role of OST in learning recovery can be in the years ahead.  Then I’ll consider what evidence needs to be built to make the most of what OST can offer. I hope you’ll read it!

I’d love to hear your thoughts on this blog. Send them my way at holly.miller@ed.gov.

Citations

Brooks S.K., Webster R.K., Smith L.E., Woodland L., Wessely S., Greenberg N., Rubin G.J. (2020). The psychological impact of quarantine and how to reduce it: rapid review of the evidence. Lancet North Am. Ed. 395(10227):912–920.

Education in a Pandemic: The Disparate Impacts of COVID-19 on America's Students 2021 U.S. Department of Education’s Office for Civil Rights (OCR).

Hu K, Godfrey K, Ren Q, Wang S, Yang X, Li Q. (2022). The impact of the COVID-19 pandemic on college students in USA: Two years later. Psychiatry Res. Sep; 315:114685.

Huck, C., & Zhang, J. (2021). Effects of the COVID-19 Pandemic on K-12 Education: A Systematic Literature Review. New Waves-Educational Research and Development Journal, 24(1), 53-84.

Irwin, V., Wang, K., Tezil, T., Zhang, J., Filbey, A., Jung, J., ... & Parker, S. (2023). Report on the Condition of Education 2023. NCES 2023-144. National Center for Education Statistics.

Kush, J. M., Badillo-Goicoechea, E., Musci, R. J., & Stuart, E. A. (2021). Teacher mental health during the COVID-19 pandemic: informing policies to support teacher well-being and effective teaching practices.

Newby, P. (2010). Research Methods for Education. Pearson Education.

Pfefferbaum B., North C.S. (2020). Mental health and the Covid-19 pandemic. N. Engl. J. Med.;383(6):510–512.

As the primary statistical agency of the U.S. Department of Education, the National Center for Education Statistics (NCES) is mandated to report complete statistics on the condition of American education. While the condition of an education system is often assessed through indicators of achievement and attainment, NCES is also mandated to report on the conditions of the education workplace.

As such, NCES has reported timely information from schools. For example, this past December, NCES released data that indicated that public schools have experienced difficulty filling positions throughout the COVID-19 pandemic.¹ In order to understand the broader labor situation, NCES is utilizing the Job Openings and Labor Turnover Survey to describe the tightness of the job market.

JOLTS Design

The Job Openings and Labor Turnover Survey (JOLTS), conducted by the U.S. Bureau of Labor Statistics (BLS), provides monthly estimates of job openings, hires, and total separations. The purpose of JOLTS data is to serve as demand-side indicators of labor shortages at the national level.²

The JOLTS program reports labor demand and turnover estimates by industry, including education.³ As such, this analysis focuses on the public state and local education industry (“state and local government education” as referred to by JOLTS),⁴ which includes all persons employed by public elementary and secondary school systems and postsecondary institutions.

The JOLTS program does not produce estimates by Standard Occupational Classification.⁵ When reviewing these findings, please note occupations⁶ within the public state and local education industry vary⁷ (e.g., teachers and instructional aides, administrators, cafeteria workers, transportation workers). Furthermore, as the JOLTS data are tabulated at the industry level, the estimates are inclusive of the elementary, secondary, and postsecondary education levels.

Analysis

In this blog post, we present selected estimates on the number and rate of job openings, hires, and total separations (quits, layoffs and discharges, and other separations). The job openings rate is computed by dividing the number of job openings by the sum of employment and job openings. All other metric rates (hires, total separations, quits, layoffs and discharges, and other separations) are defined by taking the number of each metric and dividing it by employment. Fill rate is defined as the ratio of the number of hires to the number of job openings, and the churn rate is defined as the sum of the rate of hires and the rate of total separations.⁸

Table 1. Number of job openings, hires, and separations and net change in employment in public state and local education, in thousands: January 2020 through January 2023

*Significantly different from January 2023 (p < .05).
¹ Net employment changes are calculated by taking the difference between the number of hires and the number of separations. When the number of hires exceeds the number of separations, employment rises—even if the number of hires is steady or declining. Conversely, when the number of hires is less than the number of separations, employment declines—even if the number of hires is steady or rising.
NOTE: Data are not seasonally adjusted. Detail may not sum to totals because of rounding.
SOURCE: U.S. Department of Labor, Bureau of Labor Statistics, Job Openings and Labor Turnover Survey (JOLTS), 2020–2023, based on data downloaded April 5, 2023, from https://data.bls.gov/cgi-bin/dsrv?jt.

[2] U.S. Bureau of Labor Statistics. Job Openings and Labor Turnover Survey. Retrieved March 28, 2023, from https://www.bls.gov/jlt/jltover.htm.

[3] For more information about these estimates, see https://www.bls.gov/news.release/jolts.tn.htm.

[4] JOLTS refers to this industry as state and local government education, which is designated as ID 92.

[5] For more information on the reliability of JOLTS estimates, see https://www.bls.gov/jlt/jltreliability.htm.

[6] North American Industry Classification System (NAICS) is a system for classifying establishments (individual business locations) by type of economic activity. The Standard Occupational Classification (SOC) classifies all occupations for which work is performed for pay or profit. To learn more on the differences between NAICS and SOC, see https://www.census.gov/topics/employment/industry-occupation/about/faq.html.

[7] JOLTS data are establishment based, and there is no distinction between occupations within an industry. If a teacher and a school nurse were hired by an establishment coded as state and local government education, both would fall under that industry. (From email communication with JOLTS staff, April 7, 2023.)

[8] Skopovi, S., Calhoun, P., and Akinyooye, L. Job Openings and Labor Turnover Trends for States in 2020. Beyond the Numbers: Employment & Unemployment, 10(14). Retrieved March 28, 2023, from https://www.bls.gov/opub/btn/volume-10/jolts-2020-state-estimates.htm.

June is LGBTQ+ Pride Month, and NCES is proud to share some of the work we have undertaken to collect data on the characteristics and well-being of sexual and gender minority populations.

Inclusion of questions about sexual orientation and gender identity on federal surveys allows for better understanding of sexual and gender minority populations relative to the general population. These sexual orientation and gender identity (SOGI) data meet a critical need for information to understand trends within larger population groups, and insights gained from analysis of the data can lead to potential resources and needed interventions being provided to better serve the community. Giving respondents the opportunity to describe themselves and bring their “whole self” to a questionnaire helps them to be seen and heard by researchers and policymakers.

Sometimes, NCES is asked why questions like this appear on an education survey. They can be sensitive questions for some people, after all. NCES asks these questions to be able to understand the different experiences, equity, and outcomes related to education for sexual and gender minorities, just as NCES does for groups identified by other demographic characteristics like race, ethnicity, household income, and what part of the country someone lives in. By sexual minorities, we mean people who report their sexual orientation to be something other than straight or heterosexual, and by gender minorities, we mean people whose sex as recorded at birth is different from their gender.

Over the past 10 years, NCES has researched how to best ask respondents about their sexual orientation and gender identity, how respondents react to these questions, and the quality of data that NCES has collected on these characteristics.

At NCES, several studies include background questions for adults about their sexual orientation and gender identity. These are the High School Longitudinal Study: 2009 (HSLS:09) Second Follow-up in 2016, the Baccalaureate and Beyond Longitudinal Study (B&B) 08/18 and 16/21 collections, the National Postsecondary Student Aid Study (NPSAS) in 2020, and the Beginning Postsecondary Students Longitudinal Study (BPS) 2020/22 (see table below for more details about these surveys).

The collection of these data allows NCES to describe the experiences of gender and sexual minority individuals. For example:

• In 2020, postsecondary students who identified as genderqueer, gender nonconforming, or a different identity had difficulty finding safe and stable housing at three times the rate (9 percent) of students who identified as male or female (3 percent each).¹
   
• In 2018, about 10 years after completing a 2007–08 bachelor’s degree, graduates who were gender minorities² described their financial situations. Graduates who were gender minorities were less likely to own a home (31 percent) or hold a retirement account (74 percent) than graduates who were not gender minorities (63 percent and 87 percent, respectively) (figure 1).³  

The Common Core of Data (CCD) contains basic information on public elementary and secondary schools, local education agencies (LEAs), and state education agencies (SEAs) in the United States. The CCD collects fiscal and nonfiscal data about all public schools, public school districts, and state education agencies in the United States. Both IPEDS and CCD provide a sampling frame to many survey collections, including many conducted by NCES and the Department of Education. This blog post, one in a series of posts about CCD nonfiscal data, focuses on CCD’s two major releases and their corresponding components. For information on how to access and use CCD data, read the blog post Accessing the Common Core of Data (CCD).
 

Data Releases

CCD nonfiscal data are published in two releases every school year—as preliminary files and as provisional data files—within the CCD Data File tool. Understanding the differences between the two releases is important to understand how CCD nonfiscal data are released.

• The preliminary files contain basic information about schools and districts, such as name, address, phone number, status, and NCES ID number. Many schools and districts utilize information from the directory file, such as the NCES ID, to apply for grants or other opportunities for their schools. Therefore, it is important that these files are released first, even if the data are still preliminary. 
   
• The provisional data files are the full release of the CCD nonfiscal data. These data files provide school-, district-, and state-level data on topics like enrollment, staffing, and free or reduced-price lunch. These files are much more detailed and include data that are broken down by characteristics such as grade, race/ethnicity, and gender—as well as by combinations of these characteristics. These files are not updated unless there is a significant change to the data.

Each file release includes a version that indicates the type of release. The first preliminary files have “0a” in the file names, and revised preliminary files include “0b,” “0c,” and so on. The first provisional files have “1a,” in the file names, and revised provisional files include “1b,” “1c,” and so on. Note, however, that releasing revised files is rare.
 

Components of a Release

It is important to utilize the various components that accompany each release to find additional information that is specific to the file and can help you better understand the data. In addition, there are other resources available that provide more ways to access and understand the data.
 

Documentation Components

Every data file will have documentation files that provide information about the data. These include the following:

• release notes—basic information about the data release, including details about any changes to the files, such as a change in a variable’s description or a variable that was added to the file; summary tables that include national totals and tables with selected frequencies are also included.
   
• state data notes—information on data anomalies that are discovered during NCES’s collaboration with the states; broken down by state and by file type, these notes describe things like changes to how data were collected by the state.
   
• companion files—included in each data file component, these files include a list of all the variables in the data file—including a brief description—and frequency tables; you should start with the companion files to better understand what variables are in each data file.

Resources and Tools

Along with the release of the CCD nonfiscal data files, additional resources are also updated to improve access to the data.

• Summary Tables: Released with the provisional data files, Summary Tables provide a national-level look at the data. These tables show the operational status of schools and districts by type as well as the number of schools, students, and teachers by state.
   
• Locators and ElSi: There are two primary tools that can be used to access CCD data: the Locators (School Locator and District Locator) and the Elementary/Secondary Information System (ElSi). These tools are updated as the data files are released. The Locators are updated with each release, while ElSi is updated with the release of the provisional data files. Learn more about these tools.
   
• Online Documentation: The online documentation provides some general information about CCD. This information is not year specific, but it provides a detailed explanation about how the data are collected, processed, and reviewed.

• Reference Library: The reference library includes detailed documentation on various components of the CCD files that applies to multiple years, levels, and components of the data collection. The library includes crosswalks, documentation describing changes to the collection, and guidance for utilizing the data files, such as how to aggregate free or reduced-price lunch data.

Be sure to follow NCES on Twitter, Facebook, LinkedIn, and YouTube and subscribe to the NCES News Flash to stay up-to-date on future CCD releases and resources.

By Patrick Keaton, NCES

Recently, the Office of the Chief Statistician, within the Office of Management and Budget (OMB), released an initial set of recommended revisions for OMB’s Statistical Policy Directive No. 15 (SPD 15), which provides the statistical standards for collecting and reporting race and ethnicity data across federal agencies. The revisions were proposed by an Interagency Technical Working Group.

This is the next step in a process that began last summer with a simple goal: to ensure that the standards better reflect the diversity of the American people. The initial proposals—developed by federal government staff representing more than 20 agencies—include the following:

• collecting race and ethnicity together with a single question
• adding a response category for Middle Eastern and North African that is separate and distinct from the “White” category
• updating SPD 15’s terminology, definitions, and question wording

These recommendations are preliminary—not final—and they do not represent the positions of OMB or the agencies participating in the Working Group.

The Working Group is committed to a full, transparent revision process and remains on track to reach the goal of completing these important revisions by the summer of 2024.

The Working Group Wants to Hear Directly From the American People

The public’s participation in this process will play a critical role in helping the Working Group improve the way federal agencies safely and accurately collect and use information on the race and ethnicity of our diverse population.

Interested stakeholders can read the full Federal Register Notice and provide comments, participate in one of the Working Group’s bi-monthly virtual listening sessions or upcoming virtual town halls, and schedule a listening session.

Be sure to follow NCES on Twitter, Facebook, LinkedIn, and YouTube and subscribe to the NCES News Flash to receive notifications about the revision process and opportunities to engage with the Working Group.