For more than 50 years, the RELs have collaborated with school districts, state departments of education, and other education stakeholders to help them generate and use evidence and improve student outcomes. Read more
Home Blogs New study estimates student learning changes following school building closures due to the COVID-19 pandemic
The COVID-19 pandemic continues to disrupt student learning across the country. In Illinois, all public and nonpublic school buildings were closed by executive order in March 2020 to protect the health and safety of students and educators during the pandemic. Districts across the state shifted to remote instruction for the rest of the 2019/20 school year, and many continued with distance education, or a hybrid of distance and in-person learning, in the 2020/21 school year.
To understand the relationship between these school building closures and the suspension of in-person instruction, Illinois state and district leaders sought research and evidence to inform decision making related to providing targeted or differentiated supports to students. Regional Educational Laboratory (REL) Midwest partnered with the Illinois State Board of Education to conduct a study that examines student learning changes that occurred as schools shifted to remote learning during the pandemic.
>> Read and download the full report and related infographic.
The Illinois State Board of Education requested this study to understand the nature of the learning changes that occurred in the state during the pandemic. The study team explored how student learning in mathematics and reading changed as of fall 2020 following extended school building closures due to the COVID-19 pandemic. The study also examined how changes to student learning varied by student characteristics and district size.
To conduct the study, the Illinois State Board of Education invited all Illinois districts serving students in grades 3–8 to participate by sharing their interim assessment data from fall 2020 and multiple prior years. The state agency collected data from 17 Illinois districts. The REL Midwest team then analyzed the data to measure how student learning had changed in fall 2020 relative to fall terms prior to the pandemic. The study examined a combination of fall Northwest Evaluation Association Measures of Academic Progress (NWEA MAP) data provided by the participating districts as well as student, school, and district records provided by the Illinois State Board of Education. Several additional districts were interested in the study but did not have sufficient data at the time to support participation. REL Midwest provided training to these districts to help them understand strategies they could implement on their own to assess the learning changes their students had experienced and to support their use of this information in conversations and planning.
Key findings include the following:
These findings were generated after adjusting for other factors, including students’ eligibility for the national school lunch program, race/ethnicity, English learner student status, special education status, and prior academic achievement, as well as a time trend and other factors.
In addition, the study had several limitations that should be considered when reviewing the report findings. First, because all students in Illinois experienced the effects of the pandemic at the same time, there is no comparison group against which to measure academic outcomes, and the study should not be viewed as causal. Second, changes with the fall NWEA MAP assessment may have affected the findings. Prior to the pandemic, the NWEA MAP was administered to all students in person. In fall 2020, many districts administered the assessment to students remotely. Third, the study sample only includes four comparison time points prior to the onset of the COVID-19 pandemic. Thus, if student test scores tended to fluctuate from year to year, the study may not have estimated changes in learning precisely. Finally, the study districts, and the students they serve, differ from the overall population of districts in Illinois in multiple ways. For instance, none of the districts in the study sample are in the southern regions of the state, and the sample does not include Chicago Public Schools, which enrolls 18 percent of all public school students in Illinois. For more information, see the full report.
The study’s findings can inform the strategies implemented by Illinois education leaders as they allocate resources and provide supports for students as they return to full-time in-person learning.
School and district leaders may want to identify and implement evidence-based strategies and interventions that can accelerate learning in mathematics. In five of the six grade levels analyzed, students scored worse than expected in mathematics following the onset of the pandemic, after adjusting for other factors.
Additional research is needed to further inform districts’ and schools’ efforts, as well as state-level efforts, to support students through the duration of the pandemic and beyond. District and school administrators working to allocate resources and support students with learning recovery may want to gather additional information that can help identify why student learning changed as it did. Administrators may want to focus these efforts on student groups that experienced the most unfinished learning and identify the drivers of the learning changes. Education leaders can then use the information to identify and provide appropriate assistance to support the targeted groups’ learning recovery.
Research may also explore changes in student learning for a more comprehensive and representative sample of students. This study used data from a small sample of districts, whose students differ from the Illinois population in important ways. Future research could explore opportunities for gathering and analyzing data from a larger sample of districts to produce findings that better represent the population of students in Illinois.
Finally, additional research is needed explore changes in student learning over a longer period of time. This study focused on changes in student learning as of fall 2020, after approximately six months of disruption to school and life due to the COVID-19 pandemic. However, the pandemic has persisted well beyond this time, and many districts continued to serve students in remote or hybrid formats throughout the 2020/21 school year.
"The study is still relevant and has the same sense of urgency today as it did one year ago," said Brenda Dixon, Ed.D., research and evaluation officer at the Illinois State Board of Education. "As long as the pandemic exists and continues to have an impact on students and on their learning, this study will still be relevant and important to understand. This study allows districts to know more about where to allocate their dollars. According to this study’s findings, there were larger impacts on student learning in math. We even learned what grades were most affected, which gives districts information to target their efforts with strategies such as afterschool tutoring and other interventions they’re trying to put in place to accelerate learning going forward."
For more information about the study findings, read the full report and related infographic. Watch this short video to hear from the report authors and district leaders about the study findings and implications for Illinois states and districts.
To learn more about how REL Midwest is supporting the work of districts and states during the pandemic, see the following related resources:
Author(s)
Maggi Ibis
Connect with REL Midwest