Project Activities
Follow-up data for the high school years were added to an ongoing longitudinal dataset. This included data on graduation, achievement, ACT scores, attendance, grade level, disciplinary offenses, and special education status from the state education data system available under the established partnership between the Tennessee Department of Education and Vanderbilt University – the Tennessee Educational Research Alliance (TERA). Additional annual data also continued to be collected from student, parent, and teacher interviews for a previously consented subsample of students within the full sample. Detailed data were available for the full sample on the nature of the instructional accommodations made by the districts to the COVID risk. The research team capitalized on the deep background they had on these students, including prior pre-k participation, with analyses estimating the effects of the COVID-related school disruptions on their educational trajectories and outcomes.
Structured Abstract
Setting
This project took place within a sample of high schools and associated school districts across the state of Tennessee.
Sample
The 2,990 children in the full sample began participating in this study when their guardians applied for pre-k in 2009 or 2010. The same children entered ninth and tenth grade during the 2020-2021 school year, with their high school experience beginning during the pandemic. The sample included students from 96 of the 137 school districts across Tennessee. This sample generally matched the racial and ethnic diversity of the state with 49% white, 27% Black, and 23% Hispanic, 24% non-native English speakers, and 49% male; all qualified for free and reduced price lunch. The subsample that participated in annual interviews included 625 students and their parents and teachers, also with a representative mix of subgroups.
This study has focused on the long-term effects of the Tennessee state pre-k program based on the original randomized design. Those effects were still of interest. However, given the exceptional ability of this study to investigate the effects of COVID-related instructional disruptions, the research team directed their attention to that endeavor. Those disruptions will thus constitute an intervention they studied as an interruption to the long time-series of academic outcomes available for the sample. Moreover, Tennessee did not have rigid statewide standards for instruction amidst COVID risk, resulting in variation across districts that could be explored for differential effects. Those variations were related to the populations that high schools served and there was variation in student response that was further explored with particular attention to effects on the most vulnerable students.
Research design and methods
The original evaluation study was an RCT comparing outcomes for applicants to the state pre-k programs randomly assigned to offers of admission or a waitlist that did not eventuate in an admission offer. The longitudinal background the research team had on this large sample allowed them to design an investigation of the effects of COVID-related instructional disruptions on the trajectories students established on various outcome measures during prior years. Using interrupted growth curve analyses the research team addressed questions about adverse effects on academic and behavioral trajectories, differential effects related to student characteristics and academic histories, and differential effects related to school and district characteristics and district responses to the pandemic.
Control condition
The effects of COVID-related school disruptions were assessed against students’ prior academic trajectories using growth curve analysis models analogous to interrupted time-series. There was also a randomized control for pre-k participation.
Key measures
Key measures from the state education data system included achievement test scores, ACT scores, graduation status, on-grade progress, attendance, disciplinary offenses, and special education placements. For the subsample of students, parents, and teachers who have participated in interviews, the research team collected measures of student problem behaviors, prosocial behaviors, peer relationships, and school/education attitudes, as well as reports of their experiences during the pandemic. The research team also collected data about district instructional accommodations for addressing COVID risks.
Data analytic strategy
Results were analyzed with multilevel growth curve models focused on trajectory changes between prior history and the COVID years. Moderator analyses within the growth curve model framework examined differential effects related to student characteristics, district instructional options, and district and school characteristics.
Cost analysis strategy
The cost analysis focused on additional funds that may have been provided to districts in response to the COVID-19 crisis and their relation to the services provided. However, obtaining reliable information on how districts actually spent funds proved difficult.
People and institutions involved
IES program contact(s)
Project contributors
Products and publications
This project produced evidence on the effects of COVID-related school disruptions on the educational trajectories and outcomes of low-income high school students, including how those effects differed for key student subgroups (e.g., students of color, students with an IEP) and for different COVID-related instructional plans. That evidence was provided to the Tennessee Department of Education through TERA and meetings with their staff and to the research community and education practitioners and policymakers through presentations and publications.
Project website:
Publications:
ERIC Citations: Find available citations in ERIC for this award here.
Select Publications:
Durkin, K., Lipsey, M. W., Farran, D. C., & Wiesen, S. E. (2022). Effects of a statewide pre-kindergarten program on children’s achievement and behavior through sixth grade. Developmental Psychology, 58(3), 470-484. https://doi.org/10.1037/dev0001301
Available data:
Data collected by the research team and accompanying codebooks will be made publicly available on the Open Science Framework website (https://osf.io/). Access to state datasets may be requested directly from the Tennessee Department of Education (https://www.tn.gov/content/tn/education/districts/federal-programs-and-oversight/data/data-downloads/request-data.html).
Related projects
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
