Co-Principal Investigators: Farran, Dale; Lipsey, Mark; Preacher, Kristopher

Purpose: The purpose of this high school follow-up study of this longitudinal sample of students is to assess the nature and extent of COVID-19 related school disruptions and how those effects interact with prior school experiences (including pre-k attendance), various student characteristics, as well as the extent of the school disruptions experienced. Funded since 2009, this is a sample from the only longitudinal randomized control trial of a statewide prekindergarten program, evaluating the effectiveness of Tennessee's voluntary PreK program. During this stage of the study, students are attending high school in the midst of the Covid-19 pandemic that caused disruptions in schooling expected to have adverse effects on their academic progress, especially for these youth who are all from lower income families.

Project Activities: The research team will add follow-up data for the high school years to an existing longitudinal dataset. This will include annual data on graduation, achievement, GPA, attendance, grade level, and special education status from the state education data system available under the established partnership between the TN Department of Education and Vanderbilt University — the Tennessee Educational Research Alliance (TERA). Researchers will continue to collect additional annual data from student, parent, and teacher interviews for a previously consented subsample of students within the full sample. There will also be detailed data available from TERA, updated weekly, on the nature of the instructional accommodations made by the districts and schools to the COVID risk. Researchers will conduct analyses to estimate the effects of COVID-related school disruptions on students' educational trajectories and outcomes.

Products: This project will produce evidence on the effects of COVID-related school disruptions on the educational trajectories and outcomes of low-income high school students, including how those effects differ for key student subgroups (e.g., students of color, students with an IEP) and for different COVID-related instructional plans. That evidence will be provided to the Tennessee Department of Education through meetings with their staff and to the research community and education practitioners and policymakers through social media, presentations, and publications.

Structured Abstract

Setting: This project will take place using a sample of high schools and associated school districts across the state of Tennessee.

Sample: Two cohorts of students (n = 2990) have been tracked since pre-k as part of the initial and previous follow-up studies. Cohort 1 (n = 1744) started pre-k in the 2009–10 school year, and Cohort 2 (n = 1246) started pre-k in the 2010–11 school year. The 2,990 children in the full sample began participating in this study when their parents applied for pre-k in 2009 or 2010; they will be entering tenth and eleventh grade during the 2021–2022 school year. The sample includes students from 96 of the 137 school districts across Tennessee. This sample generally matches the racial and ethnic diversity of the state with 49% white, 27% Black, and 23% Hispanic, 24% non-native English speakers, and 49% male. All students qualified for free and reduced-price lunch. The subsample that participates in annual interviews includes 700 students and their parents and teachers, also with a representative mix of subgroups.

Intervention: To date, this study has focused on the long-term effects of the Tennessee state pre-k program based on the original randomized design. In the high school follow-up study, researchers will investigate the effects of COVID-related instructional disruptions. Tennessee does not have, and is unlikely to develop, statewide standards for instruction amidst COVID risk, resulting in variation across high schools that can be explored for differential effects. Those variations will likely be related to the populations that high schools serve and, within schools, there will be variation in student response that can be further explored with particular attention to effects on the most vulnerable students.

Research Design and Methods: The original evaluation study was an RCT comparing outcomes for applicants to the state pre-k programs randomly assigned to offers of admission or a waitlist that did not result in an admission offer. Researchers have examined the long-term effects of pre-k attendance through middle school. In this project, researchers will investigate the effects of COVID-related instructional disruptions on the trajectories students have established on various outcome measures during prior years. For the full sample, researchers will collect the data of interest each fall through mid-winter of the proposed study from the state education database via TERA for the previous school year, then organize and clean those data. In the spring of each year, they will run the analyses relevant to the data collected from that previous school year and also connect them to the longitudinal data for ongoing analysis of the time-series. For the subsample, researchers will collect parent interview data in the winter, student interview data in early spring, and teacher survey data throughout the spring each year. They will clean and analyze those data each summer and connect them to the longitudinal data.  Using interrupted growth curve analyses researchers will address questions about adverse effects on academic and behavioral trajectories, differential effects related to student characteristics and academic histories, and differential effects related to school characteristics and school and district responses to the pandemic.

Control Condition: The effects of COVID-related school disruptions will be assessed against students' prior academic trajectories using growth curve analysis models analogous to interrupted time-series. There is a randomized control group for pre-k participation.

Key Measures: Key measures from the state education data system include achievement test scores, GPA, on-grade progress, attendance, disciplinary offenses, and special education placements. For the subsample of students, parents, and teachers who have participated in interviews, researchers will collect measures of student problem behaviors, prosocial behaviors, peer relationships, and school/education attitudes, as well as reports of their experiences during the pandemic. Researchers will get data from TERA about school and district instructional accommodations for addressing COVID risks.

Data Analytic Strategy: Researchers will use multilevel growth curve models focused on trajectory changes between prior history and the COVID years. They will use moderator analyses within the growth curve model framework to examine differential effects related to student characteristics, school instructional options, and district and school characteristics.

Cost Analysis: For this cost analysis, researchers will focus on additional funds provided to districts in response to the COVID-19 crisis and their relation to the services provided. Researchers will track the response to COVID-19 in the cost analysis. They will obtain information from the state department of education (DOE) about any supplements the TN-DOE and the state legislature provided to districts or schools. Researchers will document differences in school- and district level COVID-related costs and services provided in response to the pandemic.

Related IES Projects: Evaluating the Effectiveness of Tennessee's Voluntary Pre-K  Program (R305E090009)