Project Activities
The research team will analyze school- and student-level outcomes for campuses that have implemented the program to date (in the 2020–21 school year), as well as campuses that will implement the program in the 2021–22 and 2022–23 school years. The researchers will use school-level and student-level propensity score matching (PSM) to address potential selection bias in estimating the impacts of ADSY on schoolwide and learner-specific outcomes. Additional analyses will estimate student achievement effects of the higher-touch ADSY Planning and Execution Program (ADSY PEP). Additionally, the team will build a Texas-specific version of RTI's Evaluation Engine (EE) platform, which will be used to conduct separate propensity-score matching analyses. The analyses will use extant data from Texas' State Longitudinal Data System, as well as program participation data to identify participating schools.
Structured Abstract
Setting
To date, the intervention has been implemented in campuses across Texas. Before the end of the grant period, ADSY may be implemented in all campus types.
Sample
This sample will include all students in campuses participating in ADSY from the 2020-21 school year through 2022-23. This includes students from grades PK-5.
The project will study the effects of the ADSY program, which provides half-day formula funding for school systems that want to add up to 30 instructional days to any of their elementary schools.
Research design and methods
The team will use data from the Texas Student Data System (TSDS) on the population of students at ADSY campuses. The project team will estimate causal effects of ADSY on student outcomes through propensity score matching techniques. These analyses will be conducted annually and will also examine how effects vary for different student subgroups, including by race/ethnicity, students with disabilities, economically disadvantaged students, and English Language Learners.
Control condition
The comparison group for the study will consist of students in schools and districts that did not participate in ADSY. Control students will be statistically matched to ADSY students to ensure comparable contexts.
Key measures
Student outcomes of interest focus on performance on State of Texas Assessments of Academic Readiness (STAAR) assessments in end-of-grade (EOG) exams. In addition, the team will examine attendance, promotion/retention, and persistence/dropout indicators. The team will also analyze outcomes for different student subgroups. School outcomes of interest will reflect aggregation of student outcomes.
Data analytic strategy
The partnership will apply a quasi-experimental approach centered around propensity score matching. The team will first conduct school-level matching procedures to evaluate overall effects of ADSY on school outcome rates of interest compared to non-ADSY schools. After matching schools, partners will also apply propensity score matching to students, nested within matched schools, to better focus on student outcomes, especially for important subgroups of students. This plan extends an existing easy to use web-based statistical package so that TEA staff can continually assess student outcomes over time and as new research questions emerge.
State decision making
The Using Data for Recovery grant gives TEA the opportunity to use its SLDS data (TSDS) to assess the extent to which TEA's programming drives improvement in student outcomes. TEA can use rigorous estimates of the impact of ADSY on student outcomes to drive programmatic and policy decisions around resource allocation, program support, and scale-up. TEA will also be able to examine characteristics of schools that see the largest gains in student outcomes, identify districts and schools that need more targeted support during implementation, and further refine the program framework.
People and institutions involved
IES program contact(s)
Project contributors
Products and publications
Products: This project will produce interim and summative reports on the effect of ADSY on student outcomes. The partners will share reports with agency leadership to inform decision-making on scale, and with other relevant stakeholders, including the general public. Additionally, the team will present at relevant conferences, and submit relevant products for publication in peer-reviewed journals.
Supplemental information
Co-Principal Investigators: Hansen, Ben; Hussain, Amreena
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
