|Title:||Alignment, Timing and Support: The Effect of Early Intervention on Student College Readiness|
|Principal Investigator:||Xu, Zeyu||Awardee:||American Institutes for Research (AIR)|
|Program:||Improving Education Systems [Program Details]|
|Award Period:||5 years (7/1/2016-6/30/2021)||Award Amount:||$1,839,252|
|Goal:||Efficacy and Replication||Award Number:||R305A160188|
Co-Principal Investigator: Dan Goldhaber
Purpose: This study will examine the impact of the Kentucky (KY) Targeted Interventions Program (TIP) on students' college readiness outcomes, as well as the conditions under which TIP affects students. In KY, students deemed not on track to be college ready are required by law to have intervention strategies included in their individual learning plans. The TIP consists of three main components: emphasis on early identification (in Grade 8) of students who are not on track to be college ready when graduating high school, alignment of benchmark tests and transitional interventions with college-ready proficiencies, and the provision of support rather than sanctions to students in need. This project will test the efficacy of the TIP on students' college readiness outcomes and provide information about the implementation of TIP across KY districts and schools.
Project Activities: Researchers will conduct (1) an implementation study to establish the types of support that targeted students receive; (2) a causal study to evaluate the efficacy of the TIP both for the state and for large districts; and (3) several exploratory studies that investigate how program effect estimates vary by student support type, school characteristics, and student subgroups. In addition, the researchers will conduct focus groups with various educators to strengthen the understanding of program implementation and the interpretation of study findings. Finally, the researchers will conduct a cost study of the KY TIP.
Products: Researchers will produce evidence of the efficacy of, and a cost analysis for, the Kentucky TIP. The research team will publish the findings in a peer reviewed journal and produce documentation for accessing the data and creating the analytical data file and code to replicate the analyses and findings.
Setting: The study will take place in the state of Kentucky.
Sample: The sample will consist of the universe of students who attend public schools in the state, tracking them from K–12 into postsecondary education using the Kentucky Longitudinal Data System. Researchers will use individual-level administrative data from KY beginning in academic year 2010–11 and continuing to the present. The researchers estimate that they will need 3,600 students around the high school test score cutoff to ensure adequate power for the analysis.
Intervention: The intervention is the Kentucky Targeted Interventions Program (TIP). Students who fail to meet assessment benchmarks on English, math, and/or reading in grades 8, 10, and 11 are required to take transitional courses before they exit high school. These transitional courses are tailored toward individual students' performance deficiency areas and aligned with the KY Core Academic Standards. Each intervention cycle consists of a diagnostic pretest, the development of instructional targets and their associated formative assessments, direct delivery of instruction, and a posttest. The courses can be delivered as full courses within a school's regular schedule (either integrated into an existing course or as a stand-alone course) or outside of the regular schedule (before/after school, as part of an Extended School Services program, or in a self-paced course). The ultimate goal of the TIP is to prepare every student to succeed in college without needing remedial education.
Research Design and Methods: The implementation study will establish the types of support that targeted students receive, focusing on heterogeneity of program features across districts and schools. Researchers will use the KY TIP database to collect information on the targeted student interventions, including intervention type, intensity, delivery mode and setting, staffing, materials, and outcomes. Focus groups will provide practitioners' perceptions on the quality of transitional interventions and the challenges they face while implementing those interventions. The causal study will evaluate the efficacy of the TIP using a regression discontinuity design that estimates the causal impact of failing college-ready benchmarks (i.e., not meeting test score thresholds) on a variety of high school and college outcomes. Exploratory studies will investigate how program effect estimates vary by student support type, school characteristics, and student subgroups.
Control Condition: The control group consists of students who met the assessment benchmarks and therefore did not take transitional courses.
Key Measures: The researchers will focus on three proximal high school outcomes: high school dropout rates, likelihood of taking advanced coursework, and ACT test scores, as well as three postsecondary outcomes: the likelihood of enrolling in a postsecondary institution, the likelihood of persisting to the second year in college, and the likelihood of taking remedial courses during the freshman year.
Data Analytic Strategy: The project uses a regression discontinuity design to estimate the impact of failing college readiness benchmarks using 3 approaches: centering, frontier, and response surface. In addition, researchers will estimate the effects for different subgroups (e.g., race/ethnicity, FRL eligibility) and districts, and explore possible moderators (type of intervention, type of school setting, and number of tests failed). The researchers will use the Resource Cost Model for the cost analysis.