|Title:||AIR Early College Follow-Up Efficacy Study|
|Principal Investigator:||Song, Mengli||Awardee:||American Institutes for Research (AIR)|
|Program:||Postsecondary and Adult Education [Program Details]|
|Award Period:||2 1/4 years (9/1/2016-12/31/20||Award Amount:||$630,139|
|Goal:||Efficacy and Replication||Award Number:||R305A160140|
Purpose: In this project, researchers will assess whether Early College High Schools (ECHS), as implemented under routine conditions by districts across the United States, produce significant gains in college attainment for students who attend them, compared to students that attend other district high schools. This is a follow-up to a previous American Institutes for Research (AIR) efficacy study that followed students for a minimum of two and a maximum of four years after high school, depending on the year in which they entered high school. The previous study found that students offered enrollment in an ECHS were more likely to graduate from high school, more likely to enroll in college, and more likely to earn a postsecondary degree than comparable students. This study captures a much more complete picture of postsecondary attainment than the previous efficacy study by following students for a minimum of six years after scheduled high school graduation. Due to randomization of students to the Early Colleges, this study will produce causal evidence of the impact of the ECHS model on college attainment. Findings from this study will be of interest to policymakers in districts across the country, especially those currently operating, expanding, or considering the ECHS model.
Project Activities: In the first project stage, researchers will collect college enrollment and degree completion data for all students in the study through the end of the 2016-17 academic year, and merge these data into the existing database from the previous efficacy study. In the evaluation stage of the project, researchers will assess the overall policy impact of the ECHS model for students in the sample (the intent-to-treat impact) as well as the impact of the ECHS model on students who actually attended an Early College (the complier average treatment impact).
Products: Researchers will produce evidence of the efficacy of the ECHS model as implemented under routine conditions. The researchers will produce a user-friendly research brief as well as social media communications to disseminate research findings to policymakers and educators. They will also post a final technical report on the web, and produce a peer-reviewed publication of their findings.
Setting: This project includes nine Early Colleges distributed across four states, including four schools in urban areas, two in midsized cities, and three in small towns.
Sample: The sample comprises nine Early Colleges that met AIR's inclusion criteria for its previous ECHS efficacy study, which was fielded during three school years (2005–06, 2006-07, and 2007–08). During at least one of these years, a school had to enroll students in each of grades 9-12, offer a high school diploma, use a lottery to admit ninth grade students, and maintain records of its lotteries. As of fall 2007, ten of the 154 Early Colleges operating nationwide met these criteria and were included in the efficacy study. This follow-up study includes nine of those ten schools, with one school excluded because the state in which it is located has refused to release student-level data to researchers. Thus the sample for this study includes 2,260 students who applied for ninth-grade admission into one of the nine Early Colleges, of whom 920 students received enrollment offers and 1,340 students were routed to other high schools. Approximately one-half of these students are non-White and were eligible for free- or reduced-price lunch during high school.
Intervention: The research team will evaluate the generic ECHS model as it was implemented under routine conditions by a variety of districts dispersed across the United States. Key components of the model include an emphasis on college access for all students (including those traditionally underrepresented in higher education), structured exposure to college courses while in high school with an opportunity to earn one to two years of college credit, and personalized supports for students' success in high school and college coursework. Researchers expect that by creating a high school environment that promotes college going and provides support (e.g. tutoring and enhanced advising) to students enrolled in rigorous coursework, Early Colleges prepare students to complete college-level courses and college degrees. Because districts and schools implemented the model under routine conditions without an over-arching developer or agency to guide them, specific features of the model differed somewhat across schools. For example, some Early Colleges offered college credits free of charge while others required students to pay for them.
Research Design and Methods: This follow-up study extends the previous AIR efficacy study which was based on a natural experiment made possible because districts relied on lotteries to assign students to Early Colleges. Researchers assessed the integrity of random assignment produced by the lotteries, and concluded that only two percent of students denied access to an ECHS by lottery ended up attending an ECHS, and 80 percent of students assigned to an ECHS enrolled in it. This pattern of deviations from lottery assignments will tend to downwardly bias estimates of the intervention's impact. Thus, the achieved randomization design will facilitate estimation of intent-to-treat effects of being offered enrollment in an ECHS. To estimate the actual impacts of attending an ECHS given some non-compliance with lottery assignments, researchers will use an instrumental variable strategy to compute a complier average treatment effect.
Control Condition: Control group students were not offered the opportunity to enroll in the Early Colleges included in the sample for this study, but their high school choices were not constrained in any additional ways. Most control group students attended traditional comprehensive high schools in their districts that did not offer college courses.
Key Measures: Using enrollment and degree completion data from the National Student Clearinghouse, researchers are measuring whether each student ever enrolled in college and ever attained any type of degree within 6 years after high school graduation. They are also measuring the types of institutions in which students enrolled, and the types of degrees that they received. For analysis of moderators, researchers are accounting for gender, race/ethnicity, low-income status, and 8th-grade achievement scores using transcript data provided by the districts in which the sample schools are located. They are also measuring the characteristics of ECHS programs and students' experiences with them using data from surveys administered to principals, teachers, and students during the previous efficacy study.
Data Analytic Strategy: The researchers will employ logistic regression models to compute intervention impacts on the full set of college attainment outcomes. To conduct the moderation analysis, researchers will add an interaction term to the outcome model that combines the treatment status indicator with the specific student characteristic under consideration. To examine whether intervention impacts are mediated by ECHS implementation, the researchers will employ a modeling strategy similar to structural equation modeling that allows for simultaneous estimation of treatment impacts mediated through multiple factors.