|Title:||AIR Early College Follow-Up Efficacy Study|
|Principal Investigator:||Song, Mengli||Awardee:||American Institutes for Research (AIR)|
|Program:||Postsecondary and Adult Education [Program Details]|
|Award Period:||2 1/4 years (9/1/2016 – 12/31/2018)||Award Amount:||$630,139|
|Type:||Efficacy and Replication||Award Number:||R305A160140|
Purpose: In this project, researchers assessed whether early college high schools (ECHS), as implemented under routine conditions in five states, produced significant gains in postsecondary outcomes for students who attended them, compared to students that attended other district high schools. This is a follow-up to a previous American Institutes for Research (AIR) efficacy study that followed students for a minimum of two and a maximum of four years after high school, depending on the year in which they entered high school. The previous study found that students offered enrollment in an ECHS were more likely to graduate from high school, more likely to enroll in college, and more likely to earn a postsecondary degree than comparable students not offered ECHS enrollment. This follow-up study captured a more complete picture of postsecondary attainment by following students for a minimum of six years after expected high school graduation. Due to randomization of students to the early colleges, this study produced strong causal evidence on the impact of the ECHS model on college enrollment and degree attainment. Findings from this study may be of interest to policymakers in districts across the country, especially those currently operating, expanding, or considering the ECHS model.
Project Activities: In the first project stage, researchers collected college enrollment and degree completion data for all students in the study through the end of the 2016–17 academic year and merged these data into the existing database from the previous efficacy study. In the evaluation stage of the project, researchers assessed the overall impact of the ECHS model for students who were offered enrollment in an ECHS (the intent-to-treat impact) as well as the impact of the ECHS model on students who actually attended an early college (the complier average treatment impact). In addition, the project team examined the role of students' high school experiences (e.g., college-going culture and college credit accrual during high school) as potential mediators of the ECHS impact on students' postsecondary outcomes as well as the cost-effectiveness of the ECHS model.
Pre-Registration site: This study was not pre-registered.
Publicly available data: Researchers interested in accessing project data may request data from the project director, Dr. Kristina Zeiser (firstname.lastname@example.org), who will provide a data-sharing agreement. Because project data include data obtained from state administrative data systems, researchers must first obtain permission from the Ohio Department of Education and the North Carolina Education Research Data Center before the research team can share project data.
WWC Review: The WWC review for the final report for this project can be accessed here.
Setting: This project included 10 early colleges distributed across five states, including five schools in urban areas, two in midsized cities, and three in small towns.
Sample: The sample comprised 10 early colleges that met AIR's inclusion criteria for its previous ECHS efficacy study, which was fielded during three school years (2005–06, 2006–07, and 2007–08). During at least one of these years, a school had to enroll students in each of grades 9—12, offer a high school diploma, use a lottery to admit ninth grade students, and maintain records of its lotteries. As of fall 2007, 10 of the 154 early colleges operating nationwide met these criteria and were included in the initial efficacy study. This follow-up study included all 10 schools in the previous efficacy study, with 2,458 students in total, of whom 1,044 students received enrollment offers from the 10 early colleges and 1,414 students did not. Approximately half of these students are non-White and were eligible for free- or reduced-price lunch during high school.
Intervention: Early colleges are small high schools designed to increase the opportunity for students—particularly students traditionally underrepresented in higher education—to earn a postsecondary credential. Key components of the early college model include an emphasis on college access for all students, structured exposure to college courses while in high school with an opportunity to earn up to two years of college credit and an associate degree, and personalized supports for students' success in high school and college coursework. By creating a high school environment that promotes college-going and provides support (e.g., tutoring, and enhanced advising) to students enrolled in rigorous coursework, early colleges are designed to prepare students to complete college-level courses and college degrees. Because districts and schools implemented the model under routine conditions without an over-arching developer or agency to guide them, specific features of the model differed somewhat across schools. For example, some early colleges offered college credits free of charge while others required students to pay a reduced fee for them.
Research Design and Methods: By analyzing four more years of postsecondary outcome data, this follow-up study extended the previous AIR efficacy study which was based on a natural experiment made possible because districts relied on lotteries to assign students to early colleges. Key research questions addressed in this follow-up study include:
To address the question about the main impact of early colleges, the project team estimated intent-to-treat effects of being offered enrollment in an ECHS. Given that not all study participants complied with the lottery-based treatment assignment, the project team supplemented the intent-to-treat analyses with complier average treatment effect analyses, which estimated the impact of attending an ECHS. In addition, the project team conducted differential impact analyses and mediation analyses to address research questions 2 and 3.
Control Condition: Control group students were not offered the opportunity to enroll in the early colleges included in the sample for this study, but their high school choices were not constrained in any additional ways. Most control group students attended traditional comprehensive high schools in their districts.
Key Measures: Using enrollment and degree completion data from the National Student Clearinghouse, researchers measured whether each student ever enrolled in college and ever attained any type of degree within 6 years after expected high school graduation as key outcome measures. They also examined the types of institutions in which students enrolled, and the types of degrees that students received. For differential impact analyses, researchers examined each student's gender, race/ethnicity, low-income status, and 8th-grade achievement score as potential moderators using administrative data provided by the districts and states in which the sample schools are located. For mediation analyses, researchers measured students' high school experiences as potential mediators using data from a student survey administered during the previous efficacy study.
Data Analytic Strategy: To address the first research question for this study, the researchers employed multilevel logistic regression models to estimate the intent-to-treat (ITT) effects of early colleges on college enrollment and degree attainment outcomes. The ITT analyses were supplemented with complier average treatment effect analyses using an instrumental variable approach. To address the second research question, the researchers conducted differential impact analyses by adding to the ITT model an interaction term that combines the treatment status indicator with the specific student characteristic under consideration. To address the third research question, the researchers conducted mediation analyses that decomposed the total early college impact on a given outcome into a direct effect and an indirect effect as mediated through students' high school experiences.
Publications and Products
Journal article, monograph, or newsletter
Atchison, D., Mohammed, S., Zeiser, K. L., Knight, D., & Levin, J. (2020). The cost and benefits of early college high schools.Education Finance and Policy. https://direct.mit.edu/edfp/article/doi/10.1162/edfp_a_00310/97119/The-Costs-and-Benefits-of-Early-College-High
Song, M., Zeiser, K., Brodziak de los Reyes, I., & Atkinson, D. (2021). Early College, Continued Success: Longer-term impact of Early College High schools. Journal of Research on Educational Effectiveness, 14(2), 116–142. https://doi.org/10.1080/19345747.2020.1862374
Zeiser, K. & Song, M. (2020). The impact of Early Colleges on students' postsecondary education trajectories: Can Early Colleges overcome the (supposedly) diversionary role of community colleges? Journal of Research on Higher Education. https://link.springer.com/article/10.1007/s11162-020-09616-6
Atchison, D., Zeiser, K. L., Mohammed, S., Levin, J., & Knight, D. (2019). The cost and benefits of early college high schools. American Institutes for Research . Song, M., & Zeiser, K. L. (2019). Early college, continued success: Longer-term impact of early college high schools. American Institutes for Research.
In addition to publications in peer-reviewed academic journals, the researchers have also disseminated findings from this study to policymakers and educators through AIR press releases, a user-friendly policy brief, a two-minute video, infographics, as well as social media communications. All key products from this study, including a final technical report, have been posted on the project website.