|Title:||Early College High Schools at Scale: Probing Impacts and Generalizability with a Quasi-Experiment Benchmarked Against an RCT|
|Principal Investigator:||Lauen, Douglas||Awardee:||University of North Carolina, Chapel Hill|
|Program:||Improving Education Systems [Program Details]|
|Award Period:||3 (7/1/2015-6/30/2018)||Award Amount:||$799,886|
|Type:||Efficacy and Replication||Award Number:||R305A150477|
Co-Principal Investigator: Fatih Unlu, Abt Associates
Purpose: This project will provide evidence about the effectiveness of Early College High Schools (ECHS) in North Carolina (NC). ECHS are small schools of choice primarily located on campuses of two- or four-year colleges or universities. Students who attend ECHS have the opportunity to earn, at no financial cost to them, two years of transferable college credit or an associate's degree while simultaneously satisfying high school graduation requirements. This study will leverage existing data to provide information about whether, how, and for whom NC ECHS are effective. Prior IES-funded research (Follow-Up to the Study of the Efficacy of North Carolina's Early College High School Model) examined impacts from19 of the 70 ECHS sites in NC. These 19 sites used a lottery for admissions to ECHS. Lotteries approximate random assignment designs and allow for rigorous assessment of efficacy. This study builds on the prior study by examining additional mediators of effects and longer term outcomes. In addition, this study will use the data and results of the prior study to build complex statistical models that can replicate the RCT and be applied, retrospectively, to the other 51 NC ECHS to estimate their effects. A secondary aim of the project is to develop and refine this new statistical technique.
Project Activities: In order to complete this retrospective efficacy study, the researchers will obtain administrative data from all NC high schools, including ECHS, and match the data with longitudinal data from other local, state and federal databases to examine secondary, postsecondary, and other life outcomes (incarceration and voting) for the students. The sample will be limited to students who applied to ECHS. Those who were offered admission are considered to be the treatment group, and those who were not serve as the control group. Outcomes for students in the two groups will be compared.
Products: The products of this project will be evidence of the efficacy of ECHS for high school students in NC, a newly developed statistical technique, and peer-reviewed publications.
Setting: Data will be obtained from NC high schools (including ECHS), community colleges, universities, and public databases which include incarceration and voter registration information in North Carolina. In addition, data on students' postsecondary completion (within and outside NC) will be obtained from the National Student Clearinghouse.
Sample: The size of analytic samples will vary by outcome. By year two of the study, more than 40,000 students will have primary outcome measures such as achievement in high school, high school graduation, and postsecondary enrollment during and after high school. The 19 sites that participated in the lottery and the 51 sites that did not participate in the lottery have similar characteristics, though the lottery sites have slightly more African Americans and slightly more students eligible for free or reduced price lunch, while the non-RCT ECHS have slightly more Hispanic students and slightly higher 8th grade algebra passing rate. The traditional high schools in NC have similar characteristics to the ECHS, with the exception that there are more students with disabilities and fewer gifted students in traditional schools. All demographic differences between school types will be taken into account in the statistical models.
Intervention: The ECHS model includes a core set of design principles: college readiness, powerful teaching and learning, personalization, redefined professionalism, leadership, and purposeful design. ECHS are limited to 400 students. ECHS is operated through the auspices of North Carolina New Schools (NCNS).
Research Design and Methods: Using statewide administrative data and student-level information about lotteries conducted by ECHS, the best performing (lowest bias) propensity score models will be chosen from a within-study replication analysis that tests models using data from the 19 lottery sites as a benchmark. A variety of covariate sets and propensity score techniques will be tested separately for each outcome of interest. A mediational analysis will examine the mechanisms of the ECHS model design principles.
Comparison Condition: The comparison students are those attending a traditional public high school in the state of North Carolina.
Key Measures: Intermediate and ultimate outcomes include high school course taking, NC standardized end of grade test scores, and graduation; postsecondary applications, enrollment, course taking, GPA, and completion; incarceration; and voter registration. Pre-treatment covariates include levels and trends of middle school NC end of grade standardized test scores, absences and suspensions; parental education, poverty status, school enrollment changes, school-based educational classifications, and socio-demographic characteristics.
Data Analytic Strategy: The data analytic strategy is propensity score matching. The researchers will explore heterogeneity across sites, across student subgroups, and mechanisms that may mediate impacts.
Related IES Projects: Follow-Up to the Study of the Efficacy of North Carolina's Early College High School Model (R305A110085)