|Title:||Evaluation of Ninth Grade Academies in Broward County Public Schools|
|Principal Investigator:||Black, Alison||Awardee:||MDRC|
|Program:||Evaluation of State and Local Education Programs and Policies [Program Details]|
|Award Period:||5 years||Award Amount:||$4,827,957|
Purpose: The purpose of the project is to assess the impact of Ninth Grade Academies (NGAs) on student engagement and performance in ninth grade and throughout high school. NGAs in Broward County and Florida are self-contained units located in a designated area of the school, with their own dedicated teaching faculty, guidance staff, and social services to create an intact community for the transition year of ninth grade. They have interdisciplinary teacher teams that share the same students and have common planning times. The project will also study how NGAs are implemented in Broward County Public Schools (BCPS) and in other schools in Florida, what factors influence high fidelity of implementation, the effectiveness of enhancements, and how implementation and effectiveness vary within BCPS and the State of Florida.
Project Activities: The proposed evaluation will assess the impact of NGAs in BCPS on a range of short-term and longer term student engagement and performance outcomes. It will also assess the fidelity with which the NGA components are being implemented and examine the way ninth-graders are treated in high school that do not use NGAs. The evaluation will also measure the costs associated with implementing NGAs, including new resources that are brought into the high schools and existing resources that are redeployed. A second phase of the evaluation will estimate the impact of NGAs in a set of schools outside BCPS but within the State of Florida. Using early results from the evaluation, the research team will work with BCPS to develop a range of enhancements to the existing NGAs in the district and submit them to a rigorous randomized controlled trial (RCT).
Products: The evaluation will produce annual reports for each phase of the projects. The core BCPS study will provide early estimates of the effect of NGAs on ninth-grade performance and promotion and implementation findings, follow-up on several cohorts of ninth-graders, and extended follow-up through upper grade outcomes. The statewide study will provide similar products beginning in year two. The enhancement study will provide a planning year report on the development of enhancements, preliminary impact and implementation findings, as well as follow-up findings and a cost study.
Setting: The core study will include 25 schools from BCPS, which serves a unique urban/suburban mix of more than 258,000 students; 25 comparison schools not implementing NGAs for the core study will come from similar districts in Florida. A second component includes a statewide sample of 50 schools from Florida, half implementing NGAs and half not, which will also be a mix of urban and suburban settings. The third component is an evaluation of enhancements to NGAs already operating in BCPS.
Population: The evaluation focuses on outcomes for cohorts of ninth-grade students from ninth grade through graduation. In BCPS, the student population is quite diverse with 38% black, 30% white, and 26% Hispanic. Over 40% of students are economically disadvantaged. For the core study, the study body in matched comparison schools will be similar to that of BCPS. The statewide sample will focus on schools with significant populations of students from low-income families.
Intervention: NGAs are self-contained units located in a designated area of the school. The academy is staffed with its own dedicated teaching faculty, guidance staff, and social services to create an intact community for this transition year. These NGAs are organized around interdisciplinary teacher teams that have students and planning times in common. NGAs support personalized relationships among and between students and teachers. The teaming of teachers and students support more consistent classroom composition and student peer groups while decreasing anonymity and increasing students' sense of community. Students have a consistent group of teachers who are accountable for their success, and teachers have a chance to coordinate their coursework to better meet the needs of their students.
Research Design and Methods: The impact analyses will employ both comparative interrupted time series (for the core and statewide components) and school-level random assignment designs (for the enhancements) to estimate the impact of NGAs on student outcomes. The accompanying implementation analyses will use a mixed methods approach to produce a comprehensive understanding of the social and academic supports available to ninth-graders across sample schools.
Control Condition: Control schools are to continue with their ongoing practices (a "business-as-usual" control). To the extent possible comparable information is collected in the control and treatment conditions (e.g., staff composition, curriculum/course offerings and enrollments, evidence of teaching teams and common planning time for teachers, support services, sense of personalization).
Key Measures: For the impact study, primary measures will be related to student engagement, attitudes and achievement. Key outcomes include grade-level promotion, credits earned, attendance rates, graduation rates, and test scores. For the implementation analyses, measures include structural and procedural elements of NGAs, as well as measures to capture conditions and practices that enhance or limit implementation and effectiveness of NGAs. School-level surveys and site visits will be the primary sources for implementation data.
Data Analytic Strategy: Data for the impact analyses is nested—individual students are nested within schools—and observations within the same group are not statistically independent of one another. Multilevel models will be used to estimate the effect of the NGAs and examine and influence of implementation and potential mediators of effects. The study will attend to variations in school conditions and the degree to which schools are implementing core NGA elements and/or additional practices.
Nongovernment report, issue brief, or practice guide
Legters, N., and Parise, L. (2016). A Community of Practice Approach to Support for Ninth-Graders in Urban High Schools. Policy Brief. MDRC.
Legters, N., Parise, L., and Rappaport, S. (2013). Implementing Ninth Grade Academies in Broward County, Florida. MDRC.
Somers, M. A., and Garcia, I. with Quint, J. (2016). Helping Students Make the Transition into High School: The Effect of Ninth Grade Academies on Students' Academic and Behavioral Outcomes. MDRC.