Purpose: The purpose of the project is to assess the impact of Ninth Grade Academies (NGAs) on student engagement and performance in ninth grade and throughout high school. NGAs in Broward County and Florida are self-contained units located in a designated area of the school, with their own dedicated teaching faculty, guidance staff, and social services to create an intact community for the transition year of ninth grade. They have interdisciplinary teacher teams that share the same students and have common planning times. The project will also study how NGAs are implemented in Broward County Public Schools (BCPS) and in other schools in Florida, what factors influence high fidelity of implementation, the effectiveness of enhancements, and how implementation and effectiveness vary within BCPS and the State of Florida.

Project Activities: The proposed evaluation will assess the impact of NGAs in BCPS on a range of short-term and longer term student engagement and performance outcomes. It will also assess the fidelity with which the NGA components are being implemented and examine the way ninth-graders are treated in high school that do not use NGAs. The evaluation will also measure the costs associated with implementing NGAs, including new resources that are brought into the high schools and existing resources that are redeployed. A second phase of the evaluation will estimate the impact of NGAs in a set of schools outside BCPS but within the State of Florida. Using early results from the evaluation, the research team will work with BCPS to develop a range of enhancements to the existing NGAs in the district and submit them to a rigorous randomized controlled trial (RCT).

Products: The evaluation will produce annual reports for each phase of the projects. The core BCPS study will provide early estimates of the effect of NGAs on ninth-grade performance and promotion and implementation findings, follow-up on several cohorts of ninth-graders, and extended follow-up through upper grade outcomes. The statewide study will provide similar products beginning in year two. The enhancement study will provide a planning year report on the development of enhancements, preliminary impact and implementation findings, as well as follow-up findings and a cost study.

Setting: The core study will include 25 schools from BCPS, which serves a unique urban/suburban mix of more than 258,000 students; 25 comparison schools not implementing NGAs for the core study will come from similar districts in Florida. A second component includes a statewide sample of 50 schools from Florida, half implementing NGAs and half not, which will also be a mix of urban and suburban settings. The third component is an evaluation of enhancements to NGAs already operating in BCPS.

Population: The evaluation focuses on outcomes for cohorts of ninth-grade students from ninth grade through graduation. In BCPS, the student population is quite diverse with 38% black, 30% white, and 26% Hispanic. Over 40% of students are economically disadvantaged. For the core study, the study body in matched comparison schools will be similar to that of BCPS. The statewide sample will focus on schools with significant populations of students from low-income families.

Intervention: NGAs are self-contained units located in a designated area of the school. The academy is staffed with its own dedicated teaching faculty, guidance staff, and social services to create an intact community for this transition year. These NGAs are organized around interdisciplinary teacher teams that have students and planning times in common. NGAs support personalized relationships among and between students and teachers. The teaming of teachers and students support more consistent classroom composition and student peer groups while decreasing anonymity and increasing students' sense of community. Students have a consistent group of teachers who are accountable for their success, and teachers have a chance to coordinate their coursework to better meet the needs of their students.

Research Design and Methods: The impact analyses will employ both comparative interrupted time series (for the core and statewide components) and school-level random assignment designs (for the enhancements) to estimate the impact of NGAs on student outcomes. The accompanying implementation analyses will use a mixed methods approach to produce a comprehensive understanding of the social and academic supports available to ninth-graders across sample schools.

• Core Broward County Study: This component of the evaluation focuses on high schools in BCPS that have implemented Ninth Grade Academies and uses similar high schools in Florida school districts outside of Broward County as the comparison group in a comparative interrupted time series (CITS) design. The study will also examine trends in "untreated academic outcomes" to assess the validity of making causal inferences from the CITS design;
• Statewide Extension: This will include high schools from across the State of Florida that are implementing NGAs, as well as high schools in the state that are not implementing NGAs as the comparison group in a second comparative interrupted time series design. The study will also examine trends in "untreated outcomes."
• Enhancement Study: In partnership, MDRC and BCPS will work together to use early findings from the core study to identify enhancements for NGAs that have the most promise of increasing the impact on student outcomes. The enhancement study will use a school-level random assignment design within BCPS to assess the effectiveness of NGAs that have been strengthened with additional components or supports. Half of the BCPS schools will be randomly assigned to implement enhancements and half will continue "business as usual" in their NGAs.
• Implementation Study: Information from quantitative surveys of teachers and students will be used to gauge NGA implementation in treatment schools and to investigate the practices and strategies that support ninth graders in comparison schools. On-site observations and interviews will be used to verify and augment implementation survey data, identify elements of the intervention not anticipated by the survey, and gain a deeper understanding of historical, contextual and qualitative aspects of implementation that may limit or enhance the impact of the school's NGA on student engagement and achievement.

Control Condition: Control schools are to continue with their ongoing practices (a "business-as-usual" control). To the extent possible comparable information is collected in the control and treatment conditions (e.g., staff composition, curriculum/course offerings and enrollments, evidence of teaching teams and common planning time for teachers, support services, sense of personalization).

Key Measures: For the impact study, primary measures will be related to student engagement, attitudes and achievement. Key outcomes include grade-level promotion, credits earned, attendance rates, graduation rates, and test scores. For the implementation analyses, measures include structural and procedural elements of NGAs, as well as measures to capture conditions and practices that enhance or limit implementation and effectiveness of NGAs. School-level surveys and site visits will be the primary sources for implementation data.

Data Analytic Strategy: Data for the impact analyses is nested—individual students are nested within schools—and observations within the same group are not statistically independent of one another. Multilevel models will be used to estimate the effect of the NGAs and examine and influence of implementation and potential mediators of effects. The study will attend to variations in school conditions and the degree to which schools are implementing core NGA elements and/or additional practices.

Products and Publications

Nongovernment report, issue brief, or practice guide

Legters, N., and Parise, L. (2016). A Community of Practice Approach to Support for Ninth-Graders in Urban High Schools. Policy Brief. MDRC.

Legters, N., Parise, L., and Rappaport, S. (2013). Implementing Ninth Grade Academies in Broward County, Florida. MDRC.

Somers, M. A., and Garcia, I. with Quint, J. (2016). Helping Students Make the Transition into High School: The Effect of Ninth Grade Academies on Students' Academic and Behavioral Outcomes. MDRC.