Description:

Co-Principal Investigator: Somers, Marie-Andrée

Purpose: The purpose of this project is to generate rigorous evidence of the effects of the Whole School, Whole Child (WSWC) dropout prevention model and the unique value of Tier 2 services within that model. The WSWC model is a two-tiered model, developed and widely implemented by City Year (YR) that aims to provide a safe and supportive learning environment to students as well as support to students at risk of dropping out based on their academic performance, behavior, and absenteeism. City Year AmeriCorps members (ACMs), young adults doing national service, provide universal services to all students and individualized services to students who are exhibiting early warning indicators for being off track to graduation.

Project Activities: The researchers will conduct two complementary studies to test the effects of City Year on students' social and emotional skills, school engagement, behavior and academic performance: a comparative interrupted time series (CITS) study to evaluate the effects of the WSWC model on all students in grades 6, 7, and 8; and a student-level randomized controlled trial (RCT) to assess the effects of Tier 2 services provided over two years starting in Grade 6 to students who exhibit indicators of being off track to graduation. In addition to evaluating program impacts, the studies will monitor and measure program implementation and examine moderators of these impacts. The studies will take place in five districts in four different states. The CITS will take place in 20 City Year schools and approximately 30 matched comparison schools, with all Grade 6, 7, and 8 students. The RCT will take place in the same 20 City Year schools, with approximately 2,000 students.

Products: The researchers will produce evidence about the efficacy of WSWC as a whole, and for the specific Tier 2 services offered within it. They will also produce peer-reviewed publications.

Structured Abstract

Setting: These two studies will take place in middle schools in five large urban school districts. All the sample middle schools serve predominantly high-poverty as well as racial and ethnic minority student populations. They also feed into high schools with high dropout rates.

Sample: The sample for the study of the WSWC model will include all students enrolled in grades 6–8 in 20 CY schools and 30 matched comparison schools. The study of Tier 2 services will include 2,400 grade 6 students in 20 CY schools.

Intervention: The WSWC model has two tiers of services: universal Tier 1preventive services for the whole school and targeted Tier 2academic, social/emotional learning, and attendance interventions for students exhibiting early warning signs of school dropout. CY AmeriCorps members (ACMs) provide both types of services. For Tier 1 work, ACMs serve as an additional resource for teachers in classrooms; supervise common school areas; and lead schoolwide initiatives to improve school culture. For Tier 2 services, ACMs meet with teachers and other school staff to plan interventions and monitor student progress throughout the school year. Interventions may include academic tutoring, attendance monitoring, and/or social, emotional, and behavioral interventions.

Research Design and Methods: The research team will conduct both studies in parallel with the same set of CY schools and districts over a five-year period. In the first project year, the team will collect the necessary administrative data and construct the analytic dataset for the WSWC evaluation analyses. They will also conduct site visits and strengthen their relationships with partner schools and districts. The researchers will begin random assignment and data collection for the student-level RCT of Tier 2 services in Year 2, with data collection continuing through to the end of the third project year. The team will spend Years 3 and 4 completing data cleaning, preparation, and analysis for both studies, including a cost analysis.

Control Condition: For the study of the WSWC model as a whole, each CY (treatment) school will be matched to the two most similarnon-CY schools in the district that are of the same type (charter or not, magnet or not); the same grade structure (Grades 6–8 versus Grades 6–12); and that are also Title I schools. For the study of Tier 2 services, eligible students will be randomly assigned to two years of Tier 2 services or to a control condition receiving Tier 1 services only.

Key Measures: Both studies will include measures in four outcome domains: (1) English language arts, (2) mathematics, (3) school engagement, and (4) social and emotional development. The study of the WSWC model will use existing data collected by school districts (e.g., standardized test scores, course grades, attendance, office disciplinary referrals, and the districts' school climate surveys). The study of Tier 2 services will include the same data and periodic academic assessments (i.e., progress monitoring in math and reading as well as records of homework and academic assignment completion rates). It will include teacher reports of student behavior as well as student reports of emotion regulation and school engagement. Researchers will measure fidelity of implementation using principal, teacher, and student surveys; electronic daily records of duration and frequency of services by student; and site visits. Cost analysis will provide information about the cost ingredients and total cost for each of the tiers of the WSWC model and examine resource allocation in intervention schools in the context of district policies.

Data Analytic Strategy: The first study will use a comparative interrupted time series (CITS) design to evaluate the effects of the WSWC model using four years of baseline data and up to nine years of follow-up data for CY schools and a group of matched comparison schools. For the second study of Tier 2 service effects, the researchers will produce intent-to-treat (ITT) estimates of the impact of assigning students to Tier 2 services at the end of the first and second year of the study by comparing all primary outcomes of students in the intervention and control groups. This will also be modeled hierarchically, using a random-effects approach where Level 1 models the variation in outcomes among students within schools and Level 2 represents the variation in the impact across schools.