Skip Navigation
Impacts of a Violence Prevention Program for Middle Schools: Findings After 3 Years of Implementation

NCEE 2011-4017
May 2011

Analytic Approach for Estimating Program Impacts

The study team evaluated program impacts using multiple regression models that predicted each outcome's measure (e.g., violence, victimization) as a function of condition (intervention vs. control) and relevant covariates (e.g., demographic characteristics, school characteristics) using a mixed-effects regression model based on multilevel equations. Primary outcomes include self-reported counts of violent behavior and victimization occurring in the past 30 days.

The full student sample and gender subgroup analyses used a matched nested cross-sectional model (matched analysis). Under this model, students are nested in schools; schools are nested in pairs and in experimental condition; and pairs are crossed with experimental condition (i.e., each pair is represented at each level of condition). The covariate models for students in the full sample predicted the average response at follow-up, adjusting for the following covariates: baseline school mean of the response, school size, and individual demographic variables (gender, race/ethnicity, and number of parents in the household). For the gender subgroup analyses, the adjusted models included a gender-by-condition interaction effect.

The statistical models employed to assess program outcomes among high-risk youth are different from those employed to assess program outcomes on the general population of students. For the high-risk youth, the interest is in whether or not the RiPP and Best Behavior intervention led to individual change across time. To address this question, nested cohort models using difference-in-difference estimation were developed to assess changes on self-reported measures of aggression and victimization among high-risk youth in intervention schools relative to changes among high-risk youth in control schools, for both subpopulations of high-risk, nonperpetrator students and high-risk, perpetrator students. These models use data collected on the same sample of students at each measurement occasion. The repeated measures models for the high-risk subsamples contained the student's treatment condition (intervention vs. control), data collection wave, wave-by-condition interaction effect, gender, race/ethnicity, number of parents in household, and school size. Estimated program impacts reflect the net difference of the within-group change from pretest to third follow-up for intervention versus controls.

To examine teacher outcomes, we employed multivariate models where teachers are nested within schools and schools are nested within matched pairs randomized to experimental condition. Hierarchical linear models account for the correlation of teachers within schools and for schools within matched pairs assigned to condition. The models predicted the average response at follow-up, adjusting for school size.