Empirical Benchmarks for Interpreting Effect Size and Design Parameters for Planning Multilevel Randomized Trials on Social and Behavioral Outcomes

Social and behavioral measures are commonly used in education and social science research as primary and secondary outcomes of interest, and they are closely associated with the student academic achievement. To examine intervention effects on social and behavioral outcomes in education and prevention science, multilevel randomized trials that include cluster randomized trials (CRTs) and multisite randomized trials (MRTs) are now widely used. In order to design CRTs and MRTs with sufficient statistical power to detect a meaningful effect size of an intervention with precision, researchers need to make reasonable assumptions about the design parameters to estimate the required sample sizes. In addition, it is helpful to use empirical benchmarks for interpreting effect size in prevention science. But to date, there is very limited information about the empirical benchmarks for researchers and policy makers to interpret the magnitude of the prevention effects and design parameters to plan CRTs and MRTs on social and behavioral outcomes.

The purpose of the current study is threefold: (1) to provide empirical benchmarks regarding (a) normative expectations for change, (b) policy-relevant performance gaps, and (c) effect size results from similar studies for researchers and policy makers to interpret the magnitude of the intervention effects on social and behavioral outcomes, (2) to provide reference values of the design parameters (effect sizes, ICCs, R2, and their variability) on social and behavioral outcomes for researchers to conduct power analysis of CRTs and MRTs, and (3) to incorporate these reference values into PowerUp! software for power analysis.