Kristin Porter

This project extended and validated methods estimating power for different definitions of power and for several multiplicity adjustments commonly used in confirmatory analyses so that researchers who are planning randomized controlled trials (RCTs) with any number of common designs and who will adjust for multiple tests across multiple outcomes will be able to estimate power, minimum detectable effect sizes (MDESs), and sample sizes.

Multiplicity adjustments to p-values protect against spurious statistically significant findings when there are multiple statistical tests (e.g., due to multiple outcomes, subgroups or time points). An important consequence of these adjustments is a change in statistical power. It is typically argued that multiplicity adjustments result in a loss of power, which can be substantial. Current practice for ensuring that impact evaluations in education have adequate statistical power does not tak...