Developing Time-Indexed Effect Size Metrics for K–12 Reading and Math Educational Evaluation

Co-Principal Investigator: Finn, Jeremy

Conventional effect size metrics such as Cohen's d are standardized group mean differences based on the distributions of student outcome variables at one particular age or grade level. They do not take into account the time dimension (i.e., the time needed to learn at that age/grade level). This study is based on the premise that time-indexed effect size metrics can estimate how long it would take for an "untreated" control group to reach the treatment group outcome in terms familiar to educators—months of schooling. These "months of schooling" effect-size metrics will differ from conventional grade equivalent (GE) metrics as strength-of-effect measures, which suffer from several limitations. For instance, GEs are drawn from test publishers' norms derived from cross-sectional data of different cohort groups at a single year to estimate growth curves. Moreover, the assumption under GE that the study sample would grow at the same rate as the national norms could be erroneous. The new measures adjust the growth trajectory based on national longitudinal data using vertical scales of achievement along with information regarding the demographic profiles of the study sample and settings.

Prior evidence from selected experimental research (Project STAR) and quasi-experimental research (Prospects Title I) will be reevaluated using this growth curve analysis framework, and the time-indexed effect size measures will be compared to those traditional effect size measures that have been computed previously. This research contributes to enhancing our capacity to understand or provide a context for interpreting the size of an effect, a step toward bridging the gap between educational research and practice.