Bayesian Analysis of Academic Outcomes from Single-Case Experimental Designs

The team will develop and evaluate the performance of a Bayesian framework to estimate two types of SCED ESs: a multilevel between subjects metric and a generalized least squares metric developed. Both effect sizes are theoretically comparable to effect sizes used in between-group research but have demonstrated questionable technical properties when estimated within a frequentist framework in the presence of complex data patterns. They will use a combination of extant analysis and Monte Carlo simulation to develop and evaluate the Bayesian estimator. During the first stage of the project they will conduct a systematic literature review to identify and extract data from published and unpublished SCED studies that evaluated academic interventions. All extracted data will be made publicly available. They will then conduct descriptive and inferential analyses for each academic skill area to identify common conditions observed in SCED studies (e.g., number of participants, typical treatment effects, etc.) to inform subsequent simulations. During the second stage of the project researchers will conduct simulations to generate hypothetical SCED data and will use that data to compare the performance of SCED ESs estimated using frequentist approaches, Bayesian analysis using uniformed priors, and Bayesian analysis using informed priors. During the third stage the team will develop the Bayesian effect size calculator website and pilot its use with applied education researchers. Based upon quantitative and qualitative feedback we collect from those researchers, we will refine the website prior to its formal launch.