Design Comparable Incidence Rate Ratio (IRR) Effect Sizes for Count Data in Single-Case Experimental Designs

To investigate the performance of the proposed BC-IRR method, a comprehensive Monte Carlo simulation study will be conducted. Count data based on multiple baseline design will be generated. Different series length, number of cases, baseline trend, treatment effect, between-case variance, and degree of overdispersion will be considered in the data generation. A BC-IRR will be estimated for each generated dataset using the generalized linear mixed model (GLMM) and the generalized estimating equation (GEE) approach, and the statistical inferences will be conducted using a Wald test. The performance of the estimators and their statistical inferences will be evaluated based on bias, mean squared error (MSE), coverage rate of the 95% confidence interval, and empirical Type I error rate. To develop the IRR benchmarks, the project team will adopt the "across-studies comparison" approach in which a distribution of empirical effect sizes is established through meta-analysis. The work will focus on two common outcome domains in SCEDs: reading and behavior. For the behavioral outcome domain, the analysis will further distinguish between behaviors with positive and negative valence. In total, three empirical distributions of IRR effect sizes will be established and each will consist of 200 effect size estimates. For each study, the project team will estimate a within-case IRR and a between-case IRR effect size for the immediate treatment effect adjusting for baseline trend. After obtaining the distributions of the effect size estimates, ranges of effect size values will be obtained according to the quantiles (e.g., bottom 25%, 25–50%, 50–75%, and top 25%) to serve as the empirical benchmarks.