Multilevel Modeling of Single-subject Experimental Data: Handling Data and Design Complexities

The purpose of the study is to develop and communicate guidance to applied SSED researchers in setting up studies, conducting the analyses, and interpreting and reporting the results. Single-subject experimental designs (SSEDs) are being used to tackle critical research problems where group comparison designs are not feasible, due to logistical and resource constraints or to the nature of the study population. Whereas group comparison designs focus on the average treatment effect at one point of time, single-subject experimental designs (SSEDs) allow researchers to investigate at the individual level the size and evolution of intervention effects. In addition, studying a low incidence or highly fragmented population lends itself to SSEDs. Despite the increased interest in the use of SSEDs and ongoing attention to the methodological implications of this type of design, including a previous IES grant to this research team (Multilevel Synthesis of Single-Case Experimental Data: Further Developments and Empirical Validation), the use of SSEDs is still hampered by many remaining questions concerning the statistical analysis and meta-analysis of data from these designs.

In this project, the research team will conduct various Monte Carlo simulations to investigate the effects of and the estimation of parameters given different types and extents of complexity when using multilevel models to analyze single-case data. The types of complexity will include, among others, the estimation of variance components at multiple levels, outcome measures that yield ordinal or count data, and nonlinear functional forms. Researchers will then identify the best approaches through simulation and use these approaches to analyze data from previously published applied studies. The research team plans to produce peer-reviewed conference presentations and journal articles, software for conducting the estimation procedures investigated, and guidance for single-case researchers about analytical approaches, given various design complexities.