Meta-Analysis of Single Subject Designs:
An Overview

William R. Shadish
University of California, Merced

Browse the full presentation
 View and download as a Powerpoint presentation (60 KB)

The movement generally known as "Evidence Based Practice" has traditionally placed great reliance on the meta-analysis of randomized experiments. This emphasis is well-deserved given the statistical and empirical rationale behind randomization. However, especially for cases in which random assignment is not feasible or ethical, researchers have sometimes also included nonrandomized experiments in their assessment of the effects of an intervention. Typically, such quasi-experiments are structured similarly to a randomized experiment except for the lack of randomized experiments.

However, a class of nonrandomized experiments exists that has rarely been used in such syntheses: the single subject design, sometimes called N = 1 designs or ABAB designs. Such designs are typically structured as a time series of observations on a single person. Cause and effect relationships are demonstrated by the repeated introduction, removal and reintroduction of treatment to show that the effect is present when the treatment is present, but not otherwise. Such designs are viewed by many methodologists has a strong basis for causal inference.

However, to use these designs as a basis for evidence based practice decisions, the designs must be amenable to meta-analytic synthesis. To date, there has been little agreement on appropriate meta-analytic methodology for single subject designs. In this talk, we review our work over the last 9 months on this topic. During that time, we have reviewed the literature and created an annotated bibliography for the following:

1. Past meta-analyses of single subject designs (N = 22): http://faculty.ucmerced.edu/wshadish/Past%20MA%20of%20SSDs.htm
2. Scholarly works on how to do meta-analyses of single subject designs (N = 33): http://faculty.ucmerced.edu/wshadish/Methods%20for%20MA%20of%20SSD.htm
3. Sample data sets for use in our work to illustrate and test statistical methods for the meta-analysis of single subject designs in the areas of:
• Reading improvement: http://faculty.ucmerced.edu/wshadish/Reading%20Improvement.htm
• Learning disabilities: http://faculty.ucmerced.edu/wshadish/Learning%20Disabilities.htm

In addition, we have begun to address the two major kinds of meta-analytic issues that occur in the literature. The simpler problem addresses how to synthesize multiple single subject designs that all use the same design within one study but have more than one participant. The harder problem is the synthesis of multiple single subject designs that come from different studies, use different designs, different dependent variables, different though related treatments, and different time scales.

The other two participants in this panel will talk about these two problems in more detail.

Top