Meta-Analysis Training Institute

Systematic reviews and meta-analyses can provide insight about what educational interventions work, for whom, and under what conditions. Large-scale meta-analyses can examine how intervention effects might vary across studies, providing insights about what we do or do not know about how to improve outcomes for all students. Thus, increasing the quality of systematic reviews and meta-analysis in education research enhances our understanding of important educational issues and highlights areas where more research is needed. 

Participants: The training team held three summer training institutes in August 2018, July 2019, and July 2022. (The third training was delayed due to disruptions caused by COVID-19.) The team recruited participants from higher education institutions and research firms, with a particular emphasis placed on recruiting women and minority group members. They received a total of 361 applicants across the 3 sessions and accepted 107 participants. The team reviewed the applications and chose participants based on prior meta-analysis experience, statistics background that included multilevel modeling, and IES grant proposal ideas. Recruitment efforts achieved a level of ethnic diversity: 11 percent Black/African American, 28 percent Asian/Asian American, 8 percent Hispanic/Latino (any race), 58 percent White, and 2 percent Other. The participants were 35 percent male and 72 percent female. The majority (95 percent) of participants were faculty and research professionals, with 12 percent being doctoral students or recent graduates.

Program Activities: The workshop took place over the course of 1 week. Each day, participants engaged in lecture sessions, hands-on practice with the methods, and a group project that culminated in a presentation at the end of the workshop. The training team provided each group with a dataset from a published meta-analysis that they used to apply the techniques discussed. The topics covered focus on methods for systematic review and meta-analysis that are rarely covered in introductory workshops and texts such as the use of screening software, multivariate and multivariate meta-analysis models, and selection models for publication bias. Participants learned to use the free software R to conduct meta-analysis. At the end of the week, participants chose to attend sessions on more specialized topics such as evidence and gap maps, meta-analysis for structural equation models and single-case design meta-analysis.

• All workshop materials are housed in the Canvas Learning Management System at Georgia State University (see Additional Online Resources and Information section below). Modules included those on the basics of statistical programming in R and RStudio, conceptual background material on systematic reviews and meta-analyses, and all power point slide decks and R code used in the workshop. Participants have continued access to these materials after the end of the workshop.