Project Activities
The researchers will conduct the meta-analyses in multiple phases. First, they will conduct a multi-phase systematic search and screening to retrieve relevant studies of the summer effect and of summer reading interventions in grades K–5 for the two meta-analyses. Then, they will code the studies in each final corpus to generate separate datasets for Aim 1 and Aim 2. In the final phase, they will calculate effect sizes for each relevant outcome, apply various meta-analysis models to synthesize the effect sizes, and conduct moderator analyses.
Structured Abstract
Setting
For Aim 1, the research team will analyze studies conducted in public or private school settings. For Aim 2, the research team will limit their studies to those that provided summer reading interventions in schools, homes, community organizations, clinics, or research centers. The studies to be included in the meta-analysis will be those published in English.
Sample
The studies used in the meta-analyses will include only primary studies that analyzed data from K-5 students without intellectual disability, traumatic brain injury, or profound hearing or vision loss. For Aim 1, students must have been attending public or private schools during the academic year. For Aim 2, students must be participating in summer programs offered in English (the language of interest for reading outcomes) by community organizations, public libraries, public or private schools, clinics, and universities or research centers.
The factors to be studied for both meta-analyses include study designs, student characteristics, study setting, and when and how students' pre- and post-summer reading abilities are measured. Additional factors for the summer intervention meta-analysis include content and type of reading program, the dosage and duration of the program, who delivers the program, and any comparison condition.
Research design and methods
To obtain large and unbiased meta-analytic samples, the research team will use various complementary approaches for obtaining relevant group-design studies and single- group pre-post studies of summer reading, and no restrictions will be placed on publication status other than the studies must have been published in or before December 2024. The research team will create screening and coding manuals. Then, for each of the two datasets, the research team will screen the studies identified in the search against a set of inclusionary criteria and apply standardized coding procedures to extract data with reliability. For both Aims 1 and 2, the research team will describe the characteristics of studies in the corpus and will assess the risk of bias for each study. During the analysis phase of the study, the research team will calculate the effect sizes and describe their distribution. The approach will allow for estimating the summer effect for Aim 1 and the effectiveness of summer reading interventions for Aim 2, as well as determining which factors moderate those effects.
Control condition
The research team will not include control conditions for studies of the "summer effect," which only considers what happens to performance when students are not participating in a summer reading program (Aim 1). For Aim 2, the research team will (a) compare students to themselves before and after summer reading intervention (single-group studies) and (b) compare learning outcomes of students who participated in summer reading interventions with those who did not (group-design studies).
Key measures
The key measures include reading outcomes such as oral language, phonological awareness, phonics/decoding, fluency, vocabulary, and comprehension etc. The research team also will estimate separate effects by study design type (e.g., between-group, within-group, single-group pre-post).
Data analytic strategy
To determine the effectiveness of interventions (Aim 1 and 2), the researchers will apply a correlated and hierarchical meta-analysis model with robust variance estimation to account for multiple effect sizes estimated within studies. They will describe the overall distribution of effect sizes and use meta-regression to explore how study and participant characteristics are associated with effect size heterogeneity. The meta-regression approach will use multiple moderators simultaneously if sufficient numbers of studies are eligible. The research team will incorporate moderator analyses to explore heterogeneity by methods, units, treatment, observations, and settings. Additionally, the research team will conduct different sensitivity analyses to evaluate the robustness of the results.
People and institutions involved
IES program contact(s)
Project contributors
Products and publications
This project will result in preliminary evidence of the factors relevant to the effectiveness of summer reading programs. The project also will result in a final dataset to be shared, peer-reviewed publications and presentations, and additional dissemination products that reach education stakeholders such as practitioners and policymakers.
Publications:
ERIC Citations: Find available citations in ERIC for this award here.
Supplemental information
Co-Principal Investigators: Pigott, Therese; Zhang, Huibin; Gibbs, Anna S.
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
