Co-Principal Investigators: Ciullo, Stephen; Graham, Steve

Purpose: For this project, the research team will conduct a research synthesis, including a comprehensive meta-analysis, to explore what writing interventions are effective on what writing outcomes, for whom, and under what conditionsin kindergarten through grade 5. Findings will provide guidance for developing new writing interventions and informing professional development for teachers. Furthermore, the project will offer empirical evidence to support research to practice efforts that will improve writing achievement for all students.

Project Activities: Phase I of this project includes search, screening, and selection. Researchers will conduct a systematic search of relevant writing studies that provide sufficient information to include in this synthesis. In Phase II, the research team will code studies, estimate study effect sizes, and complete data analysis to evaluate the included studies.

Products: The research team will produce peer-reviewed synthesis articles and disseminate findings to researchers, practitioners, policymakers, and other stakeholders.

Structured Abstract

Setting: For inclusion in this meta-analysis, the writing intervention must have been provided in typical school settings within the United States or other countries.

Sample: Eligible studies include writing interventions designed to improve writing outcomes of students in kindergarten through grade 5. The research team will examine the following interventions (as well as other interventions that emerge): (a) opportunities to write, (b) instruction on transcription skills, (c) instruction on sentence skills, (d) instruction on strategies and writing processes, (e) instruction on writing knowledge, (f) instruction to facilitate motivation, (g) instruction procedures to support students as they write, and (h) comprehensive writing programs. For inclusion, studies must include at least one measure of writing and use a randomized control trial or quasi-experimental design with pretest data reported. No restrictions are placed on date of the study or language of instruction. Studies must be reported in English.

Malleable Factors: Researchers will review and synthesize empirical evidence from writing interventions. This meta-analysis will systematically explore (a) what interventions are effective on what writing outcomes, (a) for whom (e.g., students with disabilities, second language learners), and (c) under what conditions(e.g., dosage, instructor) in kindergarten through grade 5.

Research Design and Methods: The proposed meta-analysis methods will include two project phases. Project Phase I—Search, Screening, and Selection will identify all published and unpublished studies that meet eligibility criteria using the following systematic search methods: electronic databases searches, reference harvesting, journal hand searches, examination of previously conducted reviews, and consultation with prevalent national and international writing researchers. The research team will begin by screening titles and abstracts and then review full-text articles to select studies for inclusion. Project staff will be trained to 90 percent reliability to ensure accuracy of search, screening, and selection procedures. Project Phase II—Data Extraction and Analysis involves rigorous coding of all eligible studies, extracting data to calculate standardized mean difference effect sizes (and associated variance), and planned exploratory data analyses.

Control Condition: Due to the study design, there is no control condition.

Key Measures: Learner outcomes in this meta-analysis are the standardized mean difference effect sizes between intervention and control conditions across five writing outcomes: (1) products, (2) process, (3) skills, (4) knowledge, and (5) motivation. The research team will also examine associated variance and explore potential moderators (learner populations and instructional conditions) of study effect sizes.

Data Analytic Strategy: The research team will calculate separate standardized mean difference effect sizes between writing interventions and control conditions for the five writing outcomes. Researchers will then use robust variance estimation to estimate separate mean effect sizes for each intervention type and writing outcome after controlling for study quality. In addition, the research team will conduct additional hypothesis tests and examine heterogeneity of effect size variability across writing interventions, outcomes, populations of students, and intervention conditions.