Project Activities
The researchers will conduct a meta-analysis that leverages findings from relevant experimental and quasi-experimental studies examining the impacts of writing instruction for college and university students. This work includes a literature scan using multiple databases as they search for both published and unpublished studies (including dissertations, conference papers, etc.). Once identified, they will double-code all included studies, extract effect sizes, and analyze the studies for any dependence among effect sizes.
Structured Abstract
Setting
To be included in the meta-analysis, the studies must evaluate writing interventions set in 2- or 4-year college or university settings.
Sample
The researchers estimate including approximate 200 to 300 studies for their review. To be included, the study must estimate effects for college or university students and may estimate effects for underrepresented groups of students, including racial/ethnic minorities, international students, domestic or international dual-language learners, or students with disabilities.
Although each study may have a different writing intervention, the researchers will include only studies that evaluated the impact of writing interventions for college students such as educator-led interventions, peer tutoring, writing center support, etc. that aimed to improve general writing skill or course learning outcomes (writing to learn). They researchers will also explore heterogeneity in the effect sizes by coding for three factor sets: (1) writing and content learning outcomes, (2) participant characteristics, and (3) instructional contexts.
Research design and methods
The researchers will conduct a meta-analysis of studies that use experimental or quasi-experimental designs to test writing instruction with college students, including both published and unpublished studies, including dissertations, conference papers, and "file-drawer" studies. In phase 1, they will conduct a systematic search of the literature to obtain the studies through a digital search of online databases, a hand search of relevant publications, a backwards search (articles cited by an article in the included set), and a forward search (articles that cite an article in the included set). They will then double-screen studies for inclusion using an abstract screening tool and will also develop a codebook to code studies for their research design, sample population, setting, intervention, measure, and study quality characteristics. In phase 2, they will double-code all included studies and calculate inter-rater reliability for the overall coding, factor sets, and individual variables. In phase 3, they will then (a) extract and calculate effect sizes, (b) calculate the summary effect for all studies through a meta-analytic random effects model (using robust variance estimation to account for any dependence among effect sizes), (c) examine the heterogeneity of the summary effect, (d) conduct a meta-regression to examine the impact of moderators, (e) examine study quality, and (f) examine publication bias. Then they will prepare data and programming code to share in repositories, developing manuscripts, and other dissemination efforts.
Control condition
When scanning studies, the researchers will ensure that the prior study included a comparison group that does not include the writing intervention, such as no-treatment control conditions or alternate intervention conditions unrelated to writing (e.g., studying, rereading, etc.). They will not include studies that have only one-group or pre/post designs.
Key measures
To be included in the meta-analysis, the study under consideration for inclusion must measure writing outcomes or course learning outcomes. The researchers will l analyze these two sets of outcomes separately. Writing outcomes can include writing quality, production, genre elements, organization, planning, revision, persistence (time spent writing), etc. Course learning outcomes can include reading comprehension, content knowledge, argumentation, critical thinking, problem solving, etc. The researchers will also include and summarize any college success outcomes, such as GPA, course credits, graduation.
Data analytic strategy
The researchers will use robust-variance estimation with random effects to estimate the average weighted effect sizes and then estimate variability using I-squared and Tau-squared statistics and prediction intervals. They will then use meta-regression with robust variance estimation to examine potential moderators that might explain heterogeneity in the effect sizes, and they will examine the factor sets described above as potential moderators of the intervention effects.
People and institutions involved
IES program contact(s)
Project contributors
Products and publications
The researchers will provide information to inform frameworks for postsecondary writing as well as develop manuscripts, presentations, and other dissemination materials for both research and non-research audiences.
Publications:
ERIC Citations: Find available citations in ERIC for this award here.
Supplemental information
Co-Principal Investigators: Graham, Steve; Rodgers, Derek B.
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
