|Title:||Supporting Strategic Writers: Effects of an Innovative Developmental Writing Program on Writing and Reading Outcomes|
|Principal Investigator:||MacArthur, Charles A.||Awardee:||University of Delaware|
|Program:||Postsecondary and Adult Education [Program Details]|
|Award Period:||5 years (7/1/2016 to 6/30/2021||Award Amount:||$3,245,858|
|Type:||Efficacy and Replication||Award Number:||R305A160242|
Co-Principal Investigator(s): May, Henry; Philippakos, Zoi
Purpose: Many community college students must take non-credit developmental courses in writing to improve their basic writing skills, but little research has focused on instructional interventions designed for developmental writing. The purpose of this project was to test the efficacy of such an intervention that was developed and evaluated in a previous IES-funded grant (R305A100614). A quasi-experimental study (MacArthur et al., 2015) in the prior grant found improved outcomes in writing quality and motivation, including self-efficacy. The current study aimed to further evaluate the intervention using a larger, more diverse sample and more rigorous research design. The overall goal of the research was to provide an evidence-based curriculum to build writing skills in college developmental writing courses and, thus, improve students' chances for successfully completing college.
Project Activities: In two randomized controlled trial experiments, the researchers tested a developmental writing intervention, Supporting Strategic Writers (SSW), in which students learn strategies for planning, drafting, and revising compositions as well as strategies for self-regulation. The second study also included strategies for critical reading of sources, which is a critical achievement for success in first-year composition and other college courses that involve substantial reading and writing.
Key Outcomes: The main findings of this project are as follows:
Setting: Each of the two studies involved 2 community colleges, including campuses in urban and rural areas.
Sample: Together the two studies included 42 faculty randomly assigned within college to treatment and business-as-usual (BAU) control and 394 students (60 percent female; 55 percent non-white [Black, Latino, Asian, Native American], 17 percent non-native English speakers).
Intervention: SSW is an instructional program for college-level developmental writing based on self-regulated strategy instruction integrated with practices common in college composition. Students learn strategies for planning, drafting, and revising compositions with an emphasis on using knowledge of genre to guide planning and evaluation, including self-evaluation. In addition, they learn metacognitive self-regulation strategies for goal setting, task management, progress monitoring, and reflection. Teachers followed a consistent instructional sequence that included a discussion of genre, think-aloud modeling, collaborative and guided practice, and peer review and teacher feedback. In study 1, students learned to write in multiple genres, concluding with argumentative writing. Students wrote from background knowledge without using sources. Study 2 expanded the curriculum to include writing using sources and focused solely on argumentative writing. Strategies were added to the curriculum for critical reading of sources and integration with their own ideas.
Research Design and Methods: Both studies were multi-site, cluster-randomized trials with random assignment of instructors within college to treatment and control groups. Using a mixed-methods approach with interviews and observations, the researchers also gathered information from students and instructors to help clarify the effects and track fidelity of implementation. Treatment instructors implemented SSW for a full semester, while control instructors continued with BAU. Treatment instructors received an instructors' guide and 3 days of professional development prior to the semester and coaching during the semester.
Control Condition: Teachers in control classes continued with business-as-usual instruction per their colleges' standard developmental writing programs.
Key Measures: In both studies, at pretest and posttest, students completed three assessments: (1) argumentative essays scored for overall quality, length, and grammar; (2) a motivation questionnaire tapping goal orientation, beliefs, self-efficacy, and affect; and (3) a standardized test of reading (i.e., Accuplacer). The posttest essay in study 2 asked students to use two given source articles. Essays were scored for overall quality and length. In addition, at posttest only, students wrote persuasive essays based on a retired prompt from the National Assessment of Educational Progress. Also, a sample of treatment students were interviewed at posttest. In study 2, students also wrote a summary of a newspaper editorial, which was scored for overall quality and for inclusion of key ideas without copying. Instructor measures in both studies included (1) observations capturing fidelity of treatment and instructional practices in both treatment and control classes; (2) pretest interviews to gather information on demographics, education, experience, and prior approaches to instruction; and (3) posttest interviews of treatment instructors for responses to the program. To investigate long-term effects, participating colleges provided transcript data for 2 years after the study about completion of first-year composition, retention, GPA and credit hours earned, and completion of a degree or certificate.
Data Analytic Strategy: The primary analyses of main effects on student outcomes used hierarchical linear modeling (HLM) with students nested within instructors, and with college as a fixed factor. The researchers also used HLM to investigate moderation of effects by college, initial writing achievement, ethnicity, and native-speaker status and multilevel structural equation modeling (MSEM) to explore mediated pathways.
Cost Analysis: The researchers completed a cost analysis with the primary purpose of helping college personnel, including both administrators and faculty, make decisions about whether to use the SSW curriculum in their developmental courses and guiding them in allocating sufficient resources for implementation. The analysis includes costs to the college, faculty, and students. Costs are estimated for implementation by a group of faculty from a college. The analysis will be submitted to ERIC and posted on the project website.
Related IES Projects: Development of a Curriculum to Teach Writing in Postsecondary Developmental English Composition Classes (R305A100614), Writing in Adult Secondary Education Classes (W-ASE) (R305N210030)
ERIC Citations: Find available citations in ERIC for this award here.
Publicly Available Data:
All quantitative data, analysis code, and measures from the most recent study (MacArthur et al., in press) are available at the Center for Open Science at https://osf.io/6juqy/. Student written work and qualitative data, including observation notes and interviews with instructors and students were not included because it is impossible to ensure anonymity for such data.
Data from the follow-up analysis are available at https://osf.io/wcvr6/.
Project Website: http://www.supportingstrategicwriters.org/
MacArthur, C. A. & Traga Philippakos, Z. A. (under contract). Writing instruction for success in college and in the workplace, New York: Teachers College Press.
MacArthur, C. A. (in press). Postsecondary developmental education in writing: Issues and research. In Lui, X., Hebert, H., & Alves, R. A. (Eds.), The hitchhiker's guide to writing research: A festshrift for Steve Graham. Dordrecht, The Netherlands: Springer Nature.
MacArthur, C. A. (2022). Integrated reading and writing instruction in college. In Graham, S. & Traga Philippakos, Z. A. (Eds.), Writing and reading connections: Bridging research and practice (pp. 311–330). New York: Guilford Press.
Philippakos, Z. A. & MacArthur, C. A. (2019). Writing strategy instruction for low-skilled postsecondary students. In Perin, D. (Ed.). Wiley Handbook of Adult Literacy (pp. 495–516). New York: Wiley.
MacArthur, C. A., Traga Philippakos, Z. A., May, H., Potter, A., Van Horne, S., & Compello, J. (in press). The challenges of writing from sources in college developmental courses: Self-regulated strategy instruction. Journal of Educational Psychology.
MacArthur, C. A., Traga Philippakos, Z. A., May, H., & Compello, J. (2022). Strategy instruction with self-regulation in college developmental writing courses: Results from a randomized experiment. Journal of Educational Psychology, 114(4), 815–832. doi.org/10.1037/edu0000705
MacArthur, C. A., Jennings, A., & Philippakos, Z. A. (2019). Which linguistic features predict quality of argumentative writing for college basic writers, and how do those features change with instruction?. Reading and Writing, 32(6), 1553–1574. Full text
Traga Philippakos, Z. A., MacArthur, C. A., & Munsell, S. (2018). College student writers' use and modification of planning and evaluation strategies after a semester of instruction. Journal of Adolescent and Adult Literacy, 62, 301–310. Full text
Philippakos, Z. A. T., Wang, C., & MacArthur, C. (2023).Writing motivation of college students in basic writing and first-year composition classes: Confirmatory factor analysis of scales on goals, self-efficacy, beliefs, and affect. Journal of Learning Disabilities, 56(1), 72–92.