Co-Principal Investigator: Ianetta, Melissa

Purpose: In the early 2010s, large numbers of students were entering postsecondary programs with a high school diploma but were underprepared for college coursework. According to the NCES (2013), in 2011-12, 33% of first-year college students took at least one remedial course in reading, writing, or math (40% in public 2-year and 30% in public 4-year institutions). To help address the needs of students and faculty in development writing courses, the researchers developed a curriculum called Supporting Strategic Writers (SSW). SSW uses self-regulated strategy instruction with a strong emphasis on self-evaluation using genre-specific criteria. The initial pilot study of the intervention's promise found positive impacts on student writing and motivation, leading the researchers to propose and conduct a larger, initial efficacy study (R305A160242).

Project Activities: The researchers used design research with multiple rounds of development, implementation, and revision to design the full curriculum. They leveraged the insights of students, faculty, and an advisory board to develop and improve SSW. Once they had a fully developed curriculum, they conducted a quasi-experimental pilot study and found a positive effect of the intervention.

Products: The project developed (1) a complete curriculum for a full semester developmental writing course with options of instructional units for instructors; (2) a plan for professional development for instructors; (3) a validated measure of motivation including self-efficacy, beliefs, goal orientation, and affect; and (4) a measure of treatment fidelity with good interrater reliability.

Key Outcomes: The main findings of the project's quasi-experimental pilot study are as follows (MacArthur, Philippakos, & Ianetta, 2015):

• There was a statistically significant effect of treatment with the treatment group producing higher quality persuasive essays than the control group (ES = 1.22).
• Students showed improvement on overall quality of writing and length but not grammar.
• Significant positive effects were also found for self-efficacy for tasks/processes and mastery motivation.

Structured Abstract

Setting: The design research took place at the Delaware Technical and Community College (DTCC). The quasi-experimental study took place at two colleges in the mid-Atlantic region.

Sample: In the final round of design research, 8 instructors participated with a total of 13 classes at 2 levels of developmental writing. Complete pretest and posttest data were available for 115 students (51 percent male; 51 percent minority, 20 percent non-native English speakers). The culminating quasi-experimental study involved 2 colleges, 13 instructors with 19 classes, and 276 students (46 percent male, 48 percent minority, 10 percent non-native English speakers).

Intervention: Supporting Strategic Writers (SSW) is an instructional program for college-level developmental writing based on self-regulated strategy instruction. In this intervention, students learn strategies for planning, drafting, and revising compositions with an emphasis on using knowledge of genre to guide planning and evaluation, including self-evaluation. Students also learn strategies for self-regulation, including goal setting, task management, progress monitoring, and reflection. Each instructional unit focuses on a genre (e.g., personal narrative, argument). Teachers follow a consistent instructional sequence that includes a discussion of the genre, analysis of good and weak sample essays, think-aloud modeling, collaborative and guided practice, and peer review and editing. SSW curriculum materials include instructional units on personal narrative, procedural, comparison, cause/effect, and argument. Instructors are able to select the appropriate units and lessons to meet their college's program requirements and their own instructional goals; the strategies can be generalized to other genres.

Research Design and Methods: The project devoted 2 years to design research with 3 cycles of iterative curriculum development, implementation, and revision. During each of these cycles, the research team worked collaboratively with teachers and students and drew upon multiple methods of data collection and analysis, both quantitative and qualitative, to analyze how teachers and students understood the instruction, how aspects of the instruction enhanced and interfered with learning, and how it affected student learning. After completing the development phases, the researchers ran a quasi-experimental study to test the initial promise of the intervention to improve student outcomes.

Control Condition: The comparison condition in the quasi-experiment was business-as-usual, i.e., instructors continued with instruction as in past semesters.

Key Measures: In the quasi-experimental study, fidelity of implementation was rated by observers using a checklist of specific activities in the curriculum and ratings of the quality of key instructional components (e.g., think-aloud modeling). Treatment teachers were interviewed about their perceptions of the curriculum.  Measures of student outcomes included written argumentative essays scored for quality on a 7-point scale and a motivation questionnaire that assessed goal orientation, beliefs about writing, self-efficacy, and affect. Both measures were administered at pretest and posttest. A sample of students was interviewed after instruction. All measures were developed as part of the project. The motivation measure is available for use by others.

Data Analytic Strategy: During the development, usability, and feasibility phases of the study, the researchers used mixed qualitative and quantitative methods. Frequent observation and informal and structured interviews with teachers and students were used to analyze responses to the instructional methods and student understanding of the strategies. A fidelity of implementation measure was developed based on the observations. The motivation scale was developed and validated. Writing quality and motivation outcomes were assessed using pretest-posttest comparisons with students nested in classrooms. For the pilot quasi-experimental study, they determined the effects on posttest writing quality using HLM analysis with pretest quality and standardized writing test scores as covariates.

Related IES Projects: Supporting Strategic Writers: Effects of an Innovative Developmental Writing Program on Writing and Reading Outcomes (R305A160242)

Products and Publications

ERIC Citations: Find available citations in ERIC for this award here.

WWC Review:

Macarthur, C.A., and Philippakos, Z.A. (2013). Self-Regulated Strategy Instruction in Developmental Writing: A Design Research Project. Community College Review, 41(2): 176–195. [WWC Review]

Project Website: http://www.supportingstrategicwriters.org/

Select Publications:

Book chapter

MacArthur, C.A., and Philippakos, Z.A. (2012). Strategy Instruction With College Basic Writers: A Design Study. In C. Gelati, B. Arfe, and L. Mason (Eds.), Issues in Writing Research (pp. 87–106). Padova: CLEUP.

Journal articles

Macarthur, C.A., and Philippakos, Z.A. (2013). Self-Regulated Strategy Instruction in Developmental Writing: A Design Research Project. Community College Review, 41(2): 176–195.

MacArthur, C.A., Philippakos, Z.A., and Graham, S. (2016). A Multi-Component Measure of Writing Motivation With Basic College Writers. Learning Disability Quarterly, 39(1), 31–43.

MacArthur, C. A., Philippakos, Z. A., and Ianetta, M. (2015). Self-Regulated Strategy Instruction in College Developmental Writing. Journal of Educational Psychology, 107(3), 855–867.

Traga Philippakos, Z. A., MacArthur, C. A., and Munsell, S. (2018). College Student Writers' Use and Modification of Planning and Evaluation Strategies After a Semester of Instruction. Journal of Adolescent & Adult Literacy, 62(3), 301–310.