Previous Award Number: R305A150201
Previous Awardee: University of Pittsburgh

Co-Principal Investigators: Correnti, Richard; Matsumura, Lindsay Claire; McKeown, Margaret G.

Purpose: The purpose of this project was to develop and test the feasibility and promise of an instructional intervention, Triple Q, which was designed to support middle school students' argument writing skills. The intervention includes three units of instruction in which students read and discuss argument texts, examine the features and quality of the arguments within and across those texts, and draft and revise their own written arguments. The intervention uses Questioning the Author (QtA) queries (McKeown, Beck & Blake, 2009) to focus students' attention at three levels: 1) understanding the gist of an argument text; 2) analyzing the organizational and rhetorical features of written arguments; and 3) considering a variety of language choices that students may incorporate into their own writing to advance their arguments. These queries not only guide class discussions during intervention lessons, but also guide students through the writing process as they plan, map, draft, revise and give each other feedback on their argument essays. Triple Q showed promise for supporting students' peer-to-peer engagement during discussions to develop argumentation skills and had a positive and significant effect on argument writing skills.

Project Activities: The researchers carried out four iterative development studies in collaboration with English Language Arts teachers in three school districts. The iterative development studies resulted in three, 15-lesson instructional units. Each study was a user test of at least one Triple Q unit. Data collection to inform development and revision of units included interviews with teachers and a subsample of focal students in each classroom, analysis of lesson transcripts, student notebook entries, and students' argument drafts and final essays. After completing a full version of the curriculum, researchers pilot tested it using a cluster-randomized control trial with 27 English Language Arts teachers (12 intervention) in one large, urban school district in the southeast.

Key Outcomes: The main features of the intervention and findings of the project's pilot study are as follows:

• The fully developed intervention, with all teacher and student materials including pro and con model argument texts, quick writes, argument map templates, and peer review checklists. This publication is available for free download from the SERP Educator Resources site (www.serpinstitute.org/educator-resources)
• They also developed an innovative scoring protocol that captures coherence and elements of argument writing.

Structured Abstract

Setting: The iterative development studies took place in urban and greater metropolitan districts in Pennsylvania. The pilot study took place in an urban school district in Tennessee.

Sample: The iterative development studies involved eight teachers and approximately 80 students in 6th and 7th grade. These studies were carried out in three school districts representing diverse student backgrounds. The pilot test was conducted in 13 schools with a total of 27 teachers (12 intervention). A total of 416 sixth and seventh graders participated in the pilot study.

Intervention: Triple Q is designed to help middle school students improve their argumentative writing by using queries to focus students' attention on text gist, argument features, and language choice. The queries guide students' planning, drafting, and revising activities with their own writing. The intervention includes text sets for analysis, text discussion, cross-text discussion to examine elements of written argumentation, extended writing tasks, feedback from teachers and peers, and revision of writing. Triple Q contains three units, each focusing on a different policy-oriented question of interest to middle school students, and each lasting 15 days.

Research Design and Methods: The research team developed the intervention iteratively over the first two years of the project, guided by observations, student interviews, lesson transcripts, student writing samples, and teacher feedback. The pilot study, conducted in the third year, was a cluster-randomized control trial with school-level random assignment. All the teachers and students within a school were randomly assigned to receive either Triple Q or standard school instruction. Triple Q discussions were audio recorded, transcribed, and coded for discourse moves found to promote academically productive talk in argumentation. As assessment of argument writing skill was administered pre- and post-intervention.

Control Condition: In the pilot study, the control group received standard classroom instruction in which teachers taught a district developed curriculum that focused on argument writing.

Key Measures: Researchers used interviews with students and teachers, lesson transcripts, observations, and student writing samples during the iterative development process to revise the intervention. For the pilot study, researchers used one proximal, researcher-designed student outcome measure, namely the Response to Text Assessment (RTA), to assess argument writing quality at pre- and post-intervention. Also, for the pilot study, researchers qualitatively analyzed audio recorded class discussions and examined these descriptively to understand how the intervention influenced teacher and student's participation in discussions about argument texts. Finally, researchers collected standardized assessment results for participating students from immediately following the pilot study and from the previous year. This assessment was the TNReady and standard scores were collected for the English Language Arts test.

Data Analytic Strategy: To address questions of feasibility and usability during the development phase, researchers qualitatively examined teacher interviews and lesson transcripts including affixing thematic codes to transcripts and identifying patterns and themes in interviews. For the pilot study, researchers employed qualitative discourse analysis techniques to examine the quality of classroom discussions in Triple Q compared to the control group classrooms and to examine change over time in each of those conditions. Researchers used a differences-in-differences (DID) approach to examine the promise of Triple Q to impact the argument writing quality and performance on the standardized assessment of English Language Arts.

Products and Publications

Project website: All Triple Q intervention materials are freely available on the Strategic Education Research Partnership's website: https://www.serpinstitute.org/educator-resources

Books

McKeown, M.G., Crosson, A.C., Sartoris, M., Matsumura, L.C., & Correnti, R. (2020). Triple Q: Argument writing for middle school (Units 1–3). Pittsburgh, PA. University of Pittsburgh. Available here: https://www.serpinstitute.org/educator-resources.

Journal articles

Howe, E., & Correnti, R. (2020). Negotiating the political and pedagogical tensions of writing rubrics: Using conceptualization to work towards sociocultural writing instruction. English Education, 52, 335–360.