Co-Principal Investigators: Troia, Gary; Cho, Eunsoo

Purpose: The purpose of this project is to develop a tool, the Writing Architect, that will assist teachers in connecting evidence-based writing instruction with students' needs as identified in a digital classroom written composition assessment. The goal is to improve late elementary students' written composition in response to text (informational writing) and to use assessment results more effectively for instructional purposes.

Project Activities: Gathering feedback from teacher users, teaching experts, and writing experts, the research team will iteratively refine the Writing Architect assessment and develop the integrated instructional recommendations and supports that will improve writing performance and achievement. The research team will conduct a feasibility study to integrate professional learning supports, followed by a cost analysis, and a pilot study that compares the developed Writing Architect with business-as-usual writing assessment and differentiated writing instruction.

Products: The primary product for this project is an integrated tool that combines digital assessment of writing with differentiated instructional recommendations, and the support needed to implement differentiated writing instruction. Implementation support will include an online module for teachers and intervention teams. Additional products include practitioner articles, presentations, peer-reviewed journal articles, and an open dataset intended to reach practitioners, policymakers, and researchers.

Structured Abstract

Setting: The project will take place in language arts classrooms in elementary schools in Michigan.

Sample: Year 1 will include a focus group of 10 grades 4 and 5 teachers providing insight about the usability of writing assessment and instruction. Year 2 will include 10 teachers and their 250 students engaged in a feasibility study. The pilot study in year 3 will include 40 teachers and 800 students.

Intervention: The Writing Architect (WA) is an innovative web-based tool that integrates a previously developed diagnostic writing assessment, a report that connects theoretically relevant scores to research-based instructional recommendations and resources, a repository of those resources, and the professional learning supports involved in Practice-Based Professional Development, to implement instruction.

Research Design and Methods: In Year 1, the research team will develop the instructional components of the WA and adjust it and the assessment for usability based on descriptive feedback from teachers and experts. In Year 2, the research team will use a descriptive study with a pretest/posttest design with 10 teachers and their students to inform revision of the WA components and supports. In Year 3, the research team will conduct a pilot cluster randomized controlled trial to examine the potential for impact of the WA on teacher and student outcomes.

Control Condition: For the pilot study, the research team will randomly assign teachers to an active waitlist control. They will use their typical classroom writing assessment (e.g., rubrics) and any typical differentiated instruction practices.

Key Measures: The research team will determine usability from focus groups of teachers. Researchers will measure feasibility using teacher field testing, student measures, and pilot testing a randomized controlled trial. The primary teacher measures include: Teacher Writing Knowledge Test, Teacher Efficacy for Writing Scale, implementation checklists for fidelity of writing instruction, and estimations of dosage for writing instruction. Proximal student outcomes include curriculum-based measurement in written expression and rubric scores. Distal student outcomes include the Written Expression scale from the Oral and Written Language Scales II and the end-of-year state test of writing in Michigan.  The research team will also collect descriptive information based on teachers 'planned instructional decisions, teacher-recorded dosage of instruction, and observed instructional practices for both the Writing Architect condition and the active waitlist control condition.

Data Analytic Strategy: Researchers will use a descriptive approach to inform the development and refinement of the WA in Year 1. Mean differences and correlations in teacher and student outcomes will be explored in Year 2 to determine feasibility. In year 3, the research team will conduct regression analyses to examine the impact of the Writing Architect on teacher outcomes, student writing growth, and student writing achievement. Analyses will include multiple levels, pretest scores, and mediators as needed.

Cost Analysis: The research team will estimate costs for the WA and control condition using the "ingredients" method and an incremental cost-effectiveness analysis method.

Project Website: https://www.writingresearch.net/

Products and Publications

Valentine, K. A., Truckenmiller, A. J., Troia, G. A., & Aldridge, S. (2021). What is the nature of change in late elementary writing and are curriculum-based measures sensitive to that change? Assessing Writing, 50, 100567. https://doi.org/10.1016/j.asw.2021.100567