Skip Navigation
Funding Opportunities | Search Funded Research Grants and Contracts

IES Grant

Title: A Researcher-Practitioner Partnership Examining the Use of Automated Essay Evaluation Software for Improving Students Writing Achievement
Center: NCER Year: 2017
Principal Investigator: Wilson, Joshua Awardee: University of Delaware
Program: Researcher-Practitioner Partnerships in Education Research      [Program Details]
Award Period: 2 years (08/01/2017 – 07/31/2019) Award Amount: $399,999
Type: Researcher-Practitioner Partnership Award Number: R305H170046
Description:

Co-Principal Investigator: Beard, Gaysha

Program: Researcher-Practitioner Partnerships in Education Research

Purpose: This partnership project developed and strengthened the partnership between University of Delaware's School of Education and Red Clay School District, and explored the feasibility and promise of MI Write (formerly PEG Writing), an automated essay evaluation (AEE) software program. Red Clay, and other school districts, want to improve the writing achievement of students in elementary schools, Grades 3–5. However, teachers often do not assign enough writing or provide enough feedback because it is time consuming and writing is difficult to evaluate. AEE may provide a means to overcome these obstacles to student writing but it is an under-researched through widely used technology.

Partnership Activities: Partnership activities included monthly meetings of the university and district leads, quarterly Writing Partnership Leadership Team meetings, provision of professional development on MI Write to teachers, literacy coaches, and administrators, joint selection of study measures that were then embedded in the district's web-based learning management system, training of research assistants to conduct focus groups with teachers, joint development of publications and presentations, and collaborating on future funding proposals.

Key Outcomes:

  • Teachers and students found MI Write easy to use and an effective instructional support (Wilson, Ahrendt et al., 2021; Wilson, Huang et al., 2021). However, sustained training was needed to ensure teachers use the system consistently, effectively, and fully.
  • Teachers primarily used the main features (e.g., assigning prompts, reviewing student work) but not the supplemental features that might help intensify learning such as peer review, using teacher reports for goal setting, and differentiating instruction via assigning MI Write's interactive lessons (Wilson, Huang et al., 2021).
  • Teachers generally spoke positively about the benefits of MI Write on instruction (Wilson, Ahrendt et al., 2021), stating that they believed that students were, on the whole, more motivated to write and revise when using MI Write, produced higher quality compositions, and showed greater independence throughout stages of the writing process than in prior years.
  • These benefits were accompanied by new challenges. Some students found MI Write difficult to use independently and exhibited decreases in writing motivation. Teachers noted misalignments between their own feedback and that of MI Write (Wilson, Ahrendt et al., 2021) and struggled to find time within the daily schedule and the curriculum to focus on sustained writing projects.
  • Regarding students' writing performance, MI Write scores were excellent predictors of student performance on the state English language arts (ELA) test, even over and above prior state test ELA performance, suggesting that MI Write, and possibly other AEE systems, may have utility for benchmarking students' writing achievement (Wilson, Huang et al., 2021; Wilson & Rodrigues, 2020).

Structured Abstract

Setting: Red Clay, located in Northern New Castle County, is the largest school district in Delaware, serving students from the city of Wilmington and its surrounding towns and suburbs. Red Clay serves more than 18,000 students from pre-K–12 in its 15 elementary, 7 middle schools, and 5 high schools. Student composition is 45 percent white, 24 percent Hispanic/Latino, 22 percent African American, 6 percent Asian with 36 percent from low-income families, 10 percent English Learners, and 12 percent are students with disabilities.

Population/Sample: The project team collected student-level data from all Grade 3–5 students in the district (approximately 3500 students) and teacher/classroom-level data from all Grades 3–5 classrooms. All Grade 3–5 classrooms have access to the MI Write software. Approximately 120 teachers were included in the qualitative component of the study through surveys and focus groups.

Data Analytic Strategy: The project team used descriptive statistics, a single-group pretest/posttest design (i.e., repeated measures), and qualitative analysis (surveys and focus groups) to examine teachers' and students' use of MI Write, their perceptions of MI Write, and ways that MI Write transformed the teaching and learning of writing. Multilevel models were used to examine the degree to which use of MI Write was associated with gains in writing quality, revising proficiency, and state test language arts and writing performance.

PRODUCTS AND PUBLICATIONS

ERIC Citations: Find available citations in ERIC for this award here.

Select publications:

Potter, A., & Wilson, J. (2021). Statewide implementation of automated writing evaluation: analyzing usage and associations with state test performance in grades 4–11. Educational Technology Research and Development, 69, 1557–1578.

Wilson, J., Huang, Y., Palermo, C., Beard, G., & MacArthur, C. A. (2021). Automated feedback and automated scoring in the elementary grades: Usage, attitudes, and associations with writing outcomes in a districtwide implementation of MI Write. International Journal of Artificial Intelligence in Education, 31, 234–276. https://doi.org/10.1007/s40593-020-00236-w

Wilson, J., Ahrendt, C., Fudge, E., Raiche, A., Beard, G., & MacArthur, C. A. (2021). Elementary teachers' perceptions of automated feedback and automated scoring: Transforming the teaching and learning of writing using automated writing evaluation. Computers & Education, 168, 104208. https://doi.org/10.1016/j.compedu.2021.104208

Wilson, J., & Rodrigues, J. (2020). Classification accuracy and efficiency of writing screening using automated essay scoring. Journal of School Psychology, 82, 123–140. https://doi.org/10.1016/j.jsp.2020.08.008


Back