Project Activities
In Years 1 to 3, the researchers will conduct a series of design-based implementation research studies to develop and refine the tool with feedback from students, teachers, and researchers. The studies will focus on creating and testing a user interface with an underlying natural language processing engine combined with automated speech recognition to enable immediate summative and formative feedback. In Year 4, the researchers will conduct an underpowered efficacy trial to measure the promise of the tool to produce results to improve reading.
Structured Abstract
Setting
The iterative studies will take place in cooperating grade schools in Minnesota, with a diverse set of student participants in terms of ethnicity, culture, and language across rural and suburban settings.
Sample
Research in Years 1 to 3 will involve focus groups and usability studies with small groups of upper elementary grade students and teachers. Participants in the Year 4 pilot study will include approximately 800 students in grades 3 and 4, and 90 teachers in grades 4 and 5.
Researchers will develop iSTART-Early to provide personalized, interactive, game-based strategy instruction and practice on comprehension strategies, that is, comprehension monitoring, paraphrasing, inferencing, question asking, explanation, and summarization, with grade appropriate informational texts. Natural language processing combined with automated speech recognition technologies will allow immediate summative and formative feedback. A teacher interface will allow teachers to assign texts and monitor students' performance so that they can provide additional support and feedback when necessary, creating blended-learning opportunities.
Research design and methods
To develop the tool, the researchers will use a design-based implementation procedure whereby iterations of research and development will occur until feasibility, usability, and learning aims are met. The steps in the technological development will focus on developing the interface, the text library, lessons, and games; and computational algorithms using natural language processing to drive feedback. After development is complete, in Year 4 the team will conduct an underpowered randomized controlled trial to explore implementation in authentic classrooms and determine whether the tool shows promise of improving students' comprehension strategy use. The 14-week study will be a pretest-posttest experimental design and will include delayed treatment control classrooms as the comparison. Researchers will collect multiple sources of data, including surveys and log data to assess the impact of the tool on student reading within the real-world constraints and needs of the classroom.
Control condition
Researchers will observe business-as-usual comprehension instruction in the control classrooms to identify how it compares to iSTART-Early.
Key measures
In the randomized control trial pilot study, the researchers will use standardized reading assessments, including the Reading Comprehension Strategies Test Comprehension Monitoring Task, the Reading Comprehension Strategies Test, the Gates-MacGinitie Reading Test (GMRT), the Woodcock-Johnson III Tests of Achievement, and the Vocabulary test of the GMRT. Another set of measures will be used to assess the usability, feasibility, and fidelity of the intervention, including questionnaires, surveys, and logs.
Data analytic strategy
The team will use mixed methods, including qualitative analytic methods to analyze transcripts of design team meetings, teacher and student interviews, and classroom observation. Researchers will use hierarchical linear and a mixed-effects models to determine the promise of the intervention and moderating effects.
People and institutions involved
IES program contact(s)
Project contributors
Products and publications
Products: Researchers will develop iSTART-Early, an intelligent tutoring system that provides automated instruction to students in grades 3 and 4 on higher-order reading comprehension strategies. An interface will support teachers in administering essay assignments, which can then be either automatically scored or hand-graded by scaffolded rubrics. Students will use this tool to support reading and research findings from this project will advance understanding of how educational technologies can improve learning from texts.
Related projects
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
