Skip to main content

Breadcrumb

Home arrow_forward_ios Information on IES-Funded Research arrow_forward_ios Using Computational Linguistics to ...
Home arrow_forward_ios ... arrow_forward_ios Using Computational Linguistics to ...
Information on IES-Funded Research
Grant Closed

Using Computational Linguistics to Detect Comprehension Processes in Constructed Responses across Multiple Large Data Sets

NCER
Program: Education Research Grants
Program topic(s): Cognition and Student Learning
Award amount: $600,000
Principal investigator: Danielle McNamara
Awardee:
Arizona State University
Year: 2019
Award period: 4 years (07/01/2019 - 06/30/2023)
Project type:
Exploration
Award number: R305A190063

Purpose

The purpose of this project is to better understand student coherence building, which students use to develop a coherent mental model of a text. Understanding text is a vital activity, enabling us to fully engage in our communities, whether through printed advertisements, electronic messaging, or signs for highway navigation. Students who struggle with coherence building have difficulty achieving deep comprehension. Using previously collected data sets, researchers will examine coherence-building strategies and explore the moderating effects of individual differences across multiple constructed response tasks and texts.

Project Activities

The research team will code and analyze students' constructed responses generated from prompts to think aloud and self-explain while reading. The data sets were collected in studies carried out in face-to-face contexts as well as automated reading strategy intelligent tutoring systems. Analyses will focus on identifying indicators of coherence-building and establishing their relations to text comprehension, individual differences, and task constraints.

Structured Abstract

Setting

The data sets are from studies conducted in urban and suburban high schools and universities in Arizona and Illinois.

Sample

The data sets include 791 high school students and 1,111 college students from two- and four-year institutions, including students who have been designated as struggling readers through college admissions standards.

Factors

The malleable factor of interest is students' coherence-building strategies and processes during reading.

Research design and methods

The research team will analyze data sets collected through multiple prior studies, in which students were asked to respond to prompts, either to think-aloud or self-explain, while reading. The team will examine the moderating effects of individual differences across multiple constructed response tasks and texts. Data analyses will incorporate measures of students' skills and motivation collected during the previously conducted studies, computational linguistic analyses of students' constructed responses, and expert judgments of comprehension strategy use. In addition, researchers will conduct replication analyses across the multiple data sets to examine the reproducibility of the outcomes.

Control condition

Due to the nature of this project, there is no control condition.

Key measures

Key measures include students' constructed responses to think aloud protocols and self-explanation prompts, vocabulary knowledge, reading skills, working memory, prior knowledge, metacognition, and motivation.

Data analytic strategy

The research team will use multiple linear regression models and linear mixed-effects models to explore how comprehension depends on linguistic features of constructed responses while controlling for individual differences. They will also incorporate analyses from dynamic systems theory to understand how readers coordinate the language in their constructed responses with the language in the text. These models allow the team to quantify stability and change in the properties of constructed responses. Finally, the research team will use machine learning techniques to develop algorithms that predict coherence-building processes and comprehension performance.

People and institutions involved

IES program contact(s)

Erin Higgins

Project contributors

Joseph Magliano

Co-principal investigator

Laura Allen

Co-principal investigator

Holly O'Rourke

Co-principal investigator

Kathryn McCarthy

Co-principal investigator

Products and publications

Researchers will provide evidence of how coherence-building supports critical aspects of text comprehension and how individual differences and tasks moderate these processes. They will also produce peer-reviewed publications and presentations.

Project website:

https://www.distributedliteracy.org

Publications:

Journal Articles

Feller, D. P., Magliano, J., Sabatini, J., O'Reilly, T., & Kopatich, R. D. (2020). Relations between Component Reading Skills, Inferences, and Comprehension Performance in Community College Readers. Discourse Processes, 57(5-6), 473-490, DOI: 10.1080/0163853X.2020.1759175

Magliano, J. P., Higgs, K., Santuzzi, A., Tonks, S. M., O'Reilly, T., Sabatini, J., ... & Parker, C. (2020). Testing the Inference Mediation Hypothesis in a Post-Secondary Context. Contemporary Educational Psychology, 61 101867.

Related projects

Assessing Reading Comprehension with Verbal Protocols and Latent Semantic Analysis

R305G040055

Exploring the onPAR Model in Developmental Literacy Education

R305A150193

Exploring the Educational Game Landscape through Focused Studies and Ecological Interventions

R305A130124

iSTART: Interactive Strategy Trainer for Active Reading and Thinking

R305G040046

Questions about this project?

To answer additional questions about this project or provide feedback, please contact the program officer.

 

Tags

CognitionEducation TechnologyReading

Share

Icon to link to Facebook social media siteIcon to link to X social media siteIcon to link to LinkedIn social media siteIcon to copy link value

Questions about this project?

To answer additional questions about this project or provide feedback, please contact the program officer.

 

You may also like

Zoomed in IES logo
Workshop/Training

Innovation Science for Education Analytics (ISEA)

January 01, 2026
Read More
Zoomed in IES logo
Fact Sheet/Infographic/FAQ

Toolkit for Teaching Reading Comprehension in Earl...

Author(s): U.S. Department of Education
Read More
Blue 2 Placeholder Pattern 1
Grant

Community Hub of Accelerate, Transform, Scale (ATS...

Award number: R305N250006
Read More
icon-dot-govicon-https icon-quote