Project Activities
In Year 1, the research team will explore the extent to which self-explanation and source evaluation strategies (e.g., sourcing, contextualization, and corroboration) are used to support intratextual (within text) and intertextual (between text) inferences in the context of MD comprehension tasks. In Years 2-4, the research team will compare the effects of comprehension strategy training, source training, and no strategy training (control) on MD comprehension. The research team will also explore the extent to which students' individual differences in prior knowledge, reading skills, and writing skills act as moderators.
Structured Abstract
Setting
The research will take place in research laboratory environments at Arizona State University, Mississippi State University, and Northern Illinois University, and in high schools in urban and suburban areas of Illinois and Mississippi.
Sample
During the first two years of the study, researchers will include approximately 400 high school students from the local community. In Years 3 and 4, researchers will work with 400-500 high school students from approximately 20 classrooms.
This project will focus on malleable factors of instruction, including comprehension strategy training combined with self-explanation and source evaluation, to support MD comprehension, as well as how individual differences moderate their effects.
Research design and methods
In Year 1, the research team will explore the extent to which self-explanation and source evaluation strategies are used to support intratextual (within text) and intertextual (between text) inferences in the context of MD comprehension tasks. Researchers will randomly assign students to one of three verbal protocol conditions: think-aloud, self-explain, or evaluate sources. In this study, students in all conditions will generate verbal protocols while they read at pre-determined times. In Year 2, the research team will randomly assign students to one of three conditions: comprehension strategy training, source training, or no strategy training. Studies conducted in Years 1 and 2 will take place in research laboratory environments. In Years 3 and 4, the research team will randomly assign students to one of three conditions: comprehension strategy training, source training, or no strategy training.
For all studies, students will participate in four sessions. In the first session, the research team will collect demographic data and will administer the first MD task, which will vary by condition. In the second session, the research team will administer another MD task, which will also vary by condition. In the third and fourth sessions, the research team will administer assessments to measure students' prior knowledge, reading skills, and writing skills. Researchers will administer all tasks on a computer and will collect eye tracking data during the laboratory studies.
Control condition
All three studies involve a manipulation of whether students are in conditions that emphasize self-explanation, source evaluation, or no strategies. In the no strategies condition, which is intended as the control condition, students will complete an unrelated task.
Key measures
Key measures include verbal protocols to capture students' inference processes; researcher-developed integrative essay responses and open-ended comprehension questions to measure MD comprehension; eye-tracking (Studies 1 and 2) and telemetry data (i.e., system log data) to capture students' MD comprehension processes; the Domain Prior Knowledge Test; the Gates-MacGinitie Reading Test; a standard SAT-style independent persuasive writing essay to assess writing ability; and an adapted version of the Online Motivation Questionnaire.
Data analytic strategy
The research team will use analysis of covariance and multilevel modeling to answer key research questions across their studies.
People and institutions involved
IES program contact(s)
Project contributors
Products and publications
Researchers will produce preliminary evidence of the potential benefits of strategy training for MD comprehension. In addition, they will produce peer-reviewed publications in research journals, present their findings at research conferences, summarize key findings through their project website, produce blog posts for the general public (e.g., through Psychology Today), and communicate their findings through social media platforms.
Project website:
Publications:
ERIC Citations: Publications available in ERIC are available here.
Select Publications
Allen, L. K., Creer, S. D., & Poulos, M. C. (2021). Natural language processing as a technique for conducting text‐based research. Language and Linguistics Compass, 15(7), e12433.
Banawan, M., Butterfuss, R., Taylor, K. S., Christhilf, K., Hsu, C., O’Loughlin, C., ... & McNamara, D. S. (2023). The future of intelligent tutoring systems for writing. In Digital Writing Technologies in Higher Education: Theory, Research, and Practice (pp. 365-383). Cham: Springer International Publishing.
Balyan, R., McCarthy, K. S., & McNamara, D. S. (2020). Applying natural language processing and hierarchical machine learning approaches to text difficulty classification. International Journal of Artificial Intelligence in Education, 30(3), 337-370.
Botarleanu, R. M., Dascalu, M., Watanabe, M., Crossley, S. A., & McNamara, D. S. (2022). Age of Exposure 2.0: Estimating word complexity using iterative models of word embeddings. Behavior research methods, 54(6), 3015-3042.
Butterfuss, R., & Kendeou, P. (2021). KReC-MD: Knowledge revision with multiple documents. Educational Psychology Review, 33(4), 1475-1497.
Butterfuss, R., Kendeou, P., McMaster, K. L., Orcutt, E., & Bulut, O. (2022). Question timing, language comprehension, and executive function in inferencing. Scientific Studies of Reading, 26(1), 61-78.
Butterfuss, R., Roscoe, R. D., Allen, L. K., McCarthy, K. S., & McNamara, D. S. (2022). Strategy uptake in Writing Pal: Adaptive feedback and instruction. Journal of Educational Computing Research, 60(3), 696-721.
Crossley, S., Wan, Q., Allen, L., & McNamara, D. (2021). Source inclusion in synthesis writing: an NLP approach to understanding argumentation, sourcing, and essay quality. Reading and Writing, 1-31.
Dascalu, M. D., Ruseti, S., Dascalu, M., McNamara, D. S., Carabas, M., Rebedea, T., & Trausan-Matu, S. (2021). Before and during COVID-19: A Cohesion Network Analysis of students’ online participation in moodle courses. Computers in Human Behavior, 121, 106780.
Mason, A. E., Braasch, J. L., Greenberg, D., Kessler, E. D., Allen, L. K., & McNamara, D. S. (2023). Comprehending multiple controversial texts about childhood vaccinations: Topic beliefs and integration instructions. Reading Psychology, 44(4), 436-462.
McCarthy, K. S., & McNamara, D. S. (2021). The multidimensional knowledge in text comprehension framework. Educational Psychologist, 56(3), 196-214.
McCarthy, K. S., Watanabe, M., Dai, J., & McNamara, D. S. (2020). Personalized learning in iSTART: Past modifications and future design. Journal of Research on Technology in Education, 52(3), 301-321.
McCarthy, K. S., Yan, E. F., Allen, L. K., Sonia, A. N., Magliano, J. P., & McNamara, D. S. (2022). On the basis of source: Impacts of individual differences on multiple-document integrated reading and writing tasks. Learning and Instruction, 79, 101599.
McCrudden, M. T., Huynh, L., Lyu, B., Kulikowich, J. M., & McNamara, D. S. (2024). Coherence building while reading multiple complementary documents. Contemporary Educational Psychology, 77, 102266.
McNamara, D. S. (2021). Chasing theory with technology: A quest to understand understanding. Discourse Processes, 58(5-6), 422-448.
McNamara, D. S. (2021). If integration is the keystone of comprehension: Inferencing is the key. Discourse Processes, 58(1), 86-91.
McNamara, D. S., Roscoe, R., Allen, L., Balyan, R., & McCarthy, K. S. (2019). Literacy: From the perspective of text and discourse theory. Journal of Language and Education, 5(3), 56-69.
Sonia, A. N., Magliano, J. P., McCarthy, K. S., Creer, S. D., McNamara, D. S., & Allen, L. K. (2022). Integration in multiple-document comprehension: A natural language processing approach. Discourse Processes, 59(5-6), 417-438.
Watanabe, M., & McNamara, D. S. (2023). The Motivational Utility of Knowledge: Examining Fundamental Needs in the Context of Houselessness Knowledge. Knowledge, 3(4), 642-661.
Related projects
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
