Purpose: In this project, the researchers compared different versions of AutoTutor, an intelligent tutoring system that was developed by the research team in prior work, to examine how best to support learning of course content in computer literacy and Newtonian physics. The research team compared the use of AutoTutor in its typical interactive version to a condition where students listened to and observed the AutoTutor agent presenting the same course content without any physical interaction with the source of the materials. This enabled the research team to determine whether students were equally able to learn course content during interactive and vicarious learning. The research team also examined whether mastery of course content was improved by asking students to generate reasoning questions during study ("Why is x important?").

Structured Abstract

THE FOLLOWING CONTENT DESCRIBES THE PROJECT AT THE TIME OF FUNDING

Setting: The research is taking place in the greater Memphis metropolitan area.

Sample: Middle and high school students attending Memphis city schools are participating in this research project.

Intervention: Deep-level reasoning questions (such as why and how questions) are embedded in an intelligent tutoring system called AutoTutor, which currently tutors in computer literacy and Newtonian physics. AutoTutor serves as a conversational partner with the learner and encourages students to provide answers to questions until particular concepts are mastered. The researchers are developing guidelines for teachers to support the use of AutoTutor during classroom instruction.

Research Design and Methods: A series of studies are being completed over course the grant. In the first phase, the researchers are conducting laboratory experiments to determine if increases in learning observed when deep-level reasoning questions were used to facilitate learning by college students holds for when the strategy is used by middle and high school students. These experiments are being conducted using a between-subjects pretest/posttest design. In the first two experiments, students will be randomly assigned to one of three experimental conditions to test the effects of presenting content in an interactive or vicarious format. At the same time, the researchers are working with teacher consultants to help prepare course content to be presented in the context of deep-level reasoning questions. Teachers are developing guidelines for the use of small in-class discussion groups in the experimental condition that will complement the presentation of course content via the computer. During the second and third phases of the projects, the impact of the deep-level reasoning questions on student learning is being compared to students who receive the same course content as standard classroom activities. Classes of students are being randomly assigned to condition. In addition, the number of course units instructed via this method is being increased each phase, and two new courses at each grade level are being modified to use this technique.

Control Condition: In the laboratory experiments, the three different conditions allow explicit comparison between different versions of the AutoTutor. In the classroom experiments, control students receive the same course content presented as standard classroom instruction (e.g., delivered on the computer, but without the deep-level reasoning questions). Students who served as controls during the first half of the semester become experimental students during the second half of the semester, when they have two units of course content presented in the context of deep-level reasoning questions.

Key Measures: Both experimenter-developed and teacher-developed exams covering course content and asking for responses to deep-level questions are the primary measures used to measure learning. In the experimental evaluation of the use of deep-level questions in Newtonian physics, the Force Concepts Inventory is being used to assess pretest to posttest learning gains. Audiotapes are being collected of the in-class discussions before, during, and after course content has been presented in the context of deep-level reasoning questions.

Data Analytic Strategy: Analysis of variance and analysis of covariance are being used to determine the impact of embedding vicarious deep-level reasoning questions into AutoTutor on student learning. Audiotaped data is being used to evaluate the degree to which hearing deep-level questioning in the context of AutoTutor transfers to student generation of deep-level questions during in-class discussions.

Related IES Projects: Center for the Study of Adult Literacy (CSAL): Developing Instructional Approaches Suited to the Cognitive and Motivational Needs for Struggling Adults (R305C120001)

Products and Publications

ERIC Citations: Find available citations in ERIC for this award here.

Select Publications

Books

Hacker, D.J., Dunlosky, J., and Graesser, A.C (2009). Handbook of Metacognition in Education.Mahwah, NJ: Erlbaum/Taylor and Francis.

Book chapters

Brawner, K., and Graesser, A. (2014). Natural Language, Discourse, and Conversational Dialogues Within Intelligent Tutoring Systems: A Review. In R. Sottilare, A.C. Graesser, X. Hu, and B. Goldberg (Eds.), Design Recommendations for Intelligent Tutoring Systems: Instructional Management, Volume 2 (pp. 189–204). Orlando, FL: Army Research Laboratory.

Cai, Z., Feng, S., Baer, W., and Graesser, A. (2014). Instructional Strategies in Trialogue-Based Intelligent Tutoring Systems. In R. Sottilare, A.C. Graesser, X. Hu, and B. Goldberg (Eds.), Design Recommendations for Intelligent Tutoring Systems: Instructional Management, Volume 2 (pp. 225–235). Orlando, FL: Army Research Laboratory.

Graesser, A.C. (2007). An Introduction to Strategic Reading Comprehension. In D.S. McNamara (Ed.), Reading Comprehension Strategies: Theories, Interventions, and Technologies (pp. 3–26). Mahwah, NJ: Lawrence Erlbaum Associates Publishers.

Graesser, A.C., and Forsyth, C.M. (2013). Discourse Comprehension. In D. Reisberg (Ed.), The Oxford Handbook of Cognitive Psychology(pp. 475–491). New York: Oxford University Press.

Graesser, A.C., and McNamara, D.S. (2012). Automated Analysis of Essays and Open-Ended Verbal Responses. In H. Cooper, P.M. Camic, D.L. Long, A.T. Panter, D. Rindskopf, and K.J. Sher (Eds.), APA Handbook of Research Methods in Psychology, Volume 1: Foundations, Planning, Measures, and Psychometrics(pp. 307–325). Washington, DC: American Psychological Association.

Graesser, A.C., Chipman, P., and King, B.G. (2008). Computer-Mediated Technologies. In J.M. Spector, M.D. Merrill, J.J.G. VanMerriënboer, and M.P. Driscoll (Eds.), Handbook of Research on Educational Communications and Technology (3rd ed., pp. 211–224). London: Taylor and Francis.

Graesser, A.C., D'Mello, S., and Person, N.K. (2009). Meta-Knowledge in Tutoring. In D. Hacker, J. Donlosky, and A.C. Graesser (Eds.), Handbook of Metacognition in Education (pp. 361–382). Mahway, NJ: Taylor and Francis.

Graesser, A.C., Franceschetti, D., Gholson, B., and Craig, S. (2011). Learning Newtonian Physics With Conversational Agents and Interactive Simulation. In N.L. Stein, and S. Raudenbush (Eds.), Developmental Cognitive Science Goes to School (pp. 157–172). New York: Routledge.

Graesser, A.C., Hu, X., Nye, B., and Sottilare, R. (2016). Intelligent Tutoring Systems, Serious Games, and the Generalized Intelligent Framework for Tutoring (GIFT). In H.F. O'Neil, E.L. Baker, and R.S. Perez (Eds.), Using Games and Simulation for Teaching and Assessment (pp. 58–79). New York: Routledge.

Graesser, A.C., Lin, D., and D'Mello, S. (2010). Computer Learning Environments That Support Deep Comprehension. In M.T. Banich, and D. Caccamise (Eds.), Generalization of Knowledge (pp. 201–224). Mahwah, NJ: Erlbaum.

Graesser, A.C., Ozuru, Y., and Sullins, J. (2010). What Is a Good Question?. In M.G. McKeown, and L. Kucan (Eds.), Bringing Reading Research to Life (pp. 112–141). New York: Guilford Press.

Graesser, A.C., Rus, V., D'Mello, S., and Jackson, G.T. (2008). Autotutor: Learning Through Natural Language Dialogue That Adapts to the Cognitive and Affective States of the Learner. In D.H. Robinson, and G. Schraw (Eds.), Current Perspectives on Cognition, Learning and Instruction: Recent Innovations in Educational Technology That Facilitate Student Learning (pp. 95–125). Charlotte, NC: Information Age Publishing.

Journal articles

Craig, S.D., Chi, M.T.H., and VanLehn, K. (2009). Improving Classroom Learning by Collaboratively Observing Human Tutoring Videos While Problem Solving. Journal of Educational Psychology, 101(4): 779–789.

Craig, S.D., Sullins, J., Witherspoon, A., and Gholson, B. (2006). The Deep-Level-Reasoning-Question Effect: The Role of Dialogue and Deep-Level-Reasoning Questions During Vicarious Learning. Cognition and Instruction, 24(4): 565–591.

Gholson, B., and Craig, S.D. (2006). Promoting Constructive Activities That Support Vicarious Learning During Computer-Based Instruction. Educational Psychology Review, 18(2): 119–139.

Gholson, B., Witherspoon, A., Morgan, B., Brittingham, J.K., Coles, R., Graesser, A.C., Sullins, J., and Craig, S.D. (2009). Exploring the Deep-Level Reasoning Questions Effect During Vicarious Learning Among Eighth to Eleventh Graders in the Domains of Computer Literacy and Newtonian Physics. Instructional Science, 37(5): 487–493.

Graesser, A.C. (2011). Learning, Thinking, and Emoting With Discourse Technologies. American Psychologist, 66(8): 746–757.

Graesser, A.C., Jeon, M., and Dufty, D. (2008). Agent Technologies Designed to Facilitate Interactive Knowledge Construction. Discourse Processes, 45(4): 298–322.

Graesser, A.C., Jeon, M., Yan, Y., and Cai, Z. (2007). Discourse Cohesion in Text and Tutorial Dialogue. Information Design Journal, 15(3): 199–213.

Graesser, A.C., Li, H., and Forsyth, C. (2014). Learning by Communicating in Natural Language With Conversational Agents. Current Directions in Psychological Science, 23(5): 374–380.

Graesser, A.C., and McNamara, D.S. (2010). Self-Regulated Learning in Learning Environments With Pedagogical Agents That Interact in Natural Language. Educational Psychologist, 45(4): 234–244.

Graesser, A.C., and McNamara, D.S. (2011). Computational Analyses of Multilevel Discourse Comprehension. Topics in Cognitive Science, 3(2): 371–398.

Proceedings

Craig, S.D., Graesser, A., Brittingham, J., Williams, J., Martindale, T., Williams, G., Gray, R., Darby, A., and Gholson, B. (2008). An Implementation of Vicarious Learning Environments in Middle School Classrooms. In K. McFerrin, R. Weber, R. Weber, R. Carlsen, and D.A. Willis (Eds.), Proceedings of the 19th International Conference for the Society for Information Technology and Teacher Education (pp. 1060–1064). Chesapeake, VA: AACE.