Skip to main content

Breadcrumb

Home arrow_forward_ios Information on IES-Funded Research arrow_forward_ios Guru: A Computer Tutor that Models ...
Home arrow_forward_ios ... arrow_forward_ios Guru: A Computer Tutor that Models ...
Information on IES-Funded Research
Grant Closed

Guru: A Computer Tutor that Models Expert Human Tutors

NCER
Program: Education Research Grants
Program topic(s): Education Technology
Award amount: $1,858,176
Principal investigator: Andrew Olney
Awardee:
University of Memphis
Year: 2008
Project type:
Development and Innovation
Award number: R305A080594

Purpose

There is substantial empirical evidence that one-on-one human tutoring is an effective supplement to typical classroom instruction. Yet, human tutors cannot be provided to every child—as there are simply not enough tutors. This project will develop Guru, a computer-based intelligent tutoring system, to assist high school students with biology learning.

Project Activities

After developing prototypes of the individual software components of the intervention, researchers will conduct a series of design experiments focusing on implementation in real-world settings. Cycles of think-aloud protocols and eye tracking data from students will be collected to evaluate the usability and feasibility of the system. Expert human biology tutors will also aid in this evaluation and will provide feedback on the biology content. An evaluation will be conducted with the refined system to examine the promise of Guru for enhancing student learning.

Structured Abstract

Setting

The setting is urban school districts in Memphis, Tennessee.

Sample

The sample will include approximately 50 ninth-grade students in Memphis City Schools. In 2005 in this district, 75% of students were below proficient in science, 87% of students were African-American, 71% were eligible for free or reduced lunch, 79% were economically disadvantaged, and the graduation rate was 67%.
Intervention
Guru is an expert computer biology tutor, designed to be used as a supplement to classroom-based instruction in biology. When finalized, students will interact with the Guru animated agent by having a conversation with the tutor. During the course of the conversation, Guru and the student will work through biology topics and problems through a multimedia panel that presents movies, interactive diagrams, and other instructional media. As with expert human tutoring, an essential aspect of the Guru intervention is to precisely assess the student's answers and maintain an accurate model of the student's understanding. Guru is intended to promote educational attainment by targeting biology content that high school students must master in order to graduate in the state of Tennessee.

Research design and methods

The researchers will employ an Integrative Learning Design Framework to develop Guru. In this process, the components of the artificial intelligence system will be developed and then each component will be individually measured. Through the iterative design process, the expert human tutors and students will also participate in ongoing pilot tests and provide feedback on the biology content presented, as well as on the usability and feasibility of the system. The final pilot study will employ a counterbalanced within subjects repeated measures design with 30 students to test the promise of Guru for enhancing student learning outcomes.

Key measures

Think-aloud protocols and eye tracking data from the target population will be collected and used to evaluate the usability and feasibility of the system, as well as to gather more information about users and needs. Multiple choice assessments will be used to measure learning of the biology content presented in the tutor. Measures used in the final pilot study will come directly from the Tennessee Gateway Biology Test.

Data analytic strategy

Data from interaction studies with expert human tutors, educators, and students will be used to verify that dialogs with the Guru expert tutor are consistent with that of a human expert tutor. This will be accomplished using correlational analyses of dialog as well as bystander Turing tests in which a human makes judgments on whether dialog was generated by a human or computer. The data collected during the final pilot study will be analyzed using an analysis of variance to test whether student scores are stable across assessments of topics in which no tutoring occurred, as well as the extent to which student's scores increase across assessments of topics for which they received tutoring.

People and institutions involved

IES program contact(s)

Edward Metz

Education Research Analyst
NCER

Products and publications

Products: The intervention will consist of an expert computer tutor designed to support student achievement in biology.

Book chapter

Brawner, K., and Graesser, A. (2014). Natural Language, Discourse, and Conversational Dialogues Within Intelligent Tutoring Systems: A Review. In R. Sottilare, A.C. Graesser, X. Hu, and B. Goldberg (Eds.), Design Recommendations for Intelligent Tutoring Systems: Instructional Management, Volume 2 (pp. 189-204). Orlando, FL: Army Research Laboratory.

Cai, Z., Feng, S., Baer, W., and Graesser, A. (2014). Instructional Strategies in Trialogue-Based Intelligent Tutoring Systems. In R. Sottilare, A.C. Graesser, X. Hu, and B. Goldberg (Eds.), Design Recommendations for Intelligent Tutoring Systems: Instructional Management, Volume 2 (pp. 225-235). Orlando, FL: Army Research Laboratory.

Cai, Z., Graesser, A.C., Hu, X., & Cockroft, J. L. (2019). Self-improving components in conversational intelligent tutoring systems. In A. Sinatra, A.C. Graesser, X. Hu, K. Brawner and V. Rus (Eds.), Design Recommendations for Intelligent Tutoring Systems: Self-improving systems (Vol.7) (pp. 119-126). Orlando, FL: Army Research Laboratory.

Graesser, A.C., and D'Mello, S.K. (2012). Emotions During the Learning of Difficult Material. In B.H. Ross (Ed.), The Psychology of Learning and Motivation, Volume 57 (pp. 183-225). San Diego: Elsevier Academic Press.

Graesser, A.C., and Forsyth, C.M. (2013). Discourse Comprehension. In D. Reisberg (Ed.), The Oxford Handbook of Cognitive Psychology (pp. 475-491). New York: Oxford University Press.

Graesser, A.C., and Lehman, B. (2012). Questions Drive Comprehension of Text and Multimedia. In M.T. McCrudden, J. Magliano, and G. Schraw (Eds.), Text Relevance and Learning From Text (pp. 53-74). Greenwich, CT: Information Age Publishing.

Graesser, A.C., and McNamara, D.S. (2012). Automated Analysis of Essays and Open-Ended Verbal Responses. In H. Cooper, P.M. Camic, D.L. Long, A.T. Panter, D. Rindskopf, and K.J. Sher (Eds.), APA Handbook of Research Methods in Psychology, Volume 1: Foundations, Planning, Measures, and Psychometrics (pp. 307-325). Washington, DC: American Psychological Association.

Graesser, A.C., Baer, W., Feng, S., Walker, B., Clewley, D., Hays, D.P., Greenberg, D. (2015). Emotions in adaptive computer technologies for adults improving reading. In S. Tettegah and M. Gartmeier (Eds.), Emotions, Technology, Design, and Learning (pp. 3-25). New York: Elsevier.

Graesser, A.C., Keshtkar, F., and Li, H. (2014). The Role of Natural Language and Discourse Processing in Advanced Tutoring Systems. In T. Holtgraves (Ed.), The Oxford Handbook of Language and Social Psychology (pp. 491-509). New York: Oxford Handbooks Online.

Graesser, A.C., Feng, S., & Cai, Z. (2017). Two technologies to help adults with reading difficulties improve their comprehension. In E. Segers and P. Van den Broek (Eds.), Developmental perspectives in written language and literacy. In honor of Ludo Verhoeven (pp. 295-313). John Benjamin Publishing Company.

Graesser, A. C., Lippert, A. M., & Hampton, A. J. (2017). Successes and failures in building learning environments to promote deep learning: The value of conversational agents. In J. Buder & F. W. Hesse (Eds.), Informational Environments: Effects of Use, Effective Designs (pp. 273-298). Springer International Publishing. https://doi.org/10.1007/978-3-319-64274-1_12

Graesser, A.C., Millis, K., D'Mello, S.K., and Hu, X. (2014). Conversational Agents can Help Humans Identify Flaws in the Science Reported in Digital Media. In D. Rapp, and J. Braasch (Eds.), Processing Inaccurate Information: Theoretical and Applied Perspectives From Applied Perspectives From Cognitive Science and the Educational Sciences (pp. 139-159). Cambridge, MA: MIT Press.

Li, H., Shubeck, K. and Graesser, A. C. (2016). Using Technology in Language Assessment. In D., Tsagari, & J. V. Banerjee. (Eds.), Contemporary Second Language Assessment: Contemporary Applied Linguistics (Vol. 4, pp. 281-297). London, UK: Bloomsbury Academic.

Journal article, monograph, or newsletter

Baker, R.S., D'Mello, S.K., Rodrigo, M.T., and Graesser, A.C. (2010). Better to be Frustrated Than Bored: The Incidence, Persistence, and Impact of Learners' Cognitive-Affective States During Interactions With Three Different Computer-Based Learning Environments. International Journal of Human-Computer Studies, 68: 223-241.

D'Mello, S., and Graesser, A.C. (2010). Multimodal Semi-Automated Affect Detection From Conversational Cues, Gross Body Language, and Facial Features. User Modeling and User-Adapted Interaction, 20(2): 147-187.

D'Mello, S., Craig, S., and Graesser, A. (2009). Multi-Method Assessment of Affective Experience and Expression During Deep Learning. International Journal of Learning Technology, 4(3): 165-187.

D'Mello, S., King, B., Chipman, P., and Graesser, A.C. (2010). Towards Spoken Human-Computer Tutorial Dialogues. Human Computer Interaction, 23: 289-323.

D'Mello, S.K., and Graesser, A.C. (2009). Automatic Detection of Learner's Affect From Gross Body Language. Applied Artificial Intelligence, 23: 123-150.

D'Mello, S.K., Lehman, B., and Person, N. (2010). Monitoring Affect States During Effortful Problem Solving Activities. International Journal of Artificial Intelligence in Education, 20(4): 361-389. doi:10.3233/JAI-2010-012

D'Mello, S.K., Olney, A., and Person, N. (2010). Mining Collaborative Patterns in Tutorial Dialogues. Journal of Educational Data Mining, 2(1): 1-37.

D'Mello, S.K, and Graesser, A.C. (2012). Dynamics of Affective States During Complex Learning. Learning and Instruction, 22(2): 145-157.

Gholson, B., Witherspoon, A., Morgan, B., Brittingham, J.K., Coles, R., Graesser, A.C., Sullins, J., and Craig, S.D. (2009). Exploring the Deep-Level Reasoning Questions Effect During Vicarious Learning Among Eighth to Eleventh Graders in the Domains of Computer Literacy and Newtonian Physics. Instructional Science, 37(5): 487-493.

Graesser, A.C. (2009). Cognitive Scientists Prefer Theories and Testable Principles With Teeth. Educational Psychologist, 44(3): 193-197.

Graesser, A.C. (2011). Learning, Thinking, and Emoting With Discourse Technologies. American Psychologist, 66(8): 746-757.

Graesser, A.C. (2016). Conversations with AutoTutor help students learn. International Journal of Artificial Intelligence in Education, 26, 124 -132. Full text

Graesser, A.C., and McNamara, D.S. (2010). Self-Regulated Learning in Learning Environments With Pedagogical Agents That Interact in Natural Language. Educational Psychologist, 45(4): 234-244.

Graesser, A.C., and McNamara, D.S. (2011). Computational Analyses of Multilevel Discourse Comprehension. Topics in Cognitive Science, 3(2): 371-398.

Graesser, A. C., Cai, Z., Morgan, B., & Wang, L. (2017). Assessment with computer agents that engage in conversational dialogues and trialogues with learners. Computers in Human Behavior, 76, 607-616. Full text

Graesser, A.C., Li, H., and Forsyth, C. (2014). Learning by Communicating in Natural Language With Conversational Agents. Current Directions in Psychological Science, 23(5): 374-380. Full text

Graesser, A.C., Forsyth, C., and Lehman, B. (2017). Two Heads May be Better Than One: Learning From Agents in Conversational Trialogues. Teacher College Record, 119(3). Full text

Kopp, K., Britt, A., Millis, K., and Graesser, A. (2012). Improving the Efficiency of Dialogue in Tutoring. Learning and Instruction, 22(5): 320-330.

Lippert, A., Shubeck, K., Morgan, B., Hampton, A, & Graesser, A.C. (2020) Multiple Agent Designs in Conversational Intelligent Tutoring Systems. Technology, Knowledge and Learning, v25 n3 p443-463.

Louwerse, M.M., Graesser, A.C., McNamara, D.S., and Lu, S. (2009). Embodied Conversational Agents as Conversational Partners. Applied Cognitive Psychology, 23(9): 1244-1255.

Nye, B.D., Graesser, A.C., and Hu, X. (2014). AutoTutor and Family: A Review of 17 Years of Natural Language Tutoring. International Journal of Artificial Intelligence in Education, 24(4): 427-469. Full text

Olney, A. (2011). Large-Scale Latent Semantic Analysis. Behavior Research Methods, 43(2): 414-423.

Rus, V., McCarthy, P.M., McNamara, D.S., and Graesser, A.C. (2009). Identification of Sentence-to-Sentence Relations Using a Text Entailer. Research on Language and Computation, 7(2): 371-398.

Wiley, J., Goldman, S.R., Graesser, A.C., Sanchez, C.A., Ash, I.K., and Hemmerich, J.A. (2009). Source Evaluation, Comprehension, and Learning in Internet Science Inquiry Tasks. American Educational Research Journal, 46(4): 1060-1106.

Proceeding

Cai, Z., Li, H., Hu, X., & Graesser A. C. (2016). Can word probabilities from LDA be simply added up to represent documents? In T. Barnes, M. Chi, & M. Feng (Eds.), In Proceedings of the 9th International Conference on Educational Data Mining (pp. 577-578). Raleigh, North Carolina: EDM Society. Full text

Cai, Z., Siebert-Evernston, A., Eagan, B., Shaffer, D.W., Hu, X., & Graesser, A.C. (2019). nCoder+: A Semantic Tool for Improving Recall of nCoder Coding. In B. Eagan, M. Misfeldt, A. Siebert-Evenstone (Eds.), Advances in Quantitative Ethnography Proceedings of the First International Conference on Quantitative Ethnography (pp.41-54). Madison, WI: Springer, Cham.

Li, H., Cai, Z., & Graesser A. C. (2016). How good is popularity? Summary grading in crowdsourcing. In T. Barnes, M. Chi, & M. Feng (Eds.), Proceedings of the 9th International Conference on Educational Data Mining (pp. 430-435). Raleigh, North Carolina: EDM Society. Full text

Li, H., Cheng, C., Yu, Q., & Graesser A. C. (2015). The role of peer agent's learning competency in trialogue-based reading intelligent systems. In C. Conati, N. Heffernan, A. Mitrovic, M. F. Verdejo (Eds.), Proceedings of 17th International Conference on Artificial Intelligence in Education (pp. 694-697). Cham: Springer. Full text

Supplemental information

Purpose: The purpose of the proposed project is to develop and evaluate Guru, an expert computer biology tutor, by modeling the strategies and dialog of expert human tutors. Once developed, Guru can be used to further understand the processes and mechanisms of expert tutoring by manipulating strategies and dialog moves within the tutor and observing associated student learning outcomes.

Questions about this project?

To answer additional questions about this project or provide feedback, please contact the program officer.

 

Tags

Policies and StandardsEducation TechnologyScience

Share

Icon to link to Facebook social media siteIcon to link to X social media siteIcon to link to LinkedIn social media siteIcon to copy link value

Questions about this project?

To answer additional questions about this project or provide feedback, please contact the program officer.

 

You may also like

Zoomed in IES logo
Workshop/Training

Data Science Methods for Digital Learning Platform...

August 18, 2025
Read More
Zoomed in IES logo
Workshop/Training

Data Science for Education (DS4EDU)

April 01, 2025
Read More
Zoomed in IES logo
Request for Applications

Education Research and Development Center Program ...

March 14, 2025
Read More
icon-dot-govicon-https icon-quote