Purpose: Computer tutors have been successful in improving learning by employing "intelligent" techniques to guide online, tailored interaction appropriate for a particular user. Although such tutors are responsive to the user's on-going performance, they do not attend to the user's affective state (e.g., levels of frustration, self-confidence). The purpose of this project is to extend the capabilities of two web-based mathematics tutors to assess online the user's affect and respond in ways to minimize the likelihood of disengagement. This should promote a supportive and fruitful learning environment.

Project Activities: The researchers will develop and evaluate adaptive intelligent tutors that track individual student affect and cognition. Three important issues in mathematics education will be addressed: (1) the large impact of student emotion on learning and high-stakes testing; (2) individualized instruction and feedback to a particular user, which should not only benefit all students but especially those with learning disabilities; and (3) economizing time spent on formative assessments at the expense of instruction time. Researchers will iteratively design and enhance two intelligent tutors, Wayang Outpost and 4mality, both developed under grants from the Department of Education and the National Science Foundation, to foster student engagement with mathematical thinking. Additionally, the tutor will provide teachers with assessments for each student.

Products: The major expected product is an intelligent mathematics tutor that will contain three major components. The first component is affect detection software that estimates the emotional state of the user. This dynamic estimation of the user's current state will help determine the appropriate difficulty level for ensuing math problems. The second component is a suite of interventions that has the potential to bring disengaged students—whether from frustration, boredom, or low self-confidence—back to an engaged state. Finally, the third component consists of teacher assessment tools which inform teachers about each student's progress and affect.

Structured Abstract

Setting: Participating schools will be located in urban and rural areas of Massachusetts.

Population: About 440 students in the 4th–5th and 8th–10th grades are expected to participate.

Intervention: The researchers will develop an intelligent computer tutor that detects and assesses user affect online. This information will be used to determine if the user is in a non-engaged state (e.g., is frustrated, unengaged, etc.), and if so, will present interventions designed to bring the user back to an engaged state. Additionally, teacher assessment tools will be developed, which will enhance the teacher's ability to immediately assess the students' affect and aid in instruction planning.

Research Design and Methods: The affect detection software will distinguish among different potential states (e.g., frustration, boredom, lack of self-esteem) without sensors. The development process entails two stages. The first stage is data collection, where human observers make judgments on the students' states as they use the tutor in the classroom. The second stage is the machine learning process. The data collected in the first stage will serve as training examples for the machine learning algorithms to estimate a user's current state. The algorithms will learn rules by generalizing from these examples so that the rules can be applied to new examples. This "supervised learning" allows the researchers to form a model of affective hidden variables, which will be integrated into the student model. As a result, the tutor can refine its inference of frustration, engagement, and confidence.

When the algorithm detects student disengagement, digital intervention tools (e.g., progress visualization charts, self-referencing charts, an Advanced Emotional Notebook) will be deployed to help counteract this state. They will be designed to enable students to collect, reflect, display and share newly acquired math concepts and for students to assess and monitor their own learning.

Assessment reports of student performance will be generated so that teachers can adjust their instructional strategies and actively encourage students to assess their own learning. Teachers' use of the reports will be assessed via interviews and tracking their use of the reports' hyperlinks, which allow for in-depth examination of the data.

The feasibility of implementing the intervention in mathematics classrooms as well as the promise of the intervention for improving student mathematics outcomes will be examined by conducting 11 field studies, each including 40 students.

Control Condition: The control group will be composed of matched-sample students (by grade, proficiency level, and teachers) who will use the current version of the tutor (that is, without the affect detection software). They will be able to access the multimedia hints by selecting the Help button.

Key Measures: There will be several measures, including the effectiveness of interventions (i.e., the amount of post-intervention engagement), pre-post gains on tests administered before and after use of the tutor, scores on the Massachusetts Comprehensive Assessment System (MCAS) (comparing last year's with the current year), and the amount of learning of students starting at different emotional states.

Data Analytic Strategy: Pre- and post-test scores on the MCAS will be analyzed using a repeated-measures analysis of variance, using teacher and test conditions as factors, to assess the instructional value of the tutor. The effectiveness of the digital interventions to re-engage users after becoming disengaged will be assessed by using logistic regression. Student surveys will be used assess their perception of the tutor's utility and helpfulness.

Products and Publications

Book chapter

Arroyo, I., Cooper, D.G., Burleson, W., and Woolf, B.P. (2010). Bayesian Networks and Linear Regression Models of Students' Goals, Moods, and Emotions. In R.S.J.D. Baker, and K. Yacef (Eds.), Handbook of Educational Data Mining (pp. 323–338). New York: Routledge Press.

Cooper, D., Arroyo, I., and Woolf, B.P. (2011). Actionable Affective Processing for Automatic Tutor Intervention. In S. D'Mello, and R. Calvo (Eds.), Affect and Learning Technologies (pp. 127–140). New York: Springer Publishing.

Woolf, B. (2010). Affective Tutors: Automatic Detection of and Response to Student Emotion. In R. Nkambou, J. Bourdeau, and R. Mizoguchi (Eds.), Advances in Intelligent Tutoring Systems (pp. 207–227). Berlin-Heidelberg: Springer.

Woolf, B. (2010). Student Modelling. In R. Nkambou, J. Bourdeau, and R. Mizoguchi (Eds.), Advances in Intelligent Tutoring Systems (pp. 267–279). Berlin-Heidelberg: Springer.

Woolf, B.P., Arroyo, I., Muldner, K., Burleson, W., Cooper, D., Dolan, R., and Christopherson, R.M. (2010). The Effect of Motivational Learning Companions on Low-Achieving Students and Students With Learning Disabilities. In V. Aleven, J. Kay, and J. Mostow (Eds.), Intelligent Tutoring Systems (pp. 327–337). Pittsburg, PA.: Springer.

Journal article, monograph, or newsletter

Arroyo, I., Burleson, W., Tai, M., Muldner, K., and Woolf, B.P. (2013). Gender Differences in the Use and Benefit of Advanced Learning Technologies for Mathematics. Journal of Educational Psychology, 105(4): 957–969.

Arroyo, I., Woolf, B.P., and Burleson, W. (2011). Using an Intelligent Tutor and Math Fluency Training to Improve Math Performance. International Journal of Artificial Intelligence in Education, 21(1): 135–152.

Woolf, B.P., Burleson, W., Arroyo, I., Dragon, T., and Picard, R. (2009). Affect-Aware Tutors: Recognizing and Responding to Student Affect Emotional Intelligence for Computer Tutors, Special Issue on Modeling and Scaffolding Affective Experiences to Impact Learning. International Journal of Learning Technology, 4(3): 129–164.

Proceeding

Arroyo, I., Mehranian, H., and Woolf, B. (2010). Effort-Based Tutoring: An Empirical Approach to Intelligent Tutoring. In Proceedings of the Third International Conference on Educational Data Mining (pp. 1–10). Pittsburgh, PA.

Woolf, B. (2010). Social and Caring Tutors. In Proceedings of the International Conference on Intelligent Tutoring Systems (pp. 5–13). New York: Springer.