Description:

Co-Principal Investigators: Koedinger, Kenneth R.; Cohen, William W.; Stylianides, Gabriel

Purpose: The purpose of this project is to develop an intelligent tutoring system designed to help students master algebra concepts related to solving linear equations. In this intelligent tutoring system, students teach a computer learner, SimStudent, how to solve linear equations. The students select linear equations for SimStudent to solve and monitor SimStudent's performance, providing hints and feedback. The goal of the tutoring is for students to improve their understanding of algebraic concepts, remediate their own misconceptions, and strengthen their problem-solving ability and procedural knowledge of solving linear equations.

Project Activities: Over three years, the research team will develop and evaluate three versions of SimStudent in order to determine the degree to which each version of the system supports student learning in algebra. In Year 1, the system will be designed so that participating students will tutor SimStudent so that it can pass end-of-level quizzes. In Year 2, the research team will develop a game-show environment in which several SimStudents, each tutored by different student tutors, will compete against each other. In Year 3, the research team will modify the system so that students are asked to diagnose SimStudent'smisconceptions and then plan remedial problems that will enable SimStudent to learn from the experience and develop the desired math skill. In addition, the team will develop a meta-tutor that will provide scaffolding to students on their selected problems and gives suggestions for more strategically selected problems. In each year of the project, student performance using SimStudent will be compared to students who learn algebra using different computer software supports.

Products: Products of this development project include a fully developed, web-based intelligent tutoring system, SimStudent, designed to support student mastery of linear equations. Additionally, a meta-tutor will be developed that monitors students' suggestions for SimStudent and provides scaffolded feedback to the students. Published reports of the research findings will also be produced.

Structured Abstract

Setting: The study will be conducted in intact classrooms that participate in the LearnLab project through the Pittsburgh Science of Learning Center.

Population: Participating students will be enrolled in the second half of 8th or 9th grade Algebra I. They will only participate after they have learned basic concepts and skills of linear equation solving.

Intervention: The intervention will be an intelligent tutoring system, modeled after the Algebra Cognitive Tutor, where students with some algebra knowledge tutor the SimStudent by providing feedback and hints, and monitoring its performance. The expected duration of interacting with the SimStudent is three weeks, as part of Algebra I class work. Three different versions of SimStudent will be developed.

Research Design and Methods: All studies will employ a between-subjects experimental design. Participants will be randomly assigned to a condition, with the treatment condition participants receiving the intervention and the control condition participants exposed to a different system or performing other algebra activities. In Year 1, the treatment participants will be exposed to SimStudent (testing the learning by teaching hypothesis), with the goal of tutoring SimStudent so that it can pass the end-of-level quizzes. In Year 2, treatment and control students will both be exposed to the SimStudent, but the goals of their activities will differ. The treatment condition students will be preparing SimStudent for the game-show competition (testing the social motivation of learning hypothesis) and the control students will tutor SimStudent so that it can pass the end-of-level quizzes. In Year 3, treatment and control students will again both be exposed to the SimStudent. However, the actions of students in the treatment condition will be monitored by a meta-tutor that will provide feedback to the students regarding their choice of problem-selection and hints to SimStudent.

Control Condition: Students in the control condition will use a different intelligent tutoring system, the Carnegie Learning Algebra I Tutor, for the first study in Year 1. Students in the control condition in Year 2 will be exposed to SimStudent but they will not have the component of the game-show competition. Students in the control condition in Year 3 will not be exposed to the meta-tutor.

Key Measures: Performance measures will be collected both pre- and post-exposure to the intervention and will assess problem solving ability and conceptual knowledge. This will include measures to assess the students' ability to judge the correctness of steps taken to solve algebraic equations and to perform the next step in a series of steps. Additionally, measures of students' reasoning skills will be collected. Finally, post-exposure measures will assess both near and far transfer, using researcher developed measures to assess near transfer, and standardized measures of algebra to assess far transfer.

Data Analytic Strategy: Using analysis of variance techniques, between-subjects comparisons will be conducted between students in the treatment and control conditions. Additionally, within-subjects comparisons for all students will test for learning gains by comparing performance at pre- and post-exposure to the intervention, and performance at near and far transfer.

Products and Publications

Book chapter

Matsuda, N., Griger, C.L., Barbalios, N., Stylianides, G.J., Cohen, W. W., and Koedinger, K. R. (2014). Investigating the Effect of Meta-cognitive Scaffolding for Learning by Teaching. In S. Trausan-Matu, K.E. Boyer, M. Crosby, and K. Panourgia (Eds.), Intelligent Tutoring Systems. ITS 2014. Lecture Notes in Computer Science (pp. 104–113). Springer.

Journal article, monograph, or newsletter

Li, N., Matsuda, N., Cohen, W.W., and Koedinger, K.R. (2015). Integrating Representation Learning and Skill Learning in a Human-like Intelligent Agent. Artificial Intelligence, 219: 67–91.

Matsuda, N., Yarzebinski, E., Keiser, V., Raizada, R., Cohen, W.W., Stylianides, G.J., and Koedinger, K.R. (2013). Cognitive Anatomy of Tutor Learning: Lessons Learned with SimStudent. Journal of Educational Psychology, 105(4): 1152–1163.

Matsuda, N., Yarzebinski, E., Keiser, V., Raizada, R., Stylianides, G.J., and Koedinger, K.R. (2013). Studying the Effect of a Competitive Game Show in a Learning by Teaching Environment. International Journal of Artificial Intelligence in Education, 23(1): 1–21.

Proceeding

Carlson, R., Keiser, V., Matsuda, N., Koedinger, K., and Penstein Rosé, C. (2012). Building a Conversational SimStudent. In Intelligent Tutoring Systems: Lecture Notes in Computer Science (pp. 563–569). Berlin/Heidelberg: Springer. doi:10.1007/978–3–642–30950–2

Li, N., Cohen, W., Koedinger, K.R., and Matsuda, N. (2011). A Machine Learning Approach for Automatic Student Model Discovery. In Proceedings of the 4th International Conference on Educational Data Mining (pp. 31–40). Eindhoven, The Netherlands.

Li, N., Matsuda, N., Cohen, W. W., and Koedinger, K. R. (2012). Towards a Computational Model of Why Some Students Learn Faster than Others. In AAAI Fall Symposium: Cognitive and Metacognitive Educational Systems. MacLellan, C. J., Harpstead, E., Wiese, E.S., Zou, M., Matsuda, N., Aleven, V., and Koedinger, K.R. (2015). Authoring Tutors with Complex Solutions: A Comparative Analysis of Example Tracing and SimStudent. In Workshops at the 17th International Conference on Artificial Intelligence in Education AIED (pp. 35–44).

MacLellan, C.J., Koedinger, K.R., and Matsuda, N. (2014). Authoring Tutors with SimStudent: An Evaluation of Efficiency and Model Quality. In International Conference on Intelligent Tutoring Systems (pp. 551–560).

MacLellan, C.J., Wiese, E.S., Matsuda, N., and Koedinger, K.R. (2015). SimStudent: Authoring Expert Models by Tutoring. In Proceedings of the 2nd Annual GIFT Users Symposium (pp. 25–32).

Matsuda, N., Cohen, W.W., Koedinger, K.R., and Stylianides, G. (2010). Learning to Solve Algebraic Equations by Teaching a Computer Agent. In Proceedings of the Conference of the International Group for the Psychology of Mathematics Education, Volume 2 (pp. 69).

Matsuda, N., Keiser, V., Raizada, R., Tu, A., Stylianides, G., Cohen, W.W., and Koedinger, K.R. (2010). Learning by Teaching SimStudent: Technical Accomplishments and an Initial Use With Students. In Proceedings of the International Conference on Intelligent Tutoring Systems (pp. 317–326). Heidelberg, Germany: Springer.

Matsuda, N., Keiser, V., Raizada, R., Yarzebinski, E., Watson, S., and Stylianides, G.J. (2012). Studying the Effect of Tutor Learning Using a Teachable Agent That Asks the Student Tutor for Explanations. In Proceedings of the International Conference on Digital Game and Intelligent Toy Enhanced Learning (DIGITEL 2012) (pp. 25–32). Los Alamitos, CA: IEEE Computer Society.

Matsuda, N., Yarzebinski, E., Keiser, V., Raizada, R., Cohen, W.W., Stylianides, G., and Koedinger, K. R. (2012). Shallow Learning as a Pathway for Successful Learning Both for Tutors and Tutees. In Proceedings of the Cognitive Science Society (pp. 731–736).

Matsuda, N., Yarzebinski, E., Keiser, V., Raizada, R., Stylianides, G., and Cohen, W.W. (2012). Motivational Factors for Learning by Teaching: The Effect of a Competitive Game Show in a Virtual Peer-Learning Environment. In Proceedings of the International Conference on Intelligent Tutoring Systems (pp. 101–111). Heidelberg, Germany: Springer-Verlag.

Matsuda, N., Yarzebinski, E., Keiser, V., Raizada, R., Stylianides, G., Cohen, W., and Koedinger, K. (2011). Learning by Teaching SimStudent–An Initial Classroom Baseline Study Comparing with Cognitive Tutor. In Artificial Intelligence in Education (pp. 213–221). Berlin/Heidelberg: Springer.

Ogan, A., Finkelstein, S., Mayfield, E., D'Adamo, C., Matsuda, N., and Cassell, J. (2012). Oh Dear Stacy!: Social Interaction, Elaboration, and Learning with Teachable Agents. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 39–48). New York, NY: ACM.

Ogan, A., Finkelstein, S.L., Walker, E., Carlson, R., and Cassell, J. (2012). Rudeness and Rapport: Insults and Learning Gains in Peer Tutoring. In ITS (pp. 11–21).

Rosenberg-Kima, R.B., and Pardos, Z.A. (2015). Is this Model for Real? Simulating Data to Reveal the Proximity of a Model to Reality. In Second Workshop on Simulated Learners: AIED 2015 Workshop Proceedings (pp. 78–87).