Skip Navigation

Cognition and Student Learning

Grantees

- OR -

Investigator

- OR -

Goals

- OR -

FY Awards

- OR -

Dynamically Modifying the Learning Trajectories of Novices with Pedagogical Agents

Year: 2005
Name of Institution:
University of Southern California
Goal: Development and Innovation
Principal Investigator:
Beal, Carole
Award Amount: $1,220,822
Award Period: 3 years
Award Number: R305H050052

Description:

Purpose:  In this project, the researchers developed an intervention designed to help novice chemistry learners revise their strategy while solving chemistry problems. This intervention built on "learning trajectories", a concept based on the observation that novices and experts think and perform differently. Learning trajectories aim to capture the stages of understanding as experience is developed, namely that initially knowledge is limited and fragmented, making it difficult for students to understand problems, but that with practice, student's knowledge becomes deeper and more structured, allowing their problem solving to become more strategic until most students adopt an approach they are comfortable with. Building on prior research, the researchers refined instructional approaches to build this expertise and to develop predictive models of problem solving that provided individualized instruction to novice learners.

Structured Abstract

THE FOLLOWING CONTENT DESCRIBES THE PROJECT AT THE TIME OF FUNDING

Setting: Middle and high schools in a school district in urban Southern California are participating.

Sample: Students in the participating school district are widely diverse in terms of ethnicity and socioeconomic status. The student body is 59 percent white, 30 percent Hispanic, 8 percent Asian, and 2 percent African American.

Intervention: The variables being examined are tested within the context of a web-based problem-solving simulation program called IMMEX (Interactive Multi-Media Exercises). Through IMMEX, students learn to frame a problem from a descriptive scenario, judge what information is relevant, plan a search strategy, gather information, and reach a decision that demonstrates understanding. Students' sequences of actions are used to model learning trajectories that can predict future performance. One objective is to use IMMEX data to identify students who are likely to persist with unproductive strategies. The IMMEX program is being modified in this project in order to integrate a pedagogical model into the system that will provide neutral feedback (namely feedback that provides general encouragement but does not provide specific instructions about what students should do next in order to facilitate learning) or individualized feedback that explicitly addresses how the student is approaching the problem. In addition, the program will deliver feedback either via simple text messages or interactive, animated pedagogical agents. Students work with IMMEX in their science classes under the direction of their classroom instructor.

Research Design and Methods: Students will be randomly assigned to work with different versions of the web-based IMMEX problem solving simulations, created to evaluate the role of neutral vs. individualized feedback, text vs. pedagogical agent delivery, and agent characteristics (such as gender, ethnicity, perceived scientific credibility and authority).

In phase 1 of the project, learning outcomes are being compared for students who work with (1) classic IMMEX (overall control), (2) a version of IMMEX that includes individualized pedagogical feedback on strategy use in the form of text messages (experimental), and (3) a version of IMMEX in which students receive neutral feedback through text messages (feedback control). In phase 2, the researchers are comparing the learning trajectories of students who work with IMMEX in four versions, representing a two-by-two design, in which students receive neutral or individualized feedback, through either text or animated agents. In phase 3, the researchers are investigating the specific characteristics of agents that are most effective for groups of students.

Control Condition: The control condition is the classic IMMEX program in which students are provided with multiple opportunities to practice solving related problems but do not receive explicit feedback as they work through the problem sets.

Key Measures: Individual student problem solving sequences are automatically recorded as students work on the IMMEX system. In addition, the school district collects achievement data that is being used both as a covariate, and as a way to measure the impact of the intervention on achievement.

Data Analytic Strategy: Both analysis of covariance and hierarchical linear modeling are being used to examine student performance in the different versions of IMMEX.

Products and Publications

ERIC Citations: Find available citations in ERIC for this award here.

Select Publications

Book chapters

Beal, C.R., Shaw, E., and Birch, M. (2007). Intelligent Tutoring and Human Tutoring in Small Groups: An Empirical Comparison. In R. Luckin, K.R. Koedinger, and J. Greer (Eds.), Artificial Intelligence in Education: Building Technology Rich Learning Contexts That Work (pp. 536–538). Amsterdam: IOS Press.

Stevens, R., Beal, C.R., and Sprang, M. (2013). Assessing Students' Problem Solving Ability and Cognitive Regulation with Learning Trajectories. In Roger Azvedo and Vincent Aleven (Eds.), International Handbook of Metacognition and Learning Technologies (pp. 409–423). New York: Springer.

Journal articles

Beal, C.R., Qu, L., and Lee, H. (2008). Mathematics Motivation and Achievement as Predictors of High School Students' Guessing and Help-Seeking With Instructional Software. Journal of Computer Assisted Learning, 24(6): 507–514.

Stevens, R.H., and Thadani, V. (2007). A Value-Based Approach for Quantifying Scientific Problem Solving Effectiveness. Journal of Technology, Instruction, Cognition and Learning, 5: 325–337.