|Title:||Dynamically Modifying the Learning Trajectories of Novices with Pedagogical Agents|
|Principal Investigator:||Beal, Carole||Awardee:||University of Southern California|
|Program:||Cognition and Student Learning [Program Details]|
|Award Period:||3 years||Award Amount:||$1,220,822|
|Type:||Development and Innovation||Award Number:||R305H050052|
Purpose: To improve students' science learning, it is important to understand how students approach and solve scientific problems, and how scientific reasoning develops. The idea of "learning trajectories" is useful when thinking about such development. This idea is based on the observation that novices and experts think and perform differently and can be viewed as demonstrating stages of understanding as experience is developed. Initially, when knowledge is limited and fragmented, it can be difficult for students to understand or frame problems. Problem solving at this stage is often neither effective nor efficient. With practice, the student's knowledge becomes deeper and more structured, which may allow problem solving to become more strategic. Eventually, most students adopt an approach they are comfortable with. The challenge for science education is that once students settle on a strategy, they will often continue to use the same approach in the future, even if the approach is not effective or efficient. The purpose of this project is to develop an intervention designed to help novice chemistry learners revise their strategy while solving chemistry problems. Building on prior research, this team is developing predictive models of problem solving and using them to provide individualized instruction to novice chemistry learners.
Setting: Middle and high schools in a school district in urban Southern California are participating.
Population: Students in the participating school district are widely diverse in terms of ethnicity and socioeconomic status. The student body is 59 percent white, 30 percent Hispanic, 8 percent Asian, and 2 percent African American.
Intervention: The variables being examined are tested within the context of a web-based problem solving simulation program called IMMEX (Interactive Multi-Media Exercises). Through IMMEX, students learn to frame a problem from a descriptive scenario, judge what information is relevant, plan a search strategy, gather information, and reach a decision that demonstrates understanding. Students' sequences of actions are used to model learning trajectories that can predict future performance. One objective is to use IMMEX data to identify students who are likely to persist with unproductive strategies. The IMMEX program is being modified in this project in order to integrate a pedagogical model into the system that will provide neutral feedback (i.e., feedback that provides general encouragement, but does not provide specific instructions about what students should do next in order to facilitate learning) or individualized feedback that explicitly addresses how the student is approaching the problem. In addition, the program will deliver feedback either via simple text messages or interactive, animated pedagogical agents. Students work with IMMEX in their science classes under the direction of their classroom instructor.
Research Design and Methods: Students will be randomly assigned to work with different versions of the web-based IMMEX problem solving simulations, created to evaluate the role of neutral vs. individualized feedback, text vs. pedagogical agent delivery, and agent characteristics (e.g., gender, ethnicity, perceived scientific credibility and authority).
In Year 1 of the project, learning outcomes are being compared for students who work with (1) classic IMMEX (overall control), (2) a version of IMMEX that includes individualized pedagogical feedback on strategy use in the form of text messages (experimental), and (3) a version of IMMEX in which students receive neutral feedback through text messages (feedback control). In Year 2, the researchers are comparing the learning trajectories of students who work with IMMEX in four versions, representing a two-by-two design, in which students receive neutral or individualized feedback, through either text or animated agents. In Year 3, the researchers are investigating the specific characteristics of agents that are most effective for groups of students.
Control Condition: The control condition is the classic IMMEX program in which students are provided with multiple opportunities to practice solving related problems but do not receive explicit feedback as they work through the problem sets.
Key Measures: Individual student problem solving sequences are automatically recorded as students work on the IMMEX system. In addition, the school district collects achievement data that is being used both as a covariate, and as a way to measure the impact of the intervention on achievement.
Data Analytic Strategy: Both analysis of covariance and hierarchical linear modeling are being used to examine student performance in the different versions of IMMEX.
Project website: http://k12.usc.edu.
Beal, C.R., Shaw, E., and Birch, M. (2007). Intelligent Tutoring and Human Tutoring in Small Groups: An Empirical Comparison. In R. Luckin, K.R. Koedinger, and J. Greer (Eds.), Artificial Intelligence in Education: Building Technology Rich Learning Contexts That Work (pp. 536–538). Amsterdam: IOS Press.
Stevens, R., Beal, C.R., and Sprang, M. (2013). Assessing Students' Problem Solving Ability and Cognitive Regulation with Learning Trajectories. In Roger Azvedo and Vincent Aleven (Eds.), International Handbook of Metacognition and Learning Technologies (pp. 409–423). New York: Springer.
Journal article, monograph, or newsletter
Beal, C.R., Qu, L., and Lee, H. (2008). Mathematics Motivation and Achievement as Predictors of High School Students' Guessing and Help-Seeking With Instructional Software. Journal of Computer Assisted Learning, 24(6): 507–514.
Stevens, R.H., and Thadani, V. (2007). A Value-Based Approach for Quantifying Scientific Problem Solving Effectiveness. Journal of Technology, Instruction, Cognition and Learning, 5: 325–337.