Structured Abstract
Setting
Middle and high schools in a school district in urban Southern California are participating.
Sample
Students in the participating school district are widely diverse in terms of ethnicity and socioeconomic status. The student body is 59 percent white, 30 percent Hispanic, 8 percent Asian, and 2 percent African American.
Intervention
The variables being examined are tested within the context of a web-based problem-solving simulation program called IMMEX (Interactive Multi-Media Exercises). Through IMMEX, students learn to frame a problem from a descriptive scenario, judge what information is relevant, plan a search strategy, gather information, and reach a decision that demonstrates understanding. Students' sequences of actions are used to model learning trajectories that can predict future performance. One objective is to use IMMEX data to identify students who are likely to persist with unproductive strategies. The IMMEX program is being modified in this project in order to integrate a pedagogical model into the system that will provide neutral feedback (namely feedback that provides general encouragement but does not provide specific instructions about what students should do next in order to facilitate learning) or individualized feedback that explicitly addresses how the student is approaching the problem. In addition, the program will deliver feedback either via simple text messages or interactive, animated pedagogical agents. Students work with IMMEX in their science classes under the direction of their classroom instructor.
Research design and methods
Students will be randomly assigned to work with different versions of the web-based IMMEX problem solving simulations, created to evaluate the role of neutral vs. individualized feedback, text vs. pedagogical agent delivery, and agent characteristics (such as gender, ethnicity, perceived scientific credibility and authority).
Control condition
The control condition is the classic IMMEX program in which students are provided with multiple opportunities to practice solving related problems but do not receive explicit feedback as they work through the problem sets.
Key measures
Individual student problem solving sequences are automatically recorded as students work on the IMMEX system. In addition, the school district collects achievement data that is being used both as a covariate, and as a way to measure the impact of the intervention on achievement.
Data analytic strategy
Both analysis of covariance and hierarchical linear modeling are being used to examine student performance in the different versions of IMMEX.
People and institutions involved
IES program contact(s)
Products and publications
ERIC Citations: Find available citations in ERIC for this award here.
Select Publications
Book chapters
Beal, C.R., Shaw, E., and Birch, M. (2007). Intelligent Tutoring and Human Tutoring in Small Groups: An Empirical Comparison. In R. Luckin, K.R. Koedinger, and J. Greer (Eds.), Artificial Intelligence in Education: Building Technology Rich Learning Contexts That Work (pp. 536-538). Amsterdam: IOS Press.
Stevens, R., Beal, C.R., and Sprang, M. (2013). Assessing Students' Problem Solving Ability and Cognitive Regulation with Learning Trajectories. In Roger Azvedo and Vincent Aleven (Eds.), International Handbook of Metacognition and Learning Technologies (pp. 409-423). New York: Springer.
Journal articles
Beal, C.R., Qu, L., and Lee, H. (2008). Mathematics Motivation and Achievement as Predictors of High School Students' Guessing and Help-Seeking With Instructional Software. Journal of Computer Assisted Learning, 24(6): 507-514.
Stevens, R.H., and Thadani, V. (2007). A Value-Based Approach for Quantifying Scientific Problem Solving Effectiveness. Journal of Technology, Instruction, Cognition and Learning, 5: 325-337.
Supplemental information
In phase 1 of the project, learning outcomes are being compared for students who work with (1) classic IMMEX (overall control), (2) a version of IMMEX that includes individualized pedagogical feedback on strategy use in the form of text messages (experimental), and (3) a version of IMMEX in which students receive neutral feedback through text messages (feedback control). In phase 2, the researchers are comparing the learning trajectories of students who work with IMMEX in four versions, representing a two-by-two design, in which students receive neutral or individualized feedback, through either text or animated agents. In phase 3, the researchers are investigating the specific characteristics of agents that are most effective for groups of students.
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
