Skip Navigation
Funding Opportunities | Search Funded Research Grants and Contracts

IES Grant

Title: A Theory-Driven Search for the Optimal Conditions of Instructional Guidance in Algebra Tutor
Center: NCER Year: 2010
Principal Investigator: Anderson, John Awardee: Carnegie Mellon University
Program: Cognition and Student Learning      [Program Details]
Award Period: 4 years (3/1/2010-2/28/2014) Award Amount: $1,542,658
Goal: Exploration Award Number: R305A100109

Co-Principal Investigator: Steve Ritter, Carnegie Learning

Purpose: The purpose of this research is to better understand the cognitive processes of individual students as they solve algebra problems in the context of the Carnegie Learning Algebra Tutor software. The researchers will use computer simulations of individual students (synthetic student models) to find the optimal conditions of instructional guidance a student would need to receive to master an algebra problem. The team will explore whether techniques such as direct instruction or guided discovery learning and feedback are related to improving learning outcomes. These synthetic student models will interact with computer-based tutoring systems and predict learning outcomes of various instructional conditions. The predictions of the synthetic student models will guide the researchers to develop and test an intervention with future funding.

Project Activities: Building on prior computational synthetic student models embedded in the Adaptive Control of Thought-Rational cognitive architecture, researchers in this project will carry out three subprojects intended to extend and gather information to improve these models. In the first subproject, researchers will observe eye movements, administer verbal protocols, and use functional Magnetic Resonance Imaging in a batter of empirical laboratory studies to explore detailed information about the processes of cognition in human students. Once a valid synthetic student model has been found for an individual learner, hundreds of possible combinations of parameter settings in the tutor will be tested rapidly through automated, interactive simulations to find the optimal settings. A population of synthetic student models will then be developed to reflect differences among students. To validate the laboratory studies, predictions based on these models will be tested in classrooms. The research will culminate in a detailed validation study of real students learning in the condition identified as optimal by the synthetic student model.

Products: Products from this project include published reports and a modified synthetic student model that will be used to improve the correspondence between the synthetic student model's behavior and real student behavior.

Structured Abstract

Setting: The research will take place both in the laboratory and in classrooms that use the Carnegie Learning Algebra I tutor in rural and urban western Pennsylvania and Kentucky.

Sample: Students will be seventh to ninth graders enrolled in Algebra I or who will enroll in Algebra 1 in the following year. Approximately 70 percent of participants will be White and 30 percent of participants will be African-American, with approximately 50 percent of students eligible for free or reduced lunch. Across the participating schools researchers will recruit 6 teachers, each instructing at least two classes. Students within classrooms will be randomly assigned to a condition, providing approximately 120–150 students per condition in at least 12 classrooms.

Intervention: This research will be conducted within the context of the Algebra 1 curriculum as delivered by the Carnegie Learning Algebra Tutor. Carnegie Learning Algebra 1 is designed as a first-year Algebra course for core instruction. It is an adaptive math software that can be implemented with students at a variety of ability and grade levels.

Research Methods and Design: The core research methodology involves computer simulation in which one computer system (the synthetic student model) interacts with another (the Carnegie Learning Algebra Tutor). In a series of three subprojects, the synthetic student model will first be tested in a battery of empirical laboratory studies by its ability to predict a rich pattern of data—classroom performance, eye movements, verbal protocols, and functional Magnetic Resonance Imaging (fMRI). This data will be obtained from real students interacting with the Carnegie Learning Algebra Tutor under different conditions. A population of synthetic student models will then be developed to reflect differences among students. To validate the laboratory studies, predictions based on these models will be tested in classrooms. The research will culminate in a detailed validation study of real students learning in the condition identified as optimal by the synthetic student model.

Control Condition: Conditions vary across each of the experiments and are appropriate to the questions addressed. Experimental manipulations include type of instruction (i.e. direct vs. guided discovery); timing of feedback (i.e. delayed vs. immediate); and type of background instruction given (enhanced vs. minimal) before students begin working with the tutor. Other variables include self-explanation (where students either do or do not explain the algebraic principles); transformation mode (where either the student or the tutor types in the algebraic expression); term complexity (where the student is given either simple integers or complex rationale fractions); spacing; and amount of practice.

Key Measures: To facilitate development of accurate synthetic student models, multiple dependent measures will be collected as students perform algebra problems, including response times, accuracy, subject reports (protocols), eye movements, and fMRI data. Both proximal measures (performance within the tutor) and distal measures (end of semester tests given outside of the tutor and comprised of items from the New York's Regents Test) will be used. A six-month retention test will be given to test the prediction about differential retention in the direct instruction vs. discovery learning conditions.

Data Analytic Strategy: Alternative versions of a synthetic student model will be compared using the Bayesian Information Criterion. Statistical tests, such as mixed-model analysis of covariance, will be run to identify the strength of the relationship between variables.

Related IES Projects: The Neural Markers of Effective Learning (R305H030016)


Journal article, monograph, or newsletter

Anderson, J.R., Lee, H.S., and Fincham, J.M. (2014). Discovering the Structure of Mathematical Problem Solving. Neuroimage, 97(15): 163–177.

Lee, H.S., and Anderson, J.R. (2013). Student Learning: What has Instruction got to do With it?. Annual Review of Psychology, 64: 445–469.

Lee, H.S., Betts, S., and Anderson, J.R. (2015). Not Taking the Easy Road: When Similarity Hurts Learning. Memory and Cognition, 43(6): 939–952.

Lee, H.S., Betts, S., and Anderson, J.R. (2016). Learning Problem-Solving Rules as Search Through a Hypothesis Space. Cognitive Science, 40(5): 1036–1079.

Lee, H.S., Fincham, J.M., and Anderson, J.R. (2015). Learning From Examples Versus Verbal Directions in Mathematical Problem Solving. Mind, Brain, and Education, 9(4): 232–245.

Lee, H.S., Fincham, J.M., Betts, S., and Anderson, J.R. (2014). An fMRI Investigation of Instructional Guidance in Mathematical Problem Solving. Trends in Neuroscience and Education, 3(2): 50–60.

Seung Lee, H., Betts, S., and Anderson, J.R. (2017). Embellishing Problem-Solving Examples with Deep Structure Information Facilitates Transfer. Journal of Experimental Education.


Lee, H.S., Anderson, A., Betts, S., and Anderson, J.R. (2011). When Does Provision of Instruction Promote Learning?. In L. Carlson, C. Hoelscher, and T. Shipley (Eds.), Proceedings of the 33rd Annual Conference of the Cognitive Science Society (pp. 3518–3523). Austin, TX: Cognitive Science Society.