Purpose: In this project, the researchers created and evaluated supplemental intelligent tutoring software (ITS) for supporting student learning outcomes in grade 6 mathematics. The goal of this software, called AnimalWatch, was to improve mathematics skill and achievement, particularly for students from groups that have been traditionally under-represented in science and engineering fields. AnimalWatch software was to integrate mathematics problem solving and multimedia instruction with information about environmental science and endangered species.

Structured Abstract

THE FOLLOWING CONTENT DESCRIBES THE PROJECT AT THE TIME OF FUNDING

Setting: Three evaluation studies will be conducted in a large school district in southern California. Approximately 30 classrooms will participate in each of the studies.

Population: The Pasadena Unified School District serves approximately 23,447 students (47 percent Hispanic, 32 percent African American, and 20 percent White). Over 66 percent of students participate in the free or reduced-price lunch program. About one third of students are limited-English proficient (of these, 92 percent are Spanish speaking). In 2004, 72 percent of grade 6 students received scores in the "Below Proficient" category on the mathematics section of the California Standards Test.

Intervention: The AnimalWatch software provides multimedia instruction in grade 6 arithmetic topics, covering simple addition through unlike fractions and mixed numbers. When students log on to AnimalWatch, they adopt the role of a wildlife biologist charged with learning about and monitoring a particular endangered species. A major objective of this project is to evaluate the hypothesis that individualized supplemental software instruction-instruction that uses a model of the learner's cognitive and motivational characteristics-will yield stronger student motivation and learning outcomes than non-individualized (generic) software instruction, or no supplemental software at all.

Research Design and Methods: Researchers will conduct an evaluation study in each of the three years of the project (in addition to pilot studies to test functionality of new AnimalWatch features as they are developed). In each study, the researchers will include 8-10 classrooms in each of three conditions. Classrooms will be randomly assigned either to a software intervention (two conditions: individualized or generic) or the control condition (no supplemental software) with the constraint that schools are equally represented across condition. Students in the two software conditions will be randomly assigned by the server computer to one of the two AnimalWatch versions at the time of the first log-in and will continue to work with the same software version for the entire intervention. All students, including those in the control condition, will come to the computer lab in their school for a pre-test session and a post-test session. Students in the two software conditions will then work with AnimalWatch in twice-weekly visits to the computer laboratory under the supervision of their classroom teacher.

Control Condition: Students in the control condition will continue with their regular classroom activities other than the pre and posttest periods.

Key Measures: To provide evidence of impact on achievement, mock tests created from CAT/6 and STAR achievement test items will be employed. In addition to performance on mathematics tests, the researchers will utilize data about student performance and behavior while working on the software as another measure of learning. Moreover, several motivational and attitudinal measures will also be included.

Data Analytic Strategy: Data analyses in the AnimalWatch project will include traditional ANOVA comparisons, and HLM to investigate the effects of the intervention in relation to teacher, classroom, and school outcomes.

Related IES Projects: Efficacy Study of AnimalWatch: An Intelligent Tutoring System for Pre-Algebra (R305A090197)

Products and Publications

ERIC Citations: Find available citations in ERIC for this award here.

Select Publications:

Book chapters

Beal, C.R., and Cohen, P. (2005). Computational Methods for Evaluating Student and Group Learning Histories in Intelligent Tutoring Systems. In C.K. Looi, G. McCalla, B. Bredeweg, and J. Breuker (Eds.), Artificial Intelligence in Education: Supporting Learning Through Intelligent and Socially Informed Technology (pp. 80–88). Amsterdam, Netherlands: IOS Press.

Beal, C.R., Shaw, E., and Birch, M. (2007). Intelligent Tutoring and Human Tutoring in Small Groups: An Empirical Comparison. In R. Luckin, K.R. Koedinger, and J. Greer (Eds.), Artificial Intelligence in Education: Building Technology Rich Learning Contexts That Work(pp. 536–538). Amsterdam: IOS Press.

Journal articles

Arroyo, I., Woolf, B.P., and Beal, C.R. (2006). Addressing Cognitive Differences and Gender During Problem Solving. International Journal of Technology, Instruction, Cognition and Learning, 4: 31–63.

Beal, C.R., Adams, N., and Cohen, P.R. (2010). Reading Proficiency and Mathematics Problem Solving by High School English Language Learners. Urban Education, 45(1): 58–74.

Beal, C.R., Arroyo, I.M., Cohen, P.R., and Woolf, B.P. (2010). Evaluation of AnimalWatch: An Intelligent Tutoring System for Arithmetic and Fractions. Journal of Interactive Online Learning, 9(1): 64–77.

Cohen, P.R., and Beal, C.R. (2008). Temporal Data Mining for Educational Applications: Lecture Notes. Computer Science, 5351: 66–77.

Woolf, B.P., Arroyo, I., Beal, C.R., and Murray, T. (2006). Gender and Cognitive Differences in Help Effectiveness During Problem Solving. International Journal of Technology, Instruction, Cognition and Learning, 3: 89–95.

Proceedings

Birch, M., and Beal, C.R. (2008). Problem Posing in AnimalWatch: An Interactive System for Student-Authored Content. In Proceedings of the 21st International Florida Artificial Intelligence Research Society Conference (pp. 397–402). Palo Alto, CA: Association for the Advancement of Artificial Intelligence Press.

Cohen, P.R., Beal, C.R., and Adams, N. (2008). The Design, Deployment and Evaluation of the AnimalWatch Intelligent Tutoring System. In Proceedings of the 2008 Conference on ECAI 2008: 18th European Conference on Artificial Intelligence (pp. 663–667). IOS Press: Amsterdam.