ASSISTment Meets Science Learning (AMSL)
Co-Principal Investigator(s): Neil Heffernan, Joseph Beck, and Kenneth Koedinger
Purpose: For the 21st century workplace, students need to understand science more deeply and possess well-honed learning strategies that will allow them to apply their science knowledge in more flexible ways. By providing students with frequent, fine-grained performance assessments of science process skills, teaching and tutoring can be honed to students' individual needs. To support the acquisition of flexible science knowledge, researchers in this project will develop a computer-based intelligent tutoring system designed to tutor middle school students in science inquiry and process skills such as collecting and interpreting data, prediction and hypothesis making, experimenting with interactive models, mathematizing the data, and defending and communicating a scientific argument.
Project Activities: The researchers will develop modules to support the instruction and assessment of science process skills focusing on Earth and Life Science, aligned with content from the Massachusetts state curriculum framework. The modules will rely heavily on the use of microworlds (i.e., computer-based manipulative models) to engage, hone, and assess students' science inquiry skills. The modules will be iteratively developed, tested, and refined.
Products: The products of this project will be fully developed modules designed to assess science process skills in Earth and Life Science, and published reports describing the developed modules.
Setting: The setting for this study includes urban and suburban public middle schools in Massachusetts.
Population: Middle school students and their teachers from three schools in Massachusetts will participate in the study. The student population is ethnically diverse and contains a high percentage of students from low-income backgrounds.
Intervention: The researchers will develop new modules that address the Massachusetts Comprehensive Assessment System (MCAS) framework for middle school Earth and Life Science. Specifically, materials in the modules will be developed for the following science strands: Earth in the Solar System; Classification of Organisms; Structures and Functions of Cells; and Systems of Living Things. The modules will rely heavily on the use of microworlds to tutor students on science process skills needed to conduct inquiry. In the microworld, students are presented with a scenario and then asked to make predictions and answer questions about that scenario. The ASSISTment tutoring system, which blends student tutoring and assessment, is integrated into the microworld modules so that support and feedback is provided to students when a question is answered incorrectly and the student is provided scaffolds to lead them to the correct solution.
Research Design and Methods: Building upon prior development of the ASSISTments System for Math, the research team and content experts will build microworlds whose content and skills align with the MCAS Science standards. The microworlds will be piloted with students and their teachers. Think aloud protocols will be collected from a random sample of students within classrooms. Students and teachers will be interviewed and videotaped interacting with the microworlds. Computer log files will also be collected to provide fine-grained data that will be used to determine how students are using the models to conduct scientific inquiry. Analysis of these data will inform revisions of the ASSISTment for Science microworlds, tasks, and scaffolds. The researchers will conduct a series of experiments to examine the potential efficacy of the program for improving student science learning. In each of the experiments, students will be randomly assigned to treatment (microworld modules with tutoring support and feedback) and control (microworld modules without tutoring support and feedback) conditions. The first experiment focuses on whether inquiry tutoring leads to improved inquiry skills in Earth and Life Science. The second experiment focuses on whether the ASSISTment tutoring leads to increased science content learning, as measured by MCAS scores. Finally, the third experiment will focus on whether students' skills differ or change as they move on to a new content area, from Life Science to Earth Science or vice versa.
Control Condition: To assess the promise of the program, students assigned to the control condition will receive the Earth and Life Science microworld modules without the ASSISTment tutoring support and feedback system.
Key Measures: To determine the usability and feasibility of modules, students and teachers will be interviewed and videotaped interacting with the microworlds. Computer log files will also be collected to provide fine-grained data that will be used to determine how students are using the models to conduct scientific inquiry. To assess the potential efficacy of the intervention, pre- and post-tests of students' skills for each of the five inquiry strands—collecting data, interpreting data, predicting/hypothesizing, mathematizing data, and designing/conducting experiments—will be developed by the researchers. In addition, students' scores on science items for the MCAS test will be collected.
Data Analytic Strategy: The researchers will analyze the computer log files in order to fine tune and evaluate students' inquiry performance in each of the inquiry strands. Repeated measures multivariate analysis of variance will be used to analyze student learning gains on the pre- and post-tests of the five inquiry strands, and MCAS items for science. Learning factors analysis will be conducted to identify where the students switched content areas, as indicated by a sudden increase in haphazard inquiry behaviors, and learning decomposition analysis to estimate the relative efficacy of different types of practice on learning a skill.
Related IES Projects: The Development of an Intelligent Pedagogical Agent for Physical Science Inquiry Driven by Educational Data Mining (R305A120778)
Journal article, monograph, or newsletter
Pedro, M., Baker, R.D., Gobert, J.D., Montalvo, O., and Nakama, A. (2013). Leveraging Machine-Learned Detectors of Systematic Inquiry Behavior to Estimate and Predict Transfer of Inquiry Skill. User Modeling and User-Adapted Interaction, 23(1): 1–39.