Skip Navigation

icon of glasses and a book Cognition and Student Learning

Grantees

- OR -

Investigator

- OR -

Goals

- OR -

FY Awards

- OR -

Training in Experimental Design: Developing Scalable and Adaptive Computer-based Science Instruction

Year: 2006
Name of Institution:
Carnegie Mellon University
Goal: Development and Innovation
Principal Investigator:
Klahr, David
Award Amount: $1,485,318
Award Period: 3 years
Award Number: R305H060034

Description:

Purpose: Science is both a body of knowledge that represents our current understanding of natural systems and the processes whereby that body of knowledge has been established and is being continually extended, refined, and revised. Starting in the earliest school grades, science education comprises an extensive amount of domain specific knowledge (e.g., about physics, chemistry, biology) and a smaller but essential set of widely applicable procedures and modes of reasoning that are generally labeled as "scientific thinking." This research team argues that it is important for children to master two aspects of the scientific method: (a) the procedures for executing experiments, and (b) reasoning processes that support valid causal inferences. Both national and international assessments, and the extensive body of literature on children's understanding of experimental design, reveal not only that many students from very good schools do poorly on experimental design assessments, but also that there are substantial achievement gaps on these types of items between schools with large proportions of students from low income families and schools with students from predominantly middle and high income families. To improve student learning of scientific thinking in late elementary and middle school science, the research team is developing a computer-based intelligent tutoring system that will provide instruction in experimental design. This intelligent tutoring system provides feedback based on individual learners' knowledge and mastery in real time across a variety of tasks and science content areas.

Project Activities: The researchers are developing a computer-based intervention that can be delivered either in a one-on-one setting, or by a teacher in a whole-class setting, and will adapt to individual differences in order to promote transfer of knowledge. Computerized instructional modules include simulations, tracking of students' performance, and adaptive algorithms that provide feedback based on students' current actions and knowledge. Students who do not reach mastery in a particular module receive one-on-one instruction. The researchers will conduct studies in both classrooms and one-on-one tutoring sessions, and will determine what and how well the students learn, why some do not, and what revisions are likely to improve subsequent modules. Between 120 and 180 students in grades 5 through 8, representing six different schools, will participate in the study.

Products: The expected outcomes from this study include a computer-based science tutoring system and published reports.

Structured Abstract

Purpose: Science is both a body of knowledge that represents our current understanding of natural systems and the processes whereby that body of knowledge has been established and is being continually extended, refined, and revised. Starting in the earliest school grades, science education comprises an extensive amount of domain specific knowledge (e.g., about physics, chemistry, biology) and a smaller but essential set of widely applicable procedures and modes of reasoning that are generally labeled as "scientific thinking." This research team argues that it is important for children to master not only the procedures for executing experiments, but also the conceptual basis that enables one to design scientific experiments and make valid causal inferences. Both national and international assessments, and the extensive body of literature on children's understanding of experimental design, reveal not only that many students from very good schools do poorly on experimental design assessments, but also that there are substantial achievement gaps on these types of items between schools with large proportions of students from low income families and schools with students from predominantly middle and high income families. To improve student learning of scientific thinking in late elementary and middle school science, the research team is developing a computer-based intelligent tutoring system that will provide instruction in experimental design. This intelligent tutoring system provides feedback based on individual learners' knowledge and mastery in real time across a variety of tasks and science content areas.

Setting: The 6 urban parochial schools are in Pennsylvania.

Population: Students will come from two high-SES schools and four low-SES urban schools. In grades 5 through 8, these six schools provide 8 classrooms of students (between 120 to 180 students in total) in each grade level.

Intervention: Through a series of four increasingly adaptive instructional modules, the researchers will gradually develop the capability of the Intelligent TED Tutor to flexibly choose paths for each individual learner. Such individualization will enable each student to achieve both mastery and transfer goals. Broadly defined, an intelligent tutor is a computer-based instructional system containing an artificial intelligence (AI) component that identifies key instructional components, tracks students' knowledge, compares student performance to expert performance, and tailors multiple features of instruction to the student.

Research Design and Methods: In Modules 1 and 2, teacher-led instruction will occur for the whole class or small groups. Module 1 most closely resembles traditional teacher-led, whole-class instruction. The teacher will use a physical apparatus and a structured lesson plan to teach experimental design. Module 2 introduces a more flexible lesson plan and a computer simulation (rather than a physical apparatus) of a "virtual" experiment. In Modules 3 and 4, instruction moves to the individual level and becomes increasingly flexible, adaptive, and computer-based. The effectiveness of each module will be determined in classroom contexts and compared to the effectiveness of the module that came before it. Quasi-experimental, pretest/posttest, and non-equivalent group designs will be used with individual students (rather than classes) as the units of analyses. By comparing student learning via later modules with that of earlier modules, researchers will be able to determine whether later modules are, indeed, improvements on earlier ones.

Control Condition: Module 1 most closely resembles traditional teaching practices. The effectiveness of Module 1 in improving student learning will be compared to the effectiveness of the successive Modules 2 through 4.

Key Measures: Researchers will measure students' baseline performance on the training task and transfer measures, performance throughout instruction (i.e., formative assessment), and post-training performance on the training task immediately after instruction and transfer measures after a delay.

Data Analytic Strategy: To compare module effectiveness, general linear model analysis (GLM) is used including repeated measures ANOVA and ANCOVA. To evaluate the effectiveness of one-on-one, remedial tutoring for students below the mastery cutoff at post-test, regression discontinuity (RD) analysis is used.

Related IES Projects: From Cognitive Models of Reasoning to Lesson Plans for Inquiry (R305H030229) and Promoting Transfer of the Control of Variables Strategy in Elementary and Middle School Children via Contextual Framing and Abstraction (R305A100404)

Publications

Book chapter

Klahr, D. (2009). To Every Thing There is a Season, and a Time to Every Purpose Under the Heavens: What About Direct Instruction?. In S. Tobias, and T.M. Duffy (Eds.), Constructivist Theory Applied to Instruction: Success or Failure? (pp. 291–310). New York: Routledge.

Klahr, D., Triona, L., Strand-Cary, M., and Siler, S. (2008). Virtual vs. Physical Materials in Early Science Instruction: Transitioning to an Autonomous Tutor for Experimental Design. In J. Zumbach, N. Schwartz, T. Seufert, and L. Kester (Eds.), Beyond Knowledge: The Legacy of Competence Meaningful Computer-Based Learning Environments (pp. 163–172). New York: SpringerLink.

Siler, S.A., and Klahr, D. (2012). Detecting, Classifying and Remediating Children's Explicit and Implicit Misconceptions About Experimental Design. In R.W. Proctor, and E.J. Capaldi (Eds.), Psychology of Science: Implicit and Explicit Reasoning (pp. 137–182). New York: Oxford University Press.

Journal article, monograph, or newsletter

Siler, S.A., Klahr, D., and Price, N. (2013). Investigating the Mechanisms of Learning From a Constrained Preparation for Future Learning Activity. Instructional Science, 41(1): 191–216.

Strand-Cary, M., and Klahr, D. (2008). Developing Elementary Science Skills: Instructional Effectiveness and Path Independence. Cognitive Development, 23(4): 488–511.

Proceeding

Siler, S.A., Klahr, D., Magaro, C., Willows, K., and Mowery, D. (2010). Predictors of Transfer of Experimental Design Skills in Elementary and Middle School Children. In Proceedings of the 10th ITS 2010 Conference (pp. 198–208). Pittsburgh, PA: Springer-Verlag Berlin Heidelberg.

Siler, S.A., Mowery, Magaro, C., Willows, K., and D., Klahr, D. (2010). Comparison of a Computer-Based to a Hands-On Lesson in Experimental Design. In Proceedings of the 10th ITS 2010 Conference. Lecture Notes in Computer Science (pp. 408–410). Pittsburgh, PA: Springer-Verlag Berlin Heidelberg.