Co-Principal Investigators: Mott, Bradford; Feng, Mingyu; Ringstaff, Cathy

Purpose: This project will focus on the development and pilot testing of Science Sketch, a sketch-based science learning environment to support upper elementary students' conceptual understanding of science with an emphasis on modeling. To develop an understanding of core ideas in science, students must develop models to explain their observations, share models within their classroom community, test their models in investigations, and revise their models based on evidence. Drawing is a promising approach to supporting model development, use, and refinement, but it is underused as a learning tool in current education practice. The field is now well positioned to leverage AI technologies to create learning-by-drawing environments that will provide real-time formative assessment that is actionable by students and teachers in the classroom, thereby improving science learning.

Project Activities: During Years 1–3 of the project, the research team will iteratively develop Science Sketch. The research team will conduct focus groups, semi-structured interviews, and classroom observations of students using Science Sketch to inform the development process. In the final year of the project, the research team will conduct a pilot study to test the promise of Science Sketch for improving students' science learning.

Products: This project will result in a fully developed Science Sketch learning environment and its associated sketch-based modeling activities for Grade 4 science education.   The research team will disseminate findings from the pilot study through reports and peer-reviewed publications to researchers, practitioners, and the general public.

Structured Abstract

Setting: The research will take place in 4th grade classrooms within public elementary schools in North Carolina and California.

Sample: In Years 1, 2, and 3, the sample will consist of 4 elementary school science teachers and 16 students who will participate in usability testing for Science Sketch as it is developed. In Years 2–3, 4 elementary school science teachers and 100 4th grade students will participate in two feasibility studies to help inform iterative development. In Year 4, 480 students will participate in the pilot study.

Intervention:  Science Sketch is a sketch-based science learning environment designed to support upper elementary students' conceptual understanding of science with an emphasis on modeling. Science Sketch will enable students to develop, use, and refine models of science phenomena through sketch-based interactions. As students sketch their models, Science Sketch will track their sketching in real time and provide support via feedback, hints, and explanations. Science Sketch will include modules and associated sketch-based learning activities for four Grade 4 units: Conservation of Energy and Energy Transfer, Wave Properties, Electromagnetic Radiation, and Information Processing, and their associated crosscutting concepts (Energy and Matter, Patterns, Cause and Effect, and Systems and System Models, respectively).

Research Design and Methods: In Years 1-3, the research team will develop and iteratively refine the Science Sketch learning environment. The team will conduct focus groups with science teachers and students and collect data through classroom observations, interviews, and surveys. In Year 4, the team will conduct the pilot study to establish evidence of promise for enhancing science learning. The pilot study will use a cluster-randomized control trial design, randomly assigning teachers to use Science Sketch for sketching and modeling activities in their classrooms or to the control condition.

Control Condition: There is no control condition for the usability and feasibility studies. For the pilot study, the students assigned to the control condition will continue receiving regular classroom science instruction on the same topics as the treatment condition.

Key Measures: The main outcome measure is a researcher-developed assessment of students' conceptual understanding and modeling competencies. To inform the development of Science Sketch and to collect information on usability, feasibility, and fidelity of implementation, the research team will collect teacher instructional logs and surveys, qualitative data from interviews, Science Sketch system logs, and classroom observations.

Data Analytic Strategy The research team will analyze qualitative development data using inductive coding. For the pilot study, the research team will use hierarchical linear modeling to examine mean differences in students' post-test performance on assessments of conceptual understanding and modeling competencies between students in the treatment condition (Science Sketch) and the control group (business-as-usual), controlling for pre-test scores and other covariates.

Cost Analysis: The research team will gather cost data systematically throughout the study. The research team will use the ingredients method to gather costs systemically and estimate total costs of implementing Science Sketch during the experiment. They will also estimate incremental costs per student above and beyond the control condition.