Co-Principal Investigators: Yaron, David

Purpose: Test scores on the most recent National Assessment of Educational Progress (NAPE) suggest that nearly half of 12th grade students fail to reach basic proficiency in science. Current high school chemistry curricula focuses on quantitative problem-solving activities that are rarely presented in meaningful contexts and do not support the development of deep conceptual understanding of chemical principals or scientific inquiry skills that promote proficiency in science. To address this need, the researchers will develop Virtual Lab-based activities with embedded assessments that allow students to engage in authentic chemistry investigations while getting personalized coaching. These activities are expected to lead to improvements in students' science achievement through a deeper understanding of chemical principles and their ability ability to plan and conduct laboratory investigations.

Project Activities: The previously developed ChemCollective Virtual Lab environment allows students to flexibly apply content knowledge to realistic scenarios, but currently lacks assessment, coaching, and reporting components to monitor and support learning. Using the existing ChemCollective Virtual Lab environment as a foundation, the researchers will develop dynamic Virtual Lab activities with embedded assessments that are aligned with state and national standards in the topics of Stoichiometry and Thermochemistry. An iterative cycle of development, testing, and refinement will ensure usability and feasibility of the activities.

Products: The products of this project will include fully developed on-line modules that allow students to develop and demonstrate content and inquiry skills in Stoichiometry and Thermochemistry. Additional products will include published reports.

Structured Abstract

Setting: The setting for this study includes urban high schools in California.

Population: Approximately 600 students from four high schools will participate in the study. The targeted schools serve a high percentage of minority students, students in poverty, and English Language Learners.

Intervention: The intervention will consist of two on-line modules focusing on Stoichiometry and Thermochemistry. These modules will allow students to develop and demonstrate content and inquiry skills using the ChemCollective Virtual Lab simulation environment. Each of the modules will consist of four 40-minute Virtual Lab activities with embedded assessments that address subtopics aligned with key state and National Science Education Standards for content and inquiry. The goal of the Virtual Lab activities is not to replace physical labs, but rather to give students additional opportunities to connect procedural practice with chemical practice and to provide teachers with information about individual students that they can use to monitor student progress and guide instruction.

Research Design and Methods: The researchers will develop four 40-minute Virtual Lab activities focusing on Stoichiometry and Thermochemistry. Two rounds of usability and classroom feasibility studies will be conducted. Prior to each round of usability and feasibility testing, an advisory board will be convened to review and provide feedback on the modules. For each module, student usability will be tested by conducting cognitive labs with six high school students using think-aloud interview protocols. Teacher usability will also be tested with four high school chemistry teachers. The feedback from the teachers and students from the cognitive lab usability studies will provide an early indication of whether the instructional modules are engaging to students and are promoting science knowledge. In addition, to ensure that the modules are feasible for teachers and students to use in actual classrooms, the researchers will conduct classroom feasibility studies on each module with four teachers and their students. Following revisions based on the findings from the classroom feasibility studies, the researchers will also conduct a pilot study to assess the promise of the two modules with a sample of 12 high school chemistry teachers and their students.

Control Condition: There is no control condition.

Key Measures: Data collection will include feedback from teachers and students, student responses on assessment items (pre- and post-test of items from the American Chemistry Society Examination; classroom exams), log data on student action in the Virtual Lab environment, instructional logs, teacher questionnaires, and measures of student motivation and engagement.

Data Analytic Strategy: The promise of the intervention will be analyzed through descriptive statistics, and correlational analyses. The embedded assessments will be evaluated using classical and item response theory analyses.

Products and Publications

Book chapter

Quellmalz, E.S., Silberglitt, M.D., Buckley, B.C., Loveland, M.T., and Brenner, D. (2016). Simulations for Assessing and Supporting Science Literacy. In Y. Rosen, S. Ferrara, and M. Mosharraf (Eds.), Handbook of Research on Computational Tools for Real-World Skill Development (pp. 191–229). Hershey, PA: IGI Global.

Journal article, monograph, or newsletter

Davenport, J.L.,Leinhardt, G., Greeno, J., Koedinger, K., Klahr, D., Karabinos, M., and Yaron, D.J. (2014). Evidence-Based Approaches to Improving Chemical Equilibrium Instruction. Journal of Chemical Education, 91 (10): 1517–1525.

Proceeding

Crossley, S., Kyle, K., Davenport, J., and McNamara, D.S (2016). Automatic Assessment of Constructed Response Data in a Chemistry Tutor. In Proceedings of the 9th International Conference on Educational Data Mining .

Davenport, J.L., Raffety, A., Timms, M.J., Yaron, D., and Karabinos, M. (2012). ChemVLab+: Evaluating a Virtual Lab Tutor for High School Chemistry. In Proceedings of the 2012 International Conference of the Learning Sciences (pp. 381–385). Sydney, Australia: The University of Sydney.

Rafferty, A.N., Davenport, J.L., and Brunskill, E. (2013). Estimating Student Knowledge From Paired Interaction Data. In Proceedings of the 6th International Conference on Educational Data Mining (EDM 2013) (pp. 260–263). Memphis, TN: International Educational Data Mining Society.