Co-Principal Investigators: Meyer, Richard; Homer, Bruce; Verkuilen, Jay

Purpose: The purpose of this project is to systematically investigate how specific features of immersive virtual reality (IVR) can be used to improve student outcomes in science learning. In particular, the researchers will examine the effectiveness of visual and auditory IVR design features in enhancing learners' affective state and cognitive processing for learners in general and for specific subgroups of learners. IVR has the potential to cause a feeling of presence, emotional arousal, and situational interest in learners. IVR may also affect learners' attention and cognitive load. The goal of the project is to contribute to a theory of learning with media that includes how affective processing interacts with cognitive processing, leading to improvements in learning outcomes. With the decrease in cost and increase in power of off-the-shelf virtual reality (VR) systems, there is growing interest in using this emerging technology in education, and during the pandemic, VR emerged as a safe way of learning and collaborating online. However, there is a need for research leading to evidence-based principles on how to design academic instruction in immersive VR.

Project Activities: The researchers will design IVR tasks for use with a virtual reality headset based on existing IVR science simulations on astronomy (Cosmos) and cell biology (Cells). The researchers will conduct a systematic series of five experiments aimed at pinpointing visual and auditory design features that improve science learning in IVR. They will focus on teenage learners, with a special focus on under-represented minorized groups, economically disadvantaged students, and English language learners.

Products: The major products of this project will be a set of evidence-based principles for the design and evaluation of academic lessons in IVR as well as contributions to an affective-cognitive theory of learning with media, such as peer-reviewed publications.

Structured Abstract

Setting: Researchers will conduct value added and media comparison studies in middle schools and high schools in New York and New Jersey and university classrooms in California. Affective quality studies will take place in laboratory settings.

Sample: Researchers will recruit participants from ethnically and economically diverse middle and high schools (17 percent White, 15 percent Black, 55 percent Hispanic (any race), and 10 percent Asian students; 85 percent of all students qualify for free lunch) and from the University of California, Santa Barbara, which has a diverse student population (over 50 percent Hispanic), sampling only first-year undergraduates in the Department of Psychological and Brain Sciences.

Factors: In studies 1 through 4, researchers will examine how visual (for example the presence of an avatar) and auditory (such as the use of spatial audio) design features impact students' affective (namely sense of presence, affective arousal, and situational interest) and cognitive processing during science learning in an IVR simulation. In study 5, the researchers will measure student affective and cognitive outcomes in IVR as compared to a non-immersive two-dimensional simulation. Throughout the five studies the team will explore whether these effects differ between learners from different genders, ethnic minority groups, English language proficiency, or socioeconomic status.

Research Design and Methods: The researchers will employ experimental designs with control groups and random assignments to treatment conditions. They will explore a series of questions focused on three types of studies:

1. Affective Quality question: What is the affective quality of the specific design feature of IVR, i.e., what is their effect on learners' affective arousal?
2. Value Added question: Which of the features that were found to elicit high affective arousal also increase the educational effectiveness of the IVR, i.e., what is the effect on learning outcomes of adding emotion-arousing features to an IVR lesson?
3. Media Comparison question: Do IVR materials with features that were found to have higher affective arousal and improved learning outcomes result in better learning outcomes compared to analogous two-dimensional (2D) simulations?

Researchers will conduct two sets of affective quality studies (visual and auditory) each of which employing a within-subject design. The value-added research studies will involve a 2 x 2 experimental design to test the effects of the presence or absence of the two design features in each modality. The media comparison studies will utilize a 2 x 2 design with immersion (IVR vs. 2D) and affective features (emotional v. neutral design) as the factors.

Control Condition: In the affective quality and value-added studies, control conditions will consist of variants of the IVR tasks that do not include the design features under investigation. In the media comparison studies, control conditions will consist of the same IVR tasks presented on a lower-immersion two-dimensional medium such as a tablet or computer screen.

Key Measures: Researchers will measure affective arousal during learning, including physiological measures of emotion based on heart rate and heart rate variability using a wristband monitor and through self-report measures of emotion using the Positive and Negative Affect Schedule (PANAS). They also will measure presence and situational interest by validated self-report surveys. To measure cognitive processing during learning, the researchers will use user log files to assess attentional processes during learning and validated self-report items to assess cognitive load during learning and emotional self-regulation. Given that the researchers' goal is to use the affective power of IVR to promote academic learning, they also assess learning outcomes in the form of retention and transfer, measured by experimenter-constructed tests (based on New York state test items) to be validated by science teachers and via pilot testing that will include multiple-choice and open-ended items. They also will measure interest in science through a validated survey.

Data Analytic Strategy: The research team will use various models of analysis of variance, including multivariate designs, covariance designs, and structural equation models. The researchers also will employ the analysis of user observation protocols and think-aloud protocol analysis to conduct usability research for the IVR science learning materials.