Co-Principal Investigators: Dennin, Michael; Dicke, Anna-Lena; Safavian, Nayssan

Purpose: Postsecondary students in gateway physics and chemistry may struggle to succeed in their courses and persist in their majors for reasons beyond just the difficulty of the subject matter. Previous research with postsecondary biology students found that helping students see the utility of their studies increased their persistence, leading to better academic outcomes. The purpose of this study was to replicate and extend that work, focusing on physics and chemistry students. Additionally, the study aimed to assess whether the approach proved beneficial for first-generation college-going, underrepresented minorities, and female students.

Project Activities: Researchers conducted a double-blind, randomized controlled trial study with students enrolled in a gateway physics course and a gateway chemistry course. Students participated across two terms, and researchers analyzed students' performance on course outcomes and subsequent enrollment in college science courses. The researchers also conducted a cost analysis to determine the investment necessary for the intervention.

Key Outcomes: The main findings of this project are as follows:

• Students' beliefs about how much STEM jobs can help others or help them individually mattered for whether students felt like they belonged in the field, but, for students who were the first in the family to go to college, the goal of helping their own development did not matter as much as it did for other students (Dicke et al., 2019).
• Students' feelings of belonging often decline during their initial major experiences in college, but female students who were more confident in their physics ability did not show this negative development in the same way than less confident female students (Dicke et al., 2020).
• When students were really interested in their major, they tended to write more about why they chose it, used more positive emotional language, and highlighted enjoyment-related factors for their future goals with greater frequency (Safavian et al., 2021, Safavian et al., 2022).

Structured Abstract

Setting: This research study took place at a large university in California.

Sample: Researchers recruited 8054 undergraduate students in gateway undergraduate physics and chemistry courses, of whom around half were female, half were Hispanic or Southeast Asian, and over 40 percent were first-generation students.

Intervention: Utility-value interventions (UVIs) aim to improve student outcomes by changing students' subjective experiences of their learning. UVIs build off the finding that students are most attracted to and perform best in college courses in which the students expect to succeed or to which the students attach high personal value. In the UVI evaluated in this study, students wrote short essays or letters, discussing how a concept they were learning about was relevant to their or their family's or friends' lives. According to UVI frameworks, when students write such letters, they should discover the personal relevance of course material by making connections between course topics and their lives. This connection should, in turn, help the students appreciate the value of their course work, leading to a deeper level of engagement with the course work and, thus, improved performance. This particular UVI is short, easily implemented in existing classroom structures, and has demonstrated efficacy in other postsecondary science courses.

Research Design and Methods: Researchers used a double-blind randomized control trial, randomizing students into treatment and control conditions. They used a block-randomization procedure to ensure balanced groups of under-represented minority students, first-generation students, and females in both treatment and control groups. The researchers recruited 6 cohorts of students from a gateway physics course and a gateway chemistry course across 3 academic years. The targeted gateway science courses were taught in a 2-quarter sequence, so each cohort received the intervention (or control) over 2 quarters within each cohort. The first cohort (cohort 0) was a pilot study to ensure that the materials were appropriate for the targeted courses and institution. The remaining cohorts were recruited during the Fall quarters (cohorts 1 and 3) and Winter quarters (cohorts 2 and 4). Within each quarter, students received 2 short writing assignments (500 words) for a total of 4 assignments over the 2-quarter course. In the control condition, students selected a topic that had been covered in lecture in the preceding 2-week period, formulated a question, and wrote about it. In the treatment condition, the students selected a topic and formulated a question, just as in the control condition, but wrote one of two writing assignments: either an essay explaining how the topic was relevant to their own life or a letter to a family member or close friend explaining how the topic was relevant to the letter recipient's life. The ordering of the two choices was controlled in the first quarter, and students selected which choice they preferred (an essay or letter) in the second quarter. The researchers collected baseline and post-test data at the beginning and end of each quarter.

Control Condition: Students in the control condition chose a topic covered in a recent lecture and wrote questions and answers focusing on topic content they chose.

Key Measures: Key measures tracked students' academic progress (i.e., course grades) and academic persistence (i.e., students' interest in science and enrollment in subsequent science courses). The researchers included variables to investigate potential differential effects for students' sociodemographic characteristics (i.e., sex, ethnic minority status, or first-generation status) and level of academic preparation (i.e., high school GPA, SAT/ACT scores, enrollments in AP science courses) and science-related attitudes, including confidence in ability and perceived value.

Data Analytic Strategy: The researchers used two-level hierarchical linear models with students nested within sections to conduct impact analyses. They examined the effectiveness of the interventions for subgroups through the use of interaction terms.

Cost Analysis:  The researchers conducted a cost analysis to determine the costs associated with implementing the utility value intervention in the study context. The researchers explored costs associated with the intervention from the perspective of the university as well as from the student perspective. The average cost associated with the implementation of the intervention in a similar context to the one in this study is $11.66 per participant for the university, whereas students engaging in the intervention would forgo earnings of $29.00.

Products and Publications

ERIC Citations: Find available citations in ERIC for this award here.

Publicly Available Data: To be made available in the data archive of the Inter-university Consortium for Political and Social Research (ICPSR, https://www.icpsr.umich.edu/web/pages/ICPSR/index.html ). Interested individuals can contact the research team (adicke@uci.edu).

Project Website: https://uci-uvi.weebly.com/

Additional Online Resources and Information: https://twitter.com/projectrise2023