Project Activities
Researchers conducted a double-blind, randomized controlled trial study with students enrolled in a gateway physics course and a gateway chemistry course. Students participated across two terms, and researchers analyzed students' performance on course outcomes and subsequent enrollment in college science courses. The researchers also conducted a cost analysis to determine the investment necessary for the intervention.
Structured Abstract
Setting
This research study took place at a large university in California.
Sample
Researchers recruited 8054 undergraduate students in gateway undergraduate physics and chemistry courses, of whom around half were female, half were Hispanic or Southeast Asian, and over 40 percent were first-generation students.
Utility-value interventions (UVIs) aim to improve student outcomes by changing students' subjective experiences of their learning. UVIs build off the finding that students are most attracted to and perform best in college courses in which the students expect to succeed or to which the students attach high personal value. In the UVI evaluated in this study, students wrote short essays or letters, discussing how a concept they were learning about was relevant to their or their family's or friends' lives. According to UVI frameworks, when students write such letters, they should discover the personal relevance of course material by making connections between course topics and their lives. This connection should, in turn, help the students appreciate the value of their course work, leading to a deeper level of engagement with the course work and, thus, improved performance. This particular UVI is short, easily implemented in existing classroom structures, and has demonstrated efficacy in other postsecondary science courses.
Research design and methods
Researchers used a double-blind randomized control trial, randomizing students into treatment and control conditions. They used a block-randomization procedure to ensure balanced groups of under-represented minority students, first-generation students, and females in both treatment and control groups. The researchers recruited 6 cohorts of students from a gateway physics course and a gateway chemistry course across 3 academic years. The targeted gateway science courses were taught in a 2-quarter sequence, so each cohort received the intervention (or control) over 2 quarters within each cohort. The first cohort (cohort 0) was a pilot study to ensure that the materials were appropriate for the targeted courses and institution. The remaining cohorts were recruited during the Fall quarters (cohorts 1 and 3) and Winter quarters (cohorts 2 and 4). Within each quarter, students received 2 short writing assignments (500 words) for a total of 4 assignments over the 2-quarter course. In the control condition, students selected a topic that had been covered in lecture in the preceding 2-week period, formulated a question, and wrote about it. In the treatment condition, the students selected a topic and formulated a question, just as in the control condition, but wrote one of two writing assignments: either an essay explaining how the topic was relevant to their own life or a letter to a family member or close friend explaining how the topic was relevant to the letter recipient's life. The ordering of the two choices was controlled in the first quarter, and students selected which choice they preferred (an essay or letter) in the second quarter. The researchers collected baseline and post-test data at the beginning and end of each quarter.
Control condition
Students in the control condition chose a topic covered in a recent lecture and wrote questions and answers focusing on topic content they chose.
Key measures
Key measures tracked students' academic progress (i.e., course grades) and academic persistence (i.e., students' interest in science and enrollment in subsequent science courses). The researchers included variables to investigate potential differential effects for students' sociodemographic characteristics (i.e., sex, ethnic minority status, or first-generation status) and level of academic preparation (i.e., high school GPA, SAT/ACT scores, enrollments in AP science courses) and science-related attitudes, including confidence in ability and perceived value.
Data analytic strategy
The researchers used two-level hierarchical linear models with students nested within sections to conduct impact analyses. They examined the effectiveness of the interventions for subgroups through the use of interaction terms.
Cost analysis strategy
The researchers conducted a cost analysis to determine the costs associated with implementing the utility value intervention in the study context. The researchers explored costs associated with the intervention from the perspective of the university as well as from the student perspective. The average cost associated with the implementation of the intervention in a similar context to the one in this study is $11.66 per participant for the university, whereas students engaging in the intervention would forgo earnings of $29.00.
Key outcomes
The main findings of this project are as follows:
- Students' beliefs about how much STEM jobs can help others or help them individually mattered for whether students felt like they belonged in the field, but, for students who were the first in the family to go to college, the goal of helping their own development did not matter as much as it did for other students (Dicke et al., 2019).
- Students' feelings of belonging often decline during their initial major experiences in college, but female students who were more confident in their physics ability did not show this negative development in the same way than less confident female students (Dicke et al., 2020).
- When students were really interested in their major, they tended to write more about why they chose it, used more positive emotional language, and highlighted enjoyment-related factors for their future goals with greater frequency (Safavian et al., 2021, Safavian et al., 2022).
People and institutions involved
IES program contact(s)
Products and publications
Project website:
Publications:
ERIC Citations: Find available citations in ERIC for this award here.
Available data:
To be made available in the data archive of the Inter-university Consortium for Political and Social Research (ICPSR, https://www.icpsr.umich.edu/web/pages/ICPSR/index.html ). Interested individuals can contact the research team (adicke@uci.edu).
Additional project information
Additional Online Resources and Information: https://twitter.com/projectrise2023
Supplemental information
Co-Principal Investigators: Dennin, Michael; Dicke, Anna-Lena; Safavian, Nayssan
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
