Project Activities
During phase 1, the project team worked with teacher mentors, teachers, content experts, media experts, and software designers to develop 11 video-based visual arts professional development modules that are STELLAR Online. During phase 2, the team worked with rural schools and teachers to conduct focus groups where teachers were given access to initial STELLAR Online modules and provided the research team with feedback. The team used the feedback to improve the modules. During phase 3 in the 2019-2020 school year, the team began a pilot test of STELLAR Online, collecting data on both teacher and student outcome measures. The COVID-19 pandemic interrupted the pilot study which required the researchers to continue the pilot study for 2 additional years to reach desired sample sizes. The pilot study then included 3 waves across 3 school years, 2019-2020 (wave 1), 2020-2021 (wave 2), and 2021-2022 (wave 3), which was not included in the original study plan. Due to the COVID-19 pandemic, recruitment became increasingly difficult over time as teachers became overburdened by shifts to online instruction, staffing shortfalls, and the need to support students who fell behind on academic topics. Despite these challenges, 38 teachers participated in the STELLAR Online pilot study. Team researchers conducted a full analysis of pilot test data, in a no-cost extension year.
Structured Abstract
Setting
Educators and students from 23 Oregon schools participated in the pilot study. Of the 23 schools, 12 were rural, 6 were in a town, 4 were in a suburban setting, and 1 was in a city. Nine schools served students from kindergarten through grade 8, 6 schools served students from kindergarten through grade 5 or 6, 6 schools served grades 6 through 8, there was 1 school that served grades 4 through 6 only, and another that served grades 7 through 12.
Sample
The pilot study included a total sample size of 38 teachers (n=19 in wave 1; n=15 in wave 2, n=4 in wave 3) and 578 students. In wave 1, researchers collected pre-test data from 445 students but were unable to collect post-test data due to school closures in response to the COVID-19 pandemic. The waves 2 and 3 student sample included pre- and post-test data from 133 students.
Intervention
An online professional development program, STELLAR Online consists of 11 video-based instructional modules, each of which begins with a set of learning objectives followed by the instructional material and a brief review the objectives. The instructional material includes text, images, videos, quizzes, interactive activities, handouts, and lesson plans. Over nine months, STELLAR Online is designed to bring about the following teacher outcomes:
Research design and methods
During the materials development stage, the researchers used an iterative learning design framework that provided guidelines for creating online courses. Researchers developed the instructional modules and four mentor teachers already engaged in training teachers on visual thinking strategies in face-to-face settings provided feedback on the modules. The researchers then field-tested the learning design framework as they conducted focus groups in which researchers collected data on teachers' interaction with and implementation of the intervention. For the pilot test stage, the researchers matched participating teachers on two factors, locale status (rural or not rural and grade levels taught) and randomly assigned them to the treatment condition (access to STELLAR online) or to the control condition. The researchers used a pre-post design to measure changes in teachers' knowledge and their fidelity to implementing the intervention within their classrooms, as well as changes to their students' skills in visual literacy, critical thinking, argument writing, interest in the visual arts, and satisfaction with school.
Control condition
In the control condition, teachers did not receive the professional development provided by the intervention and conducted business-as-usual writing instruction in their classrooms.
Key measures
The researchers used teacher and student self-report data and student writing samples as key measures. The teachers completed the STELLAR Teacher Survey at baseline and posttest to assess their self-reported skills and knowledge related to STELLAR Online content. The survey directed teachers to rate their skills and knowledge with a 5-point Likert-type scale on a total of 32 items used to create four scales: perceived knowledge of critical thinking concepts, perceived ability to teach critical thinking concepts, perceived knowledge of concepts related to argument writing, and perceived ability to teach argument writing. Scales were computed based on the average of multiple items related to each construct and produced acceptable reliability estimates (coefficient a = 0.88–0.97). The student students completed a survey that used or adapted existing scales to assess the implementation of activities related to STELLAR Online, critical thinking, self-efficacy for close observation, self-efficacy for argument writing, sense of belonging, intrinsic enjoyment to write, and openness to different perspectives (see Anderson et al., 2023). For the student writing samples, the research team developed a rubric based and ensured that the rubric criteria aligned with Common Core State Standards for Argumentative Writing. Three project staff rated the sample resulting in intraclass correlation coefficients ranging from .67 to .89 across the rating criteria.
Data analytic strategy
During the field test stage, researchers used structured protocol analysis to organize and interpret data from teacher focus groups. To address the intervention efficacy hypothesis, researchers tested condition differences on gains in teacher outcomes and for student outcomes, gains with students nested within teachers.
Key outcomes
The project met its primary objective, creating an accessible online professional development system to help educators teach argument writing through visual art, particularly in under-resourced or rural schools. Additional findings of this project will be posted once they are available.
People and institutions involved
IES program contact(s)
Products and publications
ERIC Citations: Find available citations in ERIC for this award here.
Project Website: https://blogs.uoregon.edu/stellar/
Additional Online Resources and Information:
- The curriculum developed through this project will be publicly available here: https://stellarproject-jsma.org
Select Publications:
Visual thinking and argumentative writing: A social-cognitive pairing for student writing developmentProject website:
Supplemental information
Co-Principal Investigator: Smolkowski, Keith
- The project-developed student survey and argument writing rubric demonstrated adequate reliability. The project team found positive correlations between the generated argument writing total score and self-perceived knowledge and skills associated with critical-thinking, self-efficacy for argumentation, and openness to different perspectives were observed (Anderson et al., 2023).
- increased knowledge and skill in the use of "visual thinking strategies" for visual arts instruction
- increased knowledge and skill in teaching evidence-based reasoning and argument writing aligned with the Common Core State Standards (CCSS) for writing
- increased knowledge and skill in the use of technology tools and online resources for supporting students' visual thinking, evidence-based reasoning, and argument writing across the curriculum
To train teachers in teaching students these skills, STELLAR Online provides teachers with a 9-month online course comprising modules. Within the online application, teachers engaged with the video-based materials as they make their way through the learning modules.
Anderson, R.C., Chaparro, E.A., Smolkowksi, K., Cameron, R. (2023). . Assessing Writing, 55(1). DOI: 10.1016/j.asw.2023.100694Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
