Evaluation of We Have Skills, A Multimedia Classroom Level Social Skills Program for Elementary Students
Co-Principal Investigator: Keith Smolkowski (Oregon Research Institute)
Purpose: The purpose of this study is to test whether the We Have Skills! (WHS) program (developed at IRIS Media through support of an IES development grant) can help young students learn social skills and behaviors to support success in school. Early elementary school is an ideal time to teach children these social skills and behaviors, yet few teachers feel prepared to do so, lacking both time and resources to support their students adequately. The WHS program teaches students seven social and academic behaviors identified by teachers as important to achievement in early elementary school. The curriculum is designed as a universal (Tier 1) prevention strategy, explicitly teaching the skills to all students, and offers more intensive (Tier 2) supports for those students with more challenging behaviors.
Project Activities: First grade teachers are randomly assigned in three cohorts to implement WHS or to continue with their typical classroom practices. Students will be followed into second grade to determine if the impact of the program persists beyond the implementation year. Teachers randomly assigned to the comparison group will be offered WHS the following year.
Products: This project will produce evidence of the efficacy of the We Have Skills! (WHS) program to improve behavior and learning in first and second grade students. Researchers will also produce peer reviewed publications.
Setting: This study will take place in Oregon, Washington and California.
Sample: Study participants for this study include about 160 first and second grade teachers and approximately 3,040 first grade students who will be followed into second grade.
Intervention: The We Have Skills! (WHS) program consists of professionally produced high-definition videos used to teach students seven important social and academic behaviors: listening to the teacher, following directions, doing your best work, asking for help, following the rules, getting along with others, and managing strong feelings. The videos are supplemented with music, activities, and print materials for teachers, students, and their parents. Video and music are delivered through via the Internet or DVD. The WHS program includes a universal screening and progress-monitoring tool (the irisPMTTM) to help teachers track the gains made by their students and monitor those who need additional support. The program also includes a professional development module to address any concerns that teachers might have regarding implementation (initial group training, in-class coaching, and online coaching). WHS has been adopted by eight large to medium school districts in the United States and over 350 schools in the United States, Norway, and Sweden. It was recently licensed for commercial dissemination by Houghton Mifflin Harcourt's Riverside Publishing division.
Research Design and Methods: To determine the impact of WHS on student social and academic outcomes, researchers will randomly assign 160 classrooms in three waves (40 classrooms in Year 1, 60 classrooms in Year 2 and another 60 classrooms in Year 3) to WHS or a business as usual control group. During pretest and posttest, the research team will use a measure of general fidelity to evaluate which classroom behavioral skills programs are being used by control classrooms. Control teachers will be trained in how to use the irisPMTTM to support data collection. Teachers in the WHS condition will receive the complete WHS program, training in using the WHS program and the irisPMTTM, and coaching. Teachers are expected to teach eight lessons (Introduction and 7 Skills) at a rate of one per week, provide children with independent activities, and send home parent materials that support the WHS classroom activities. Once the initial instruction has been given, teachers will use the irisPMT™ to identify skills and students that need additional attention. Teachers will give additional instruction and allow opportunities for practice over the remainder of the school year.
Control Condition: In this study, the comparison classrooms will conduct standard instruction.
Key Measures: Student outcomes are measured using teacher ratings on the Elementary Social Behavior Assessment (ESBA), the Walker-McConnell Scale of Social Competence and School Adjustment – Elementary Version (WMS-EV), irisPMTTM, and the Reading Rating Form. Other measures include classroom observations conducted by the researchers using the Assessing School Setting: Interactions of Students and Teachers (ASSIST) and items derived from the Classroom Atmosphere Rating Scale. School record data is collected to characterize participating schools. Teacher surveys are used to collect student-level demographic information, such as gender, IEP status, and English learner status. Teacher characteristics are assessed using the Teacher Sense of Efficacy Scale-Short Form and the Teacher Stress Inventory.
Data Analytic Strategy: The impact of WHS on student outcomes will be analyzed using mixed-model analysis of covariance or nested random coefficients analysis. Covariates and their interactions with the test of condition will be added to the models to look at potential moderators of impact such as teacher stress and student gender.
Related IES Projects: Online Teacher Training: Promoting Student Social Competence to Improve Academic and Behavioral Outcomes in Grades K-3 (R324A080150), Establishing Positive Behavior Supports in Elementary School Instructional Settings (R305A090107), Student Self-Management System (SSMS): Reducing Problem Behavior in Upper Elementary Classrooms by Transferring Externally Applied Teacher Controls to Internally Applied Student Controls (R324A110074)
Journal article, monograph, or newsletter
Arnesen, A., Smolkowski, K., Ogden, T., and Melby-Lervåg, M. (2017). Validation of the Elementary Social Behaviour Assessment: Teacher Ratings of Students' Social Skills Adapted to Norwegian, Grades 1–6. Emotional and Behavioural Difficulties, 1–16.