Systems Conditions for SEL Implementation
April 2019
Question
What does the research say about the systems conditions in schools and districts that support quality SEL implementation and positive school climate that ultimately help students succeed? In particular, what can we learn from implementation science and the broader evidence base around school transformation and innovation that may apply to SEL?
Ask A REL Response
Thank you for your request to our Regional Educational Laboratory (REL) Reference Desk. Ask A REL is a collaborative reference desk service provided by the 10 RELs that, by design, functions much in the same way as a technical reference library. Ask A REL provides references, referrals, and brief responses in the form of citations in response to questions about available education research.
Following an established REL Northwest research protocol, we conducted a search for evidence- based research. The sources included ERIC and other federally funded databases and organizations, research institutions, academic research databases, Google Scholar, and general Internet search engines. For more details, please see the methods section at the end of this document.
The research team has not evaluated the quality of the references and resources provided in this response; we offer them only for your reference. The search included the most commonly used research databases and search engines to produce the references presented here. References are listed in alphabetical order, not necessarily in order of relevance. The research references are not necessarily comprehensive and other relevant research references may exist. In addition to evidence-based, peer-reviewed research references, we have also included other resources that you may find useful. We provide only publicly available resources, unless there is a lack of such resources or an article is considered seminal in the topic area.
References
Cahill, H., Kern, M. L., Dadvand, B., Cruickshank, E. W., Midford, R., Smith, C., et al. (2019). An integrative approach to evaluating the implementation of social and emotional learning and gender-based violence prevention education. International Journal of Emotional Education, 11(1), 135–152. https://eric.ed.gov/
From the Abstract:
"Evaluation studies often use stand-alone and summative assessment strategies to examine the impacts of Social and Emotional Learning (SEL) and Gender-based Violence (GBV) prevention education programs. However, implementation research is yet to offer an integrative framework that can be used to investigate the implementation drivers that lead to the uptake of programs that pursue SEL and GBV prevention agendas. We address this gap in research by presenting a framework developed to investigate factors affecting the implementation of the Resilience, Rights and Respectful Relationships program, an SEL and GBV prevention education program developed for primary and secondary schools in the state of Victoria, Australia. Drawing upon and advancing a conceptual framework for implementation fidelity proposed by Carroll and colleagues we discuss the iterative process designed to investigate the individual, school and system level factors within the wider political and ideological setting(s) of the program that impact on its implementation. Within this iterative process, we highlight the need to focus on 'the ecology of relations' that exists between various implementation elements, and their possible mediating impact on program delivery, uptake and outcomes."
Domitrovich, C. E., Pas, E. T., Bradshaw, C. P., Becker, K. D., Keperling, J. P., Embry, D. D., & Ialongo, N. (2015). Individual and school organizational factors that influence implementation of the PAX good behavior game intervention. Prevention Science, 16(8), 1064–1074. Retrieved from https://www.researchgate.net/
From the Abstract:
"Evidence-based interventions are being disseminated broadly in schools across the USA, but the implementation levels achieved in community settings vary considerably. The current study examined the extent to which teacher and school factors were associated with implementation dosage and quality of the PAX Good Behavior Game (PAX GBG), a universal classroom-based preventive intervention designed to improve student social-emotional competence and behavior. Specifically, dosage (i.e., number of games and duration of games) across the school year and quality (i.e., how well the game is delivered) of PAX GBG implementation across four time points in a school year were examined. Hierarchical linear modeling was used to examine the association between teacher-level factors (e.g., demographics, self-reports of personal resources, attitudes toward the intervention, and workplace perceptions) and longitudinal implementation data. We also accounted for school-level factors, including demographic characteristics of the students and ratings of the schools' organizational health. Findings indicated that only a few teacher-level factors were significantly related to variation in implementation. Teacher perceptions (e.g., fit with teaching style, emotional exhaustion) were generally related to dosage, whereas demographic factors (e.g., teachers' age) were related to quality. These findings highlight the importance of school contextual and proximal teacher factors on the implementation of classroom-based programs."
Evans, R., Murphy, S., & Scourfield, J. (2015). Implementation of a school-based social and emotional learning intervention: Understanding diffusion processes within complex systems. Prevention Science, 16(5), 754–764. Retrieved from https://link.springer.com/
From the Abstract:
"Sporadic and inconsistent implementation remains a significant challenge for social and emotional learning (SEL) interventions. This may be partly explained by the dearth of flexible, causative models that capture the multifarious determinants of implementation practices within complex systems. This paper draws upon Rogers (2003) Diffusion of Innovations Theory to explain the adoption, implementation and discontinuance of a SEL intervention. A pragmatic, formative process evaluation was conducted in alignment with phase 1 of the UK Medical Research Council's framework for Developing and Evaluating Complex Interventions. Employing case-study methodology, qualitative data were generated with four socio-economically and academically contrasting secondary schools in Wales implementing the Student Assistance Programme. Semi-structured interviews were conducted with 15 program stakeholders. Data suggested that variation in implementation activity could be largely attributed to four key intervention reinvention points, which contributed to the transformation of the programme as it interacted with contextual features and individual needs. These reinvention points comprise the following: intervention training, which captures the process through which adopters acquire knowledge about a programme and delivery expertise; intervention assessment, which reflects adopters' evaluation of an intervention in relation to contextual needs; intervention clarification, which comprises the cascading of knowledge through an organisation in order to secure support in delivery; and intervention responsibility, which refers to the process of assigning accountability for sustainable delivery. Taken together, these points identify opportunities to predict and intervene with potential implementation problems. Further research would benefit from exploring additional reinvention activity."
Faria, A. M., Kendziora, K., Brown, L., O'Brien, B., & Osher, D. (2013). PATHS implementation and outcome study in the Cleveland Metropolitan School District: Final report. Washington, DC: American Institutes for Research. Retrieved from https://www.air.org/
From the Document:
"The Cleveland Metropolitan School District (CMSD) implemented an evidence-based social and emotional learning program—Promoting Alternative Thinking Strategies (PATHS)—districtwide beginning in the 2009–10 school year. The PATHS curriculum is designed to help elementary-age children increase self-control, choose effective conflict-resolution strategies, reject aggressive responses to frustrating situations, improve problem-solving skills, as well as to facilitate educational processes in the classroom and has strong evidence of improving students' social development and academic performance (PATHS; Greenberg, Kusché, Cook, & Quamma, 1995; Kusché & Greenberg, 1994). Implementing PATHS in all elementary schools in CMSD represents the first large-district scale up of an evidence-based social and emotional learning program in the U.S. The American Institutes for Research conducted an evaluation of the CMSD's implementation of PATHS. The goals of the evaluation were to: 1. document and describe how PATHS was implemented in Year 1 (2010–11) and Year 2 (2011–12), 2. document change in students' social emotional outcomes in PATHS classrooms over time, 3. examine how PATHS implementation was related to students' social emotional outcomes, including social competence, aggression, and attention, and 4. examine how contextual factors such as school climate and teacher morale were related to both PATHS implementation and student outcomes."
Flaspohler, P. D., Meehan, C., Maras, M. A., & Keller, K. E. (2012). Ready, willing, and able: Developing a support system to promote implementation of school-based prevention programs. American Journal of Community Psychology, 50(3-4), 428–444. Retrieved from https://www.researchgate.net
From the Abstract:
"While the number and scope of evidence-based health, education, and mental health services continues to grow, the movement of these practices into schools and other practice settings remains a complex and haphazard process. The purpose of this paper is to describe and present initial support for a prevention support system designed to promote high-quality implementation of whole school prevention initiatives in elementary and middle schools. The function and strategies of a school-based prevention support system are discussed, including key structures and activities undertaken to identify, select, and provide technical assistance to school personnel. Data collected over a 5 year period are presented, including evidence of successful implementation support for 5 different evidence-based programs implemented with fidelity at 12 schools and preliminary evidence of goal attainment. Findings suggest the ongoing collection of information related to organizational readiness assists in the adoption and implementation of effective practices and initiatives and provide valuable insight into the development of results-oriented approaches to prevention service delivery. Problems, progress, and lessons learned through this process are discussed to frame future research and action steps for this school-based prevention support system."
Gould, L. F., Dariotis, J. K., Greenberg, M. T., & Mendelson, T. (2016). Assessing fidelity of implementation (FOI) for school-based mindfulness and yoga interventions: A systematic review. Mindfulness, 7(1), 5–33. Retrieved from https://www.ncbi.nlm.nih.gov
From the Abstract:
"As school-based mindfulness and yoga programs gain popularity, the systematic study of fidelity of program implementation (FOI) is critical to provide a more robust understanding of the core components of mindfulness and yoga interventions, their potential to improve specified teacher and student outcomes, and our ability to implement these programs consistently and effectively. This paper reviews the current state of the science with respect to inclusion and reporting of FOI in peer-reviewed studies examining the effects of school-based mindfulness and/or yoga programs targeting students and/or teachers implemented in grades kindergarten through twelve (K-12) in North America. Electronic searches in PsychInfo and Web of Science from their inception through May 2014, in addition to hand searches of relevant review articles, identified 312 publications, 48 of which met inclusion criteria. Findings indicated a relative paucity of rigorous FOI. Fewer than 10% of studies outlined potential core program components or referenced a formal theory of action, and fewer than 20% assessed any aspect of FOI beyond participant dosage. The emerging nature of the evidence base provides a critical window of opportunity to grapple with key issues relevant to FOI of mindfulness-based and yoga programs, including identifying essential elements of these programs that should be faithfully implemented and how we might develop rigorous measures to accurately capture them. Consideration of these questions and suggested next steps are intended to help advance the emerging field of school-based mindfulness and yoga interventions."
Lyon, A. R., Cook, C. R., Brown, E. C., Locke, J., Davis, C., Ehrhart, M., & Aarons, G. A. (2018). Assessing organizational implementation context in the education sector: Confirmatory factor analysis of measures of implementation leadership, climate, and citizenship. Implementation Science, 13(5), 1–14. Retrieved from https://implementationscience.biomedcentral.com
From the Abstract:
"Background: A substantial literature has established the role of the inner organizational setting on the implementation of evidence-based practices in community contexts, but very little of this research has been extended to the education sector, one of the most common settings for the delivery of mental and behavioral health services to children and adolescents. The current study examined the factor structure, psychometric properties, and interrelations of an adapted set of pragmatic organizational instruments measuring key aspects of the organizational implementation context in schools: (1) strategic implementation leadership, (2) strategic implementation climate, and (3) implementation citizenship behavior. Method: The Implementation Leadership Scale (ILS), Implementation Climate Scale (ICS), and Implementation Citizenship Behavior Scale (ICBS) were adapted by a research team that included the original scale authors and experts in the implementation of evidence-based practices in schools. These instruments were then administered to a geographically representative sample (n = 196) of school-based mental/behavioral health consultants to assess the reliability and structural validity via a series of confirmatory factor analyses. Results: Overall, the original factor structures for the ILS, ICS, and ICBS were confirmed in the current sample. The one exception was poor functioning of the Rewards subscale of the ICS, which was removed in the final ICS model. Correlations among the revised measures, evaluated as part of an overarching model of the organizational implementation context, indicated both unique and shared variance. Conclusions: The current analyses suggest strong applicability of the revised instruments to implementation of evidence-based mental and behavioral practices in the education sector. The one poorly functioning subscale (Rewards on the ICS) was attributed to typical educational policies that do not allow for individual financial incentives to personnel. Potential directions for future expansion, revision, and application of the instruments in schools are discussed."
McCormick, M. P., Cappella, E., O'Connor, E. E., & McClowry, S. G. (2015). Context matters for social-emotional learning: Examining variation in program impact by dimensions of school climate. American Journal of Community Psychology, 56(1-2), 101–119. Retrieved from https://www.researchgate.net
From the Abstract:
"This paper examines whether three dimensions of school climate—leadership, accountability, and safety/respect—moderated the impacts of the INSIGHTS program on students' social-emotional, behavioral, and academic outcomes. Twenty-two urban schools and N = 435 low-income racial/ethnic minority students were enrolled in the study and received intervention services across the course of 2 years, in both kindergarten and first grade. Intervention effects on math and reading achievement were larger for students enrolled in schools with lower overall levels of leadership, accountability, and safety/respect at baseline. Program impacts on disruptive behaviors were greater in schools with lower levels of accountability at baseline; impacts on sustained attention were greater in schools with lower levels of safety/respect at baseline. Implications for Social-Emotional Learning program implementation, replication, and scale-up are discussed."
Methods
Keywords and Search Strings: The following keywords, subject headings, and search strings were used to search reference databases and other sources: Systems, Social-emotional, Social and emotional, Implementation
Databases and Resources: We searched ERIC for relevant resources. ERIC is a free online library of more than 1.6 million citations of education research sponsored by the Institute of Education Sciences (IES). Additionally, we searched Google Scholar and EBSCO databases (Academic Search Premier, Education Research Complete, and Professional Development Collection).
Reference Search and Selection Criteria
When we were searching and reviewing resources, we considered the following criteria:
Date of publications: This search and review included references and resources published in the last 10 years.
Search priorities of reference sources: Search priority was given to study reports, briefs, and other documents that are published and/or reviewed by IES and other federal or federally funded organizations, as well as academic databases, including ERIC, EBSCO databases, and Google Scholar.
Methodology: The following methodological priorities/considerations were given in the review and selection of the references:
- Study types: randomized control trials, quasi experiments, surveys, descriptive data analyses, literature reviews, and policy briefs, generally in this order
- Target population and samples: representativeness of the target population, sample size, and whether participants volunteered or were randomly selected
- Study duration
- Limitations and generalizability of the findings and conclusions
This memorandum is one in a series of quick-turnaround responses to specific questions posed by stakeholders in Alaska, Idaho, Montana, Oregon, and Washington, which is served by the Regional Educational Laboratory (REL) Northwest. It was prepared under Contract ED-IES-17-C-0009 by REL Northwest, administered by Education Northwest. The content does not necessarily reflect the views or policies of IES or the U.S. Department of Education, nor does mention of trade names, commercial products, or organizations imply endorsement by the U.S. Government.