Skip Navigation
Funding Opportunities | Search Funded Research Grants and Contracts

IES Grant

Title: The Organizational Assessment for Strategic Implementation in Schools (OASIS): A Measurement Suite to Support Educator Adoption of Evidence-Based Practices
Center: NCER Year: 2016
Principal Investigator: Lyon, Aaron Awardee: University of Washington
Program: Social, Emotional, and Behavioral Context for Teaching and Learning      [Program Details]
Award Period: 3 years (7/1/2016–6/30/2019) Award Amount: $1,393,121
Type: Measurement Award Number: R305A160114
Description:

Co-Principal Investigator: Cook, Clayton R.

Purpose: In this project, researchers developed a suite of measures called the Organizational Assessment for Strategic Implementation in Schools (OASIS). OASIS is designed for teachers to provide school and district leaders with information regarding how best to support and sustain the use of evidence-based practices to prevent emotional and behavioral problems among students. While research has shown that these are practices teachers and schools can use to reduce emotional and behavioral problems for students, teachers and schools are not necessarily using these practices in their classrooms. Researchers developed four measures that assess aspects of a school culture, including attitudes, school climate, and school leadership characteristics. Teachers complete the assessments and school and district leaders use data from OASIS to make decisions about how best to devote resources to carrying out effective practices in the classroom.

Project Activities: The research team revised measures that are used to assess features of health organizations that support the use of evidence-based health practices so that they address the features of schools. Researchers revised and developed items and then used a series of focus groups, expert review, and field testing to further revise and refine the items. The testing occurred in schools that were implementing one of two evidence-based programs to aim to reduce the emotional and behavioral problems of elementary school students.

Key Findings: The main features of the assessment and outcomes of the validation study are as follows:

  • Focus groups with central office district administrators, principals, and teachers identified new aspects of strategic implementation that are relevant to schools (Locke et al., 2019).
  • The revised and expanded School Implementation Leadership Scale (SILS) is reliable and has a structure similar to the original Implementation Leadership Scale as well as evidence for its relationship to related concepts such as general leadership (Lyon et al., 2018) .

Structured Abstract

Setting: This project took place in elementary schools in the Midwest and on the West Coast.

Sample: Participants across phases of this study were 528 teachers and other school personnel who work in or with elementary schools.

Assessment: This computer-based suite of assessments is designed to be completed by teachers and to provide school and district personnel with information regarding barriers to the implementation of evidence-based practices and to guide decision-making regarding the distribution of resources to support implementation of these practices. The assessments are revised from the Exploration, Preparation, Implementation, Sustainment (EPIS) measures which were originally developed by members of the research team to examine implementation-related organizational factors in the health field. The final OASIS assessments cover four core constructs: (1) strategic implementation climate, which encompasses staff norms and expectations about implementing new programs and practices; (2) strategic implementation leadership, which involves specific leadership behaviors that support or inhibit successful implementation; (3) implementation citizenship behavior, which are behaviors that go beyond standard core job responsibilities and may mediate the influence of climate and leadership on implementation success; and (4) attitudes toward evidence-based practices, which encompasses the attitudes of teachers and which may mediated the influence of climate and leadership on implementation. The assessment also includes feedback reports that create accessible visualizations of instrument data to guide implementation decision-making.

Research Design and Methods: This project included five studies: three assessment development studies, a large-scale validation study, and a pilot usability study. In the first phase, the team carried out three assessment development studies: (1) focus group studies with teachers and school personnel to adapt and add items to form a preliminary item pool; (2) expert review of the tools by researchers and practitioners in order to examine conceptual representativeness and appropriateness of the measures across a range of education settings; and (3) field testing and refinement, which involves two small-scale pilots to estimate completion time, troubleshoot technical problems, and revise the item pool. In the second phase, the researchers examined construct validity by administering the measures to a large sample of teachers from schools implementing one of two universal evidence-based prevention programs: School-Wide Positive Behavioral Interventions and Supports (SWPBIS) or Promoting Alternative Thinking Strategies (PATHS). By administering OASIS in these schools, researchers were able to determine associations between OASIS and successful implementation of SWPBIS or PATHS. In the final phase, the researchers conducted a usability study that involved qualitative interviews with administrators and other stakeholders to maximize utility and usability of the OASIS system and refine the OASIS reports.

Control Condition: Due to the nature of the research design, there was not a control condition.

Key Measures: In order to examine the validity of OASIS, researchers used measures of classroom behavioral outcomes, student academic outcomes such as standardized tests and attendance, and related organizational measures (Organizational Health Inventory, the Principal Instructional Management Rating Scale, the Organizational Citizenship Behavior Scale, and the Public School Teacher Questionnaire). Researchers also examined the associations between OASIS and the successful implementation of SWPBIS and PATHS by using each program's fidelity measure and other indicators of implementation outcomes.

Data Analytic Strategy: Researchers coded focus group and interview data using directed content analysis. They analyzed validity by examining response distributions, factor analysis using latent variable modeling, and item response theory. They used multilevel latent variable models to examine the multi-level factor structure of the OASIS items and the convergent and discriminant validity.

Related IES Projects: Helping Educational Leadership Mobilize Evidence (HELM): An Organizational Intervention to Promote Strategic Implementation in Schools (R305A200023)

Publications and Products

ERIC Citations: Find available citations in ERIC for this award here.

Cook, C. R., Davis, C. Brown, E. C., Locke, J., Ehrhart, M. G., Aarons, G. A., Larson, M., and Lyon, A. R. (2018). Confirmatory factor analysis of the Evidence-Based Practice Attitudes Scale with school-based behavioral health consultants. Implementation Science, 13:116.

Cook, C., Lyon, A. R., Locke, J., Waltz, T. J., and Powell, B. J. (2019). Adapting a compilation of implementation strategies to advance school-based implementation research and practice. Prevention Science, 20: 914–935.

Corbin, C.M., Hugh, M.L., Ehrhart, M.G., Locke, J., Davis, C., Brown, E.C., Cook, C.R., and Lyon, A.R. (2022). Teacher perceptions of implementation climate related to feasibility of implementing schoolwide positive behavior supports and interventions. School Mental Health, 14(4): 1057-1069.

Locke, J., Lee, K., Cook, C. R., Frederick, L., Vázquez-Colón, C., Ehrhart, M. G., ... Lyon, A. R. (2019). Understanding the organizational implementation context of schools: A qualitative study of school district administrators, principals, and teachers. School Mental Health, 11: 379-399.

Lyon, A. R., Cook, C. R., Brown, E. C., Locke, J., Davis, C., Ehrhart, M., & Aarons, G. A. (2018). Assessing organizational implementation context in the education sector: implementation leadership, climate, and citizenship. Implementation Science, 13(5).

Lyon, A.R., Corbin, C. M., Brown, E.C., Ehrhart, M.G., Locke, J., Davis, C., Picozzi, E., Aarons, G. A., and Cook, C. R. (2022). Leading the charge in the education sector: Development and validation of the School Implementation Leadership Scale (SILS). Implementation Science, 17: 48.

Lyon, A. R., Whitaker, K., Locke, J., Cook, C. R., King, K. M., Duong, M., Davis, C., Weist, M. D., Ehrhart, M. G., and Aarons, G. A. (2018). The impact of inter-organizational alignment (IOA) on implementation outcomes: Evaluating unique and shared organizational influences in education sector mental health. Implementation Science, 13: 1-11.

Merle, J. L., Cook, C. R., Locke, J. J., Ehrhart, M. G., Brown, E. C., Davis, C. J., and Lyon, A. R. (2023). Teacher attitudes toward evidence-based practices: Exploratory and confirmatory analyses of the school-adapted Evidence-Based Practice Attitude Scale. Implementation Research and Practice, 4: 1-16.

Thayer, A. J., Cook, C. R., Davis, C., Brown, E. C., Locke, J., Ehrhart, M. G., Aarons, G. A., Picozzi, E., and Lyon, A. Construct validity of the School-Implementation Climate Scale. Implementation Research and Practice, 3: 1-14.

Zhang, Y., Cook, C. R., Fallon, L., Corbin, C., Ehrhart, M., Brown, E., Locke, J., and Lyon, A. (2022). The interaction between general and strategic leadership and climate on their multilevel associations with implementer attitudes toward universal prevention programs for youth mental health: A cross-sectional study. Administration and Policy in Mental Health and Mental Health Services Research, 50(3):427–449.


Back