Large-scale surveys can provide state and district education leaders with a deeper understanding of how recommendations are enacted and resources are used. However, survey data are only actionable if the survey asks the right questions and yields sufficient responses.
May 24 Webinar: High Response Rates Through Collaborative Survey Practices
Register for REL Northwest's Webinar on May 24 (12 p.m. Pacific) that will further detail the steps, strategies, and lessons learned that are outlined in this blog. Attendees will receive actionable takeaways to use in their own survey development and administration.
Register now on our events page.
Recently, REL Northwest helped the Alaska Department of Education and Early Development (AK DEED) and its partners develop and administer a statewide school-level survey to better understand how schools and districts implement trauma-engaged practices and policies across the state.
To ensure survey responses reflected perspectives across a wide variety of roles, from teachers to counselors to school leaders, AK DEED asked each school to convene a team of staff familiar with the school's use of trauma-engaged practices (e.g., teachers, school counselors, administrators). Teams gathered for an hour to complete the survey. Despite the complexity of the survey, nearly 60% of public K-12 schools across Alaska responded.
Below, we walk through each step of the survey process, summarize Alaska's approach, and provide the strategies and lessons learned that may be useful for similar survey efforts by other state education agencies or school districts.
The Need for a Trauma-Engaged Schools Implementation Survey
Children that experience trauma are at a higher risk for poor educational, social, health, and economic outcomes. However, providing children with positive and supportive experiences helps to counteract the impact of adverse childhood experiences (Bethell et al., 2014; Blodgett & Lanigan, 2018; Duke, 2020; Flaherty et al., 2013; Hughes et al., 2017; Hunt et al., 2017).
Schools can play a vital role in facilitating positive and supportive experiences for students. In Alaska, two in three children are exposed to traumatic experiences (Alaska Department of Health and Social Services, 2015). In 2019, Alaska released Transforming Schools: A Framework for Trauma-Engaged Practice in Alaska to better equip educators to aid students experiencing trauma. Since its release, the Transforming Schools Framework and suite of accompanying resources have been disseminated to districts and schools statewide. However, little was known about how the framework was used in schools across Alaska.
To ensure schools had the resources needed to use a trauma-engaged approach, the Alaska Department of Education and Early Development (AK DEED) and its partners – the Association of Alaska School Boards, Alaska Department of Health, Alaska Mental Health Trust Authority, and the University of Alaska Anchorage Center for Human Development – wanted to create a survey to better understand how schools and districts are implementing the Transforming Schools framework.
"Many Alaskan schools report that supporting students' mental health, especially those who are coping with trauma and adversity is essential to making schools a place to learn and thrive. This survey and subsequent support will help to focus on what trauma-engaged practices districts are using and ultimately plan for how best to educate and care for students." Patrick Sidmore, program coordinator in the Alaska School Health and Safety Team at AK DEED, explained.
The Transforming Schools framework encourages educators to co-create, reflect, and respect cultural expertise as they develop and refine a trauma-engaged approach. Thus, REL Northwest sought to ensure every step of the survey development and administration process was purposefully designed to elevate co-creation, collaboration, reflection, and flexibility.
REL Northwest facilitated a working group to develop the trauma-engaged practices survey over the course of five meetings that spanned six months. The working group included highly engaged individuals representing state agencies, universities, associations, and school districts. After an iterative process, the resulting survey reflected collective priorities for understanding how schools and districts implement trauma-engaged approaches across Alaska.
Informed by coaching from REL Northwest, Alaska deployed a coordinated communications strategy to promote the survey. The strategy included messaging from multiple individuals in varied roles at AK DEED, as well as messaging from partner organizations and agencies. Throughout the survey administration process, REL Northwest supported the survey team to check response rates weekly, allowing purposeful time to think through gaps in responses and craft outreach plans to target those who hadn't responded. REL Northwest also developed guidance for AK DEED outlining the steps to regularly check response rates.
Although AK DEED began the survey administration process with a plan in place, REL Northwest encouraged agency staff to maintain a flexible mindset and pivot as needed to achieve a high response rate. For example, during the survey administration process, coaching from REL Northwest informed AK DEED's decision to add additional outreach strategies that included broad messaging from multiple leaders in the state and individualized outreach from AK DEED staff well-acquainted with schools across the state. AK DEED also added school-level cash incentives for survey completion and modified how survey links were distributed to make it easier for recipients to respond (e.g., using a general link that could be easily shared via personal correspondence in addition to individualized survey links distributed through the SurveyMonkey platform).
Each school principal was asked to convene a team of diverse voices and provide an hour for collaborative survey completion. Although AK DEED requested that a school team, rather than an individual, complete the survey, the exact number of team members and roles were left to the principal's discretion. This flexible team approach with dedicated time for the survey allowed participants to reflect and engage in discussion to provide survey responses that reflected the school's perspective rather than one individual's perceptions.
The trauma-engaged schools implementation survey example highlights how applying principles of co-creation, collaboration, cultural respect, and flexibility can lead to a survey tool that reflects multiple perspectives and priorities, a high response rate, and valuable data representing multiple perspectives via a single school-level survey.
With data collection complete for the trauma-engaged schools' implementation survey, REL Northwest's Alaska Trauma-Engaged Schools Partnership will explore these data to provide information on which trauma-engaged strategies schools use, whether factors like school size and location influence the use of trauma-engaged strategies, and how the use of these strategies is related to student and staff outcomes (e.g., attendance, school climate).
Findings from this study will aid AK DEED and its partners in refining the state's trauma-engaged services and resources.
Bethell, C. D., Newacheck, P., Hawes, E., & Halfon, N. (2014). Adverse childhood experiences: assessing the impact on health and school engagement and the mitigating role of resilience. Health Affairs, 33(12), 2106–2115. https://doi.org/10.1377/hlthaff.2014.0914.
Blodgett, C., & Lanigan, J. D. (2018). The association between adverse childhood experience (ACE) and school success in elementary school children. School Psychology Quarterly, 33(1), 137. https://doi.org/10.1037/spq0000256.
Duke, N. N. (2020). Adolescent adversity, school attendance and academic achievement: School connection and the potential for mitigating risk. Journal of School Health, 90(8), 618–629.
Flaherty, E. G., Thompson, R., Dubowitz, H., Harvey, E. M., English, D. J., Proctor, L. J., & Runyan, D. K. (2013). Adverse childhood experiences and child health in early adolescence. JAMA Pediatrics, 167(7), 622–629.
Hughes, K., Bellis, M. A., Hardcastle, K. A., Sethi, D., Butchart, A., Mikton, C., Jones, L., & Dunne, M. P. (2017). The effect of multiple adverse childhood experiences on health: A systematic review and meta-analysis. The Lancet Public Health, 2(8), e356–e366. https://doi.org/10.1016/S2468-2667(17)30118-4.
Hunt, T. K., Slack, K. S., & Berger, L. M. (2017). Adverse childhood experiences and behavioral problems in middle childhood. Child Abuse & Neglect, 67, 391–402. https://doi.org/10.1016/j.chiabu.2016.11.005.
Alaska Department of Health and Social Services (DHSS). (2015). Adverse childhood experiences: Overcoming ACEs in Alaska. http://dhss.alaska.gov/abada/ace-ak/Documents/ACEsReportAlaska.pdf.