By Marguerite Huber
December 28, 2017
The Wisconsin Department of Public Instruction (DPI) collects a substantial amount of data on students, teachers, schools, and libraries. With such an abundance of data, Kurt Kiefer, a member of the Midwest Alliance to Improve Knowledge Utilization and assistant state superintendent in the Division for Libraries and Technology at DPI, was interested in increasing his colleagues’ capacity to use data and apply research. He wanted to create a data-savvy culture in which staff have the skills and tools to collect, interpret, and use data to support their mission.
This idea led to the creation of a workshop series in partnership with Regional Educational Laboratory (REL) Midwest. Learn more about how this series originated in this blog post. The series of six workshops, which started in the fall, are geared toward general DPI staff with a focus on best practices for collecting, using, and sharing data. It also aims to connect the material to their work at DPI.
Although it is a more distant relationship, students, in addition to staff, will benefit from the workshops. Jill Walston, Ph.D., of REL Midwest is leading the workshop series sessions. She explained, “Increasing DPI staff ability to collect accurate and relevant data and to be critical consumers of research and evidence makes it more likely that practices and policies will be put into place that are most likely to improve student outcomes.”
The first session covering research methods was held in September 2017, and the sixth and final session on communicating about research findings will happen in April 2018. The sessions are structured so that attendees participate in a presentation about the topic followed by an activity that applies the lessons to their own DPI projects or a hypothetical project.
After completing the sessions, DPI staff who participate will have the tools they need to collect and analyze data, report findings, and use results to inform decisions. The impact the workshops will have on DPI staff’s work will vary by session.
Session 1: Research Methods Overview (September 2017)
In this first session, participants received an overview of research methodology. Dr. Walston shared information about the appropriate use of different data collection methods, and participants practiced selecting the best methods for various scenarios. In addition, Dr. Walston shared introductory information about different types of research designs and how to judge program effectiveness according to the level of evidence provided by existing research studies. Participants discussed how DPI uses and supports districts’ use of data and research evidence in the context of the continuous improvement cycle.
Feedback from the first session underscored that DPI staff appreciated the concrete examples of collecting data, as well as what data collection method is best suited for different scenarios. One participant planned to reconsider the ways that DPI has been collecting data, and another noted, “We plan to look at a self-assessment our team is drafting. We will think about the most relevant ways to gather data for this and then put it to the test!”
Resources: Session 1 Materials
Session 2: Survey Development and Administration (October 2017)
Dr. Walston shared best practices for planning a survey project, and participants worked in pairs to complete a planning tool for a real or hypothetical survey project. After hearing a presentation about best practices for writing effective survey items, participants worked in small groups to identify flaws and revise a set of example survey items, including draft survey items for an upcoming DPI survey. Participants discussed the resources and processes needed for ensuring that DPI conducts high-quality surveys.
Our Survey Development Guidance Checklist provides an overview of the steps involved in developing a survey and planning for its administration. Use our Survey Planning Gantt Chart to help prepare a timetable for planning, survey development, administration, and data analysis.
Session 2 led one participant to want to check the readability of their surveys and revise them with goals in mind, while another found, “The most helpful aspect of the workshop was learning how to write meaningful and answerable survey questions.”
Resources:
- Session 2 Materials
- Workshop on Survey Methods in Education Research: Facilitator’s Guide and Resources
Session 3: Interviews and Focus Groups (November 2017)
This session covered best practices for recruiting interviewees, preparing protocols, conducting interviews and focus groups, as well as analyzing and summarizing their results. Participants worked in small groups to create focus group protocols and engaged in a mock focus group.
The most recent session generated excitement to conduct focus groups or to reevaluate how DPI conducted focus groups previously. Staff appreciated the small groups and hands-on aspect of the session along with being able to put their questions to the test in a mock focus group. “As a result, I will examine how I could improve my work and DPI processes using focus groups,” remarked one participant.
Resources: Session 3 Materials
Session 4: Observations and Rubric Development (January 2018)
This session will present tools for collecting observational data and the process for developing analytic rubrics for observational research. In small groups, participants will get to create and test a short self-assessment rubric relating to DPI’s work.
Session 5: Data Analysis (March 2018)
In this session, participants will dive into basic statistical concepts along with an activity on preparing an example analysis plan aligned with research questions of interest to the participants.
Session 6: Communicating Research Findings (April 2018)
The last workshop session will cover best practices for communicating research findings and for creating graphical displays of data to engage audiences and support interpretation. The session will highlight strategies for supporting the use of data and research findings to drive decision-making and support policy and practice. Participants will learn how to choose the right graph and will explore the data inquiry process.
Katherine McCoy, an evaluation consultant at DPI, reflected on the sessions she attended, noting, “These trainings provide useful tools that I can share with our grantees in the field. The resources provide helpful guides that can help them to better interpret the information we provide to them, as well as to take the next step toward better understanding the needs of their students or colleagues. I think the REL resources can empower school personnel to engage more meaningfully with their data and, therefore, become more confident in their ability to problem-solve.”
For more information, please contact Jill Walston at jwalston@air.org.
Author(s) Information
