The 2017 North Dakota Legislative Assembly passed a bill, signed into law, that allows schools and districts to develop comprehensive plans altering the delivery of education, which sometimes includes a waiver request of some state education regulations. In doing so, it gave the North Dakota Department of Public Instruction (NDDPI) district and school administrators the flexibility to think big in proposing, developing, and implementing innovative education programs and systems with the goal of improving education delivery, administration, and opportunities, as well as the academic success of students. However, with no standardized way to score and respond to applicant improvement proposals, NDDPI was concerned its staff could not provide consistent feedback to help applicants ensure proposals met legislative intent. To address this issue, NDDPI, as part of the North Dakota Innovative Schools Research Partnership, collaborated with REL Central researchers to develop standardized rubrics and supports to evaluate proposals, implementation applications, and ultimately program success.
Not wanting to miss out on potential innovations sometimes developed by educators at the local level, legislators passed changes to the North Dakota Century Code in 2017. Those changes allow NDDPI to waive state education regulations for districts and schools who want to bring innovative programs and plans into a classroom or address districtwide systemic changes. Under the law, school or district proposed innovative education program proposals can be approved if they are designed to improve the delivery of education, the administration of education, the academic success of students, or provide an increase in educational opportunities for students. Applicants seeking waivers create and submit their Innovative Education Program Planning Proposal; work on their implementation proposal, termed an Innovative Education Implementation Application, for a year; submit that proposal; and implement the program upon implementation proposal approval. At each stage, NDDPI reviews applicant proposals and provides feedback to strengthen them. To facilitate this process, NDDPI was required to develop criteria to review each applicant proposal.
In creating the criteria, NDDPI used the state’s administrative code and legislative intent to develop key elements such as public school district board program approval and stakeholder engagement. However, the vast range of innovative education programs possible under the law revealed a question as to how each proposal could and should be measured against those key elements.
Gail Schauer, director of NDDPI’s Office of School Approval and Opportunity, explained the concern.
“The sky is the limit in terms of the innovations that schools can bring to us. We don’t want to limit their innovation,” said Schauer. “But how do you create a scoring system for plans that are as granular as classroom techniques and as broad as systems change? That was where we were at.”
Schauer said that, in reviewing the first few proposals, many lacked critical elements such as stakeholder engagement in the development of proposals. Schauer noted that one school contacted NDDPI with an interest in tying in core course competencies with career exploration courses. The school had discussed the possibilities with their staff, but had not yet reached out to other key stakeholders including parents, students, and community members. Tammy Mayer, NDDPI deputy director for the Office of Assessment, and Schauer met with their school team to enhance the elements of stakeholder engagement in the proposal to ensure the stakeholder conversations necessary to make the proposal viable would occur.
Schauer was able to help early applicants see and address missing elements; however, as other proposals continued to be submitted, and a larger pool of staff began reviewing them, it became clear that a more formal and standardized vetting system was needed to provide consistent feedback. To address this need, NDDPI enlisted REL Central to support them in developing rubrics and processes for reviewing and scoring innovative education programs, including the planning proposal.
“While we wanted our rubric to be flexible, we also needed a way to ensure plans were meeting the intent of the law and that we had a basis for offering suggested improvements,” said Schauer.
To develop the rubric, Schauer joined Ann Ellefson, NDDPI academic support director, Mayer, and REL Central staff to arrange and operationalize the key elements required of North Dakota education agencies to provide as part of their planning proposal. Those elements include what the vision or rational for the innovative education program is, how it will engage stakeholders in planning, how it will receive approval from its school district board, and how it will establish professional development for staff. The NDDPI team then reviewed and chose topic priorities necessary for completing each key element, rating their importance and adding them to a checklist to help the NDDPI staff score and provide feedback on proposals. For example, the NDDPI staff decided that the topic priority that proposals should address for the key element “stakeholder engagement” is whether a concrete plan exists for gathering stakeholder input on possible innovations to explore and adopt. Finally, the team decided on a scoring system for the topic priorities that included whether each priority “Meets Criteria,” “Needs Revisions,” or is rated “Evidence not Found.”
According to Ellefson, the rating system does not reject improvement plans, but provides a framework for NDDPI staff to provide suggestions that applicants can use to help address challenges to program implementation, sustainability, and effectiveness.
The new rubric is just the first phase of the collaboration between NDDPI and REL Central that will ultimately provide NDDPI with a means to review districts” innovative education program proposals and implementation applications, resources to support local evaluation of innovation programs, and the knowledge to develop similar guides for future NDDPI initiatives.
“When you are dealing with an issue as important as student learning and growth, the question must be ‘how is our education system growing with them?’” said Ellefson. “This program and rubric are a great step in making certain our innovations are aligned with the needs of our students, today and tomorrow.”
