Accountability Community of Practice
REL-Mid Atlantic is hosting a community of practice for state education agencies that are seeking to refine and improve school accountability measures in 2022 and beyond. The disruptions of the pandemic will require most states to make changes to the way they assess school performance and identify low-performing schools. To support states in developing creative solutions for 2022 and future years, IES’s new community of practice will bring state agencies together with measurement experts to develop and refine school performance indicators that are robust to the data disruptions of the last two years and valid, reliable, and actionable for years to come.
NJDOE, DC Office of the State Superintendent of Education (OSSE), Maryland State Department of Education (MSDE), AIR, Pennsylvania Department of Education (PDE), Nevada DOE, Delaware DOE, North Dakota University System, Policy Studies, Tennessee DOE, Pittsburgh Public Schools, School District of Philadelphia (SDP), Nebraska Department of Education (NDE), Hawaii DOE, REL West, REL Appalachia
This project aims to increase the robustness of the Pennsylvania Department of Education’s (PDE’s) accountability system to measurement error, thereby improving its accuracy. The REL team will apply a Bayesian model to historical subgroup data and compare the reliability of the stabilized Bayesian proficiency rates to the un-stabilized proficiency rates typically used to inform Targeted Support and Improvement (TSI) and Additional Targeted Support and Improvement (ATSI) determinations. If the stabilized subgroup rates prove more reliable than the corresponding un-stabilized subgroup rates, REL staff will then apply the Bayesian approach to 2022 proficiency data, allowing PDE to include those results into the TSI and ATSI determinations in late 2022.
The Accountability Community of Practice is working with NJDOE to ensure that the measures of school ELP used in its ESSA accountability are reliable and valid in the wake of COVID-related assessment disruptions and for the long term. The REL is providing support for the design of diagnostic analyses to assess potential biases in the schoolwide measures of English language proficiency (ELP) due to missing data from testing disruptions in 2020 and 2021, assistance in interpreting the findings from the proposed diagnostic analyses, providing technical specifications to impute missing data and weight existing data on ELP measures to be representative of each school’s full population of English learners, and coaching and dissemination support on how to effectively describe the proposed alternative ELP indicator weights system and the techniques used to handle missing student-level ELP data for a non-technical audience and stakeholders.
The Accountability Community of Practice is working with NJDOE to explore Bayesian methods, and to evaluate whether using Bayesian methods would improve the reliability of the accountability data, especially when applied to smaller subgroups. The project will provide training, code and other coding resources, and an educator-friendly infographic to NJDOE technical program support staff. Short-term outcomes include a better understanding of Bayesian methods and measures, distribution of an infographic on Bayesian stabilization, and improved capacity to implement refinements to the NJDOE accountability system. Medium-term outcomes include an increase in the reliability of school-performance measures used in NJ’s accountability system, and successful communication of the merits of those measures to NJ educators.
No projects at this time.
REL Mid-Atlantic Logic Model (128 KB)