Skip Navigation

Home REL Mid-Atlantic About REL Mid-Atlantic Accountability Community of Practice

Accountability Community of Practice

REL-Mid Atlantic is hosting a community of practice for state education agencies that are seeking to refine and improve school accountability measures in 2022 and beyond. The disruptions of the pandemic will require most states to make changes to the way they assess school performance and identify low-performing schools. To support states in developing creative solutions for 2022 and future years, IES’s new community of practice will bring state agencies together with measurement experts to develop and refine school performance indicators that are robust to the data disruptions of the last two years and valid, reliable, and actionable for years to come.

Partner Organizations:
NJDOE, DC Office of the State Superintendent of Education (OSSE), Maryland State Department of Education (MSDE), AIR, Pennsylvania Department of Education (PDE), Nevada DOE, Delaware DOE, North Dakota University System, Policy Studies, Tennessee DOE, Pittsburgh Public Schools, School District of Philadelphia (SDP), Nebraska Department of Education (NDE), Hawaii DOE, REL West, REL Appalachia

Current Projects

The Accountability Community of Practice is working with NJDOE to explore Bayesian methods, and to evaluate whether using Bayesian methods would improve the reliability of the accountability data, especially when applied to smaller subgroups. The project will provide training, code and other coding resources, and an educator-friendly infographic to NJDOE technical program support staff. Short-term outcomes include a better understanding of Bayesian methods and measures, distribution of an infographic on Bayesian stabilization, and improved capacity to implement refinements to the NJDOE accountability system. Medium-term outcomes include an increase in the reliability of school-performance measures used in NJ’s accountability system, and successful communication of the merits of those measures to NJ educators.
Under this project, REL Mid-Atlantic is providing DC Public Charter School Board (PCSB) with technical support as it develops a new framework of school performance measures. We are advising PCSB about issues related to the reliability, validity, and robustness of measures; and exploring implications of different approaches for combining and weighting the measures—with particular attention to PCSB’s desire to promote equity in its framework. The project team will provide this feedback to DC PCSB over the course of 3 to 4 virtual sessions during the first half of 2023 as PCSB seeks to establish its new framework of school performance measures.
NJDOE would like to ensure that the measures of school performance used in its ESSA accountability system are reliable, both in the wake of COVID-related assessment disruptions and for the long term. Recognizing that measurement error could challenge reliability, the state is exploring the potential of a Bayesian stabilization approach to improve the accuracy of accountability measures. In an existing project, REL staff are providing training to NJDOE staff on how to implement Bayesian stabilization. In this complementary project, the REL team will design and test Bayesian stabilization models for a public report that will document the methods and their implications. We will apply Bayesian hierarchical modeling to historical subgroup-by-school-level data to stabilize accountability measures, with a short-term goal for NJDOE to develop a better understanding of how modifications to its accountability measures could increase reliability and accuracy.

REL Mid-Atlantic is partnering with the New Jersey Department of Education (NJDOE) to evaluate the state’s current accountability models and methods for tracking progress and achievement in English language proficiency (ELP). At the student level, the project team is analyzing how well NJDOE’s current ELP student growth model aligns with observed growth trajectories in the state, particularly in the wake of the COVID-19 pandemic. At the school level, the project team is seeking to evaluate NJDOE’s school accountability measure as a valid and reliable indicator of a school’s contribution to students’ ELP growth. At both the student and the school levels, NJDOE seeks to collect evidence that may inform its future decisionmaking about how best to set ambitious achievement standards for students and schools, identify students and schools that need support to meet these standards, and provide meaningful support and guidance to schools and educators whose students are identified for support.

Education agencies need good measures of school performance to know which schools and subgroups of students need support the most. Identifying subgroups most in need has been a long-standing challenge due to the instability of conventional measures for small subgroups of students. With respect to small subgroups, states have faced a difficult tradeoff between accuracy and equity: Excluding the smallest subgroups from accountability makes these students and their needs invisible, compromising equity; but including small subgroups risks making decisions based on unreliable data due to the large amount of random error implicit in small numbers of students. Key findings from our 2023 report, using data from the Pennsylvania Department of Education, demonstrate how Bayesian stabilization increases the reliability of school performance measures, especially for small subgroups of students—making it possible to simultaneously promote accuracy and equity. But not every state has the internal expertise or resources needed to implement stabilization. To make stabilization available to state and local education agencies across the country, we’re developing a free tool, hosted by the U.S. Department of Education, that will allow them to stabilize their own performance measures. The Accuracy for Equity (a4e) tool will be useful not only for improving the accuracy of accountability measures, but also for diagnostic purposes in support of equitable outcomes, by giving states, districts, and schools better information on student groups most in need of support.

Completed Projects

This project aimed to increase the robustness of the Pennsylvania Department of Education’s (PDE’s) accountability system to measurement error, thereby improving its accuracy. The REL team applied a Bayesian statistical model to historical subgroup data and compared the reliability of the stabilized Bayesian proficiency rates to the un-stabilized proficiency rates typically used to inform Targeted Support and Improvement (TSI) and Additional Targeted Support and Improvement (ATSI) determinations. Stabilization substantially improved statistical reliability, especially for small subgroups. Education agencies seeking to make the best use of data on subgroups in schools (or any data with small numbers of students) should consider stabilizing their measures to increase reliability and reduce the risk of being fooled by random variation.

Final report: Stabilizing Subgroup Proficiency Results to Improve the Identification of Low-Performing Schools.

REL Mid-Atlantic worked with the New Jersey Department of Education to ensure that the measures of school-level English language proficiency (ELP) used in its ESSA accountability were reliable and valid in the wake of COVID-related assessment disruptions and for the long term. The REL provided support for the design of diagnostic analyses to assess potential biases in the schoolwide measures of ELP due to testing disruptions in 2020 and 2021 and assistance in interpreting the findings. The REL also provided information about other states’ English learner accountability practices as well as technical specifications to weight existing data on ELP measures using a sliding scale approach in order to be representative of each school’s full population of English learners.

Event Type: Training Material | Event Date: March 06, 2024
Time: 3:30-4:30 pm EST Description: REL Mid-Atlantic staff will hold a follow-up question-and-answer session with New Jersey Department of Education staff following stakeholder outreach.
Product Type: Blog | Publication Date: February 29, 2024
Measures of school performance—whether used for high-stakes accountability or for lower-stakes diagnostic purposes—have become increasingly complex and technical in the past three decades. State accountability systems that initially examined only student...
Event Type: Webinar | Event Date: November 15, 2023
Time: 11:00 am – 12:00 pm EST Location: Evidence Summit, U.S. Department of Education Description: In partnership with state and local education agencies, REL Mid-Atlantic has conducted technical support and applied research projects to improve the validity...
Event Type: In-person Event | Event Date: September 29, 2023
Time: 4:30-6:00 pm EDT Location: Society for Research on Educational Effectiveness (SREE) Conference, Arlington, Virginia Description: The Every Student Succeeds Act requires states to report each school’s performance for individual subgroups of students...
Product Type: Infographic | Publication Date: June 2023
This framework offers an improved measurement strategy states and districts can implement to answer three critical questions about school performance: 1. How are students doing? 2. How much does the school contribute to how students are doing? and 3....
Product Type: Blog | Publication Date: June 14, 2023
Attention to historically underserved student groups has long been a focus of federal accountability. In 2022, for the first time in three years, states had to identify low-performing schools, including those where particular subgroups of students are...
Event Type: Training Material | Event Date: June 12, 2023
Building off the insights of previous coaching sessions, the REL team will work with the New Jersey Department of Education to provide a comprehensive overview of sliding scale methodologies for measuring English language proficiency growth.
Event Type: Training Material | Event Date: May 31, 2023
In this session, the REL team plans to train members of the New Jersey Department of Education on how to communicate complex Bayesian methods for school accountability systems to a non-technical audience that may include school principals and teachers...
Product Type: Infographic | Publication Date: May 2023
School performance measures can be misleading—especially when they involve small numbers of students. While attention to historically underserved student groups has long been a focus of federal accountability under the Every Student Succeeds Act, examining...
Event Type: Training Material | Event Date: May 22, 2023
Time: 4:00 pm EDT Description: The REL Mid-Atlantic team will meet with the DC Public Charter School Board (PCSB) to discuss results from the recent data contractor's modeling work.
Event Type: Training Material | Event Date: May 01, 2023
Time: 2:00-3:30 pm EDT Description: Building off the insights of previous coaching sessions, the REL team will work with the New Jersey Department of Education (NJDOE) to reflect on the diagnostic outputs of their historical English language proficiency...
Event Type: Training Material | Event Date: April 03, 2023
Description: REL MA will meet with members of the Maryland State Department of Education to review follow up materials to Ask an Expert 7 which focused on suggested changes to the Expert Review Team (ERT) report template used to communicate information...
Event Type: Training Material | Event Date: April 03, 2023
Time: 12:00 - 1:00 pm EDT Description: REL Mid-Atlantic and the DC Public Charter School Board (PCSB) will discuss key issues related to the weighting of different performance measures and their combination into an overall performance level, including...
Event Type: Training Material | Event Date: March 22, 2023
Time: 12:30-2:00 pm EDT Description: The purpose of this session is for the REL team to synthesize the work that other states are doing to address varying English learner populations across schools and propose approaches that may be more suitable and...
Event Type: Training Material | Event Date: March 07, 2023
REL Mid-Atlantic will assist the Maryland State Department of Education (MSDE) with the development and refinement of a site-visit rubric and report template for Expert Review Teams that conduct school quality inspections in Maryland.
Event Type: Training Material | Event Date: March 07, 2023
Time: 9:00-10:00 am EST Description: The REL and the DC Public Charter School Board discuss key issues related to the weighting of different performance measures and their combination into an overall performance level, including student groups and group...
Event Type: Training Material | Event Date: March 03, 2023
Time: 11:00 am-12:00 pm EST Description: During this coaching session, the REL team and the New Jersey Department of Education (NJDOE) will review the results of additional analyses of diagnostic growth trends in English language proficiency (ELP). Specifically...
Product Type: Descriptive Study | Publication Date: February 2023
The Every Student Succeeds Act (ESSA) requires states to identify schools with low-performing student subgroups for Targeted Support and Improvement (TSI) or Additional Targeted Support and Improvement (ATSI). Random differences between students’ true...
Event Type: Training Material | Event Date: February 24, 2023
The REL team held a homework Q&A call with partners from the New Jersey Department of Education Office of Performance Management to review solutions to the practical assignment on Bayesian methods.
Event Type: Training Material | Event Date: February 23, 2023
Time: 1:00-2:00 pm EST Description: The REL will hold a working session with the DC Public Charter School Board on key issues related to the weighting of different performance measures and their combination into an overall performance level. REL will...
Product Type: Blog | Publication Date: February 21, 2023
In the wake of a pandemic that caused a nationwide nosedive in student achievement and disproportionate harms to students who were already disadvantaged, good measures of school performance may be more important than ever. States are now identifying lowperforming...
Event Type: Training Material | Event Date: February 06, 2023
Time: 4:00-5:00 pm EST Description: REL Mid-Atlantic and partners at the DC Public Charter School Board will hold their introductory technical support session to discuss key issues related to the weighting of different performance measures and their combination...
Event Type: Training Material | Event Date: January 04, 2023
The REL team met with partners from the New Jersey Department of Education Office of Performance Management to review a practical assignment covering Bayesian methods for school accountability. The REL team also trained NJDOE on school-specific time trends...
Product Type: Blog | Publication Date: December 19, 2022
For the first time in three years, states across the country are identifying low-performing schools under the Every Student Succeeds Act (ESSA). One of the long-recognized technical challenges in this process involves the reliability of the measures of...
Event Type: Training Material | Event Date: October 19, 2022
REL Mid-Atlantic will meet with the New Jersey Department of Education (NJDOE) to discuss application of the code to stabilize the school's median student-growth percentiles.
Event Type: Training Material | Event Date: October 07, 2022
Through a follow-up support, REL Mid-Atlantic will supply the New Jersey Department of Education (NJDOE) with code to fit NJDOE scaled school and subgroup growth.
Event Type: Training Material | Event Date: October 05, 2022
REL Mid-Atlantic will assist the New Jersey Department of Education (NJDOE) with improving their plan to calculate average scaled-score changes as a substitute for median student-growth percentiles in their accountability system in the absence of usable...
Product Type: Infographic | Publication Date: September 2022
The accountability provisions of the Every Student Succeeds Act are back in effect after 2 years of pandemic-related disruptions. States need to know that they are using their data appropriately to identify schools for targeted or comprehensive support...
Product Type: Blog | Publication Date: June 21, 2022
School accountability requirements under the Every Student Succeeds Act (ESSA) are back in force in 2022...
Event Type: Training Material | Event Date: April 07, 2022
REL Mid-Atlantic will meet with members of the New Jersey Department of Education (NJDOE) to suggest alternative methods aimed at improving accountability measures. During this meeting REL Mid-Atlantic will guide members of NJDOE through strategies for...
Event Type: Training Material | Event Date: April 01, 2022
REL Mid-Atlantic will assist the New Jersey Department of Education (NJDOE) with improving confidence in English-Language Proficiency (ELP) assessments by recommending changes to the way ELP indicators are measured in CSI/TSI assignments.
Event Type: Training Material | Event Date: March 29, 2022
REL Mid-Atlantic will meet with members of the New Jersey Department of Education (NJDOE) to suggest alternative methods aimed at improving accountability measures. This meeting will focus on Comprehensive Support and Improvement (CSI) exit decisions...
Event Type: Training Material | Event Date: March 17, 2022
REL Mid-Atlantic will meet with members of the the Pennsylvania Department of Education (PDE) to suggest alternative methods for identifying low-performing schools. The team will present on Bayesian methods, which aid in stabilizing measures and mitigate...
Product Type: Blog | Publication Date: February 18, 2022
For the past two years, in recognition of the educational disruptions of the COVID-19 pandemic, the U.S. Department of Education (ED) waived key requirements of the Every Student Succeeds Act related to schools’ accountability for performance. State assessments...

REL Mid-Atlantic Logic Model (57 KB)

REL Mid-Atlantic Logic Model