Stay Up-to-Date:

Accountability in the ESSA era About the Accountability in the ESSA Era Alliance

The Every Student Succeeds Act (ESSA) gives states discretion to design their school accountability systems while mandating that they incorporate measures in addition to proficiency on state assessments. States must select those measures, which can include any of a wide range of indicators of student success or school quality. This alliance brings together state education agencies and interested staff from school districts, supporting them in developing and refining indicators of school and student performance.

What Stakeholders Say

Developing and Validating Socio-Emotional Learning Measures

  • Brandee Tate, formerly with the District of Columbia Public Schools
  • Elizabeth Kim, District of Columbia Public Schools

Featured Video

The REL Mid-Atlantic: Accountability in the ESSA Era Research Alliance

Schools Targeted for Improvement icon Why school accountability systems disproportionately identify middle schools' SWD subgroups for TSI (1.15 MB)
The purpose of this study was to understand why middle schools in two mid-Atlantic states were apparently disproportionately identified for targeted support and improvement (TSI) based on the performance of their students with disabilities (SWD) subgroups. The study used publicly available data to examine differences across school levels – elementary, middle, and high – and between all students and SWDs. It is relevant to compare all students to SWDs because accountability systems designate TSI schools as those with subgroups of students that perform poorly relative to all students. The study focused on two aspects of school accountability systems: (1) the number of schools in each school level in which enough SWDs took state assessments for the school to be held accountable for the academic proficiency of its SWD subgroup, and (2) the average performance on accountability indicators, by school level and subgroup. In both states, SWDs in middle schools were over 20 percentage points more likely to take state assessments than were SWDs in elementary or high schools, meaning that middle schools’ SWD subgroups were substantially more likely to meet states’ minimum sample size requirements for including academic performance in accountability scores. Also, SWD subgroups across school levels consistently performed worse than all students on academic proficiency accountability indicators. Taken together, these findings suggest that middle schools’ SWD subgroups are more likely than elementary or high school SWD subgroups to be identified for TSI because their accountability scores are more likely to include academic proficiency indicator scores – which tend to be substantially lower than the all students group’s academic proficiency indicator scores. This research suggests that when designing school accountability systems, state education agencies may wish to consider how sample size affects estimates of subgroups’ performance, and in particular how sample size exclusions may mask poor performance for small subgroups.

Development of a School Survey and Index as a School Performance Measure in Maryland: A REL–MSDE Research Partnership Development of a School Survey and Index as a School Performance Measure in Maryland: A REL–MSDE Research Partnership (803 KB)
This project expanded on previous work with Maryland on measuring school climate. In this follow-on work, the REL worked with MSDE to build its internal capacity to effectively employ the newly developed measure of school climate and its associated index by analyzing data from the spring 2019 survey. REL staff worked with MSDE staff to build skills to independently validate and revise the survey, develop a school climate index, and develop school-level reports on the student and staff surveys.

The REL will facilitate a community of practice with state accountability officials to discuss the development of exit rules for Comprehensive School Improvement (CSI) and Targeted Support and Improvement (TSI) schools. The REL will review ESSA plans from states across the country and facilitate virtual meetings with state education agency staff in the mid-Atlantic region to examine the various options for establishing exit rules.
This study aims to develop measures of the effectiveness of individual high schools in helping their students achieve long-term success. Specifically, the REL will help the District of Columbia (DC) Office of the State Superintendent for Education (OSSE) measure each DC high school’s power to promote high school graduation and college enrollment. Similar to growth models, these “promotion power” models generate a relative performance index on which schools can be compared. Rather than simply examining college enrollment rates of high school graduates, these models will allow OSSE to measure the impact of each high school on college enrollment.

NJDOE requested the REL’s assistance to support and help develop the department's capacity to interpret measures of school-level performance before and after the shortening of its statewide student assessments. The REL is examining school-level variation in school performance measures over time to assess the extent to which the change in assessment is related to the reliability of school-level measures.

This study will assist DCPS by exploring the properties and uses of data from their districtwide survey of social and emotional learning (SEL). The survey assesses a range of SEL competencies, including perseverance, self-management, self-efficacy, social awareness, rigorous expectations, and sense of belonging. The study will analyze existing administrative and survey data for students, parents, and teachers who were eligible to complete the survey. The results will help inform how DCPS can effectively use its data to reach and track progress towards its time-sensitive strategic goals.

The study’s key research questions are:

  1. How do SEL competencies evolve with age and grade for different types of students?
  2. To what extent do SEL competencies and individual students’ growth in SEL competencies vary within and across schools?
  3. How do SEL measures complement other data that DCPS has access to for predicting future outcomes related to its strategic goals, such as test scores, graduation, absences, and successful re-enrollment?
  4. How do survey-based SEL measures align between various types of respondents (students, parents, and teachers)?
New Jersey is interested in analyses that will provide the state with better comparative information about the three assessments it currently allows for purposes of granting a high-school equivalency certificate. This study will inform oversight of the assessments, including the setting of cut scores and policies that could address barriers to access to some exams for certain groups.

The design of information displays like school report cards can have a substantial effect on users. The REL is working with the District of Columbia Office of the State Superintendent of Education to test the impacts of different design factors. In this experiment, participants will view report cards that can vary in the factors we are investigating. Each participant can view up to two visually distinct report card designs. Across the entire sample, we will examine all possible combinations of design choices. After displaying each report card, we will measure parents' comprehension of information presented in the report card and their self-reported evaluations of the designs. The findings from this study will provide evidence of how different school report card designs affect usability.

REL researchers will explore the properties of Pennsylvania’s school climate surveys with the aim of developing a summary index of school climate that could be calculated for each school. This project will provide diagnostic information that is useful to local educators in their school improvement efforts.

Schools Targeted for Improvement icon Why school accountability systems disproportionately identify middle schools' SWD subgroups for TSI (1.15 MB)
The purpose of this study was to understand why middle schools in two mid-Atlantic states were apparently disproportionately identified for targeted support and improvement (TSI) based on the performance of their students with disabilities (SWD) subgroups. The study used publicly available data to examine differences across school levels – elementary, middle, and high – and between all students and SWDs. It is relevant to compare all students to SWDs because accountability systems designate TSI schools as those with subgroups of students that perform poorly relative to all students. The study focused on two aspects of school accountability systems: (1) the number of schools in each school level in which enough SWDs took state assessments for the school to be held accountable for the academic proficiency of its SWD subgroup, and (2) the average performance on accountability indicators, by school level and subgroup. In both states, SWDs in middle schools were over 20 percentage points more likely to take state assessments than were SWDs in elementary or high schools, meaning that middle schools' SWD subgroups were substantially more likely to meet states' minimum sample size requirements for including academic performance in accountability scores. Also, SWD subgroups across school levels consistently performed worse than all students on academic proficiency accountability indicators. Taken together, these findings suggest that middle schools' SWD subgroups are more likely than elementary or high school SWD subgroups to be identified for TSI because their accountability scores are more likely to include academic proficiency indicator scores – which tend to be substantially lower than the all students group's academic proficiency indicator scores. This research suggests that when designing school accountability systems, state education agencies may wish to consider how sample size affects estimates of subgroups' performance, and in particular how sample size exclusions may mask poor performance for small subgroups.

Development of a School Survey and Index as a School Performance Measure in Maryland: A REL–MSDE Research Partnership Development of a School Survey and Index as a School Performance Measure in Maryland: A REL–MSDE Research Partnership (803 KB)
This project expanded on previous work with Maryland on measuring school climate. In this follow-on work, the REL worked with MSDE to build its internal capacity to effectively employ the newly developed measure of school climate and its associated index by analyzing data from the spring 2019 survey. REL staff worked with MSDE staff to build skills to independently validate and revise the survey, develop a school climate index, and develop school-level reports on the student and staff surveys.

The REL conducted a descriptive analysis to examine how specific aspects of the accountability system rules in two mid-Atlantic states affect the types of schools identified for support for student subgroups. The report found that SWDs in middle schools were over 20 percentage points more likely to take state assessments than were SWDs in elementary or high schools, meaning that middle schools’ SWD subgroups were substantially more likely to meet states’ minimum sample size requirements for including academic performance in accountability scores.

Schools Targeted for Improvement icon Schools Targeted for Improvement: Are Small Sample Sizes Masking Poor School Performance? (479 KB)
This infographic summarizes the findings from the report Why School Accountability Systems Disproportionately Identify Middle Schools' SWD Subgroups for TSI. It highlights how specific aspects of the accountability system rules can affect the types of schools identified for support for student subgroups.

Using Chronic Absenteeism for School Accountability Using Chronic Absenteeism for School Accountability (798 KB)
The REL provided support to the Pennsylvania Department of Education (PDE) to develop its capacity on analytic issues related to chronic absenteeism, one of PDE’s School Quality and Student Success indicators identified in the state’s ESSA plan. This project improved PDE staff’s knowledge about how other state ESSA plans measure chronic absenteeism; about how well Pennsylvania’s measure usefully differentiates among schools; and about how potential variations in the planned measure of chronic absenteeism would alter the list of identified schools. The REL taught PDE staff to conduct their own analyses of their measures, enhancing their ability to inform potential decisions on the refinement of the state’s measures in future years.

Like other states across the country, Maryland is including additional measures of school performance, such as a measure of school climate, in its ESSA plan. The REL assisted the Maryland State Department of Education (MSDE) with selecting and refining school climate surveys for school staff, parents, and students. The REL provided coaching and technical support towards analyzing pilot survey data and interpreting survey results.

Infographic on the Importance of School Climate School Climate Infographic (3.15 MB)
School climate data can drive school improvement. This REL infographic discusses why school climate is important and how school climate can be measured.

The REL assisted the District of Columbia Public Schools (DCPS) in its analysis of data from a survey designed to assess SEL competencies and helped DCPS develop reliable and valid scales and indices of the resulting data. Review a recent presentation to learn more about this work.

During the workshop, the REL consolidated and disseminated evidence on best practices for communicating school performance data to parents, taxpayers, educators, and other stakeholders through online school report cards. The workshop focused on three themes: (1) content, (2) design, and (3) process. A recording and highlights of the workshop can be found here.

Research Alliance Facilitator: Brian Gill, PhD, JD
Name Title Organizational affiliation Organization location
Chandra Haislet Director of Accountability and Data Systems Maryland State Department of Education Maryland
Rosemary Hughes Special Advisor to the Secretary on School Improvement Pennsylvania Department of Education Pennsylvania
Chantel Janiszewski Accountability and Performance Management Delaware Department of Education Delaware
Donna Johnson Strategic Planning & Performance Officer/ Director of Accountability Office of the State Superintendent of Education District of Columbia
Erin Kupferberg Senior Manager of the Financial and Academic Quality (FAQ) Team DC Public Charter School Board District of Columbia
Jessica Merville Director of Performance Management New Jersey Department of Education New Jersey
Gene Pinkard, Jr. Assistant Director, Practice and Leadership The Aspen Institute District of Columbia
Adam Schott Special Assistant to the Secretary Pennsylvania Department of Education Pennsylvania
Dara Shaw Executive Director, Research and Strategic Data Use Maryland State Department of Education Maryland
Interested in joining us or learning more? Contact us.

Goals

  • Help develop, refine, analyze, and interpret school climate measures.
  • Assess the extent to which high school equivalency measures predict student outcomes.
  • Develop thoughtful and strategic ways of using school-climate measures in accountability frameworks and school improvement.
  • Explore school performance measures to enhance the breadth and depth of accountability frameworks.
  • Provide a forum to share lessons about the design, development, and implementation of accountability systems and measures to improve their efficacy.

Upcoming Events

Recent Events