Stay Up-to-Date:

Accountability in the ESSA era About the Accountability in the ESSA Era Alliance

The Every Student Succeeds Act (ESSA) gives states discretion to design their school accountability systems while mandating that they incorporate measures in addition to proficiency on state assessments. States must select those measures, which can include any of a wide range of indicators of student success or school quality. This alliance brings together state education agencies and interested staff from school districts, supporting them in developing and refining indicators of school and student performance.

What Stakeholders Say

Developing and Validating Socio-Emotional Learning Measures

  • Brandee Tate, formerly with the District of Columbia Public Schools
  • Elizabeth Kim, District of Columbia Public Schools

Featured Video

The REL Mid-Atlantic: Accountability in the ESSA Era Research Alliance

OSSE Promotion Power cover Using Promotion Power to Identify High School Effectiveness (1.3 MB)
Check out a new infographic on our study to examine the promotion power of public high schools in DC- an approach that accounts for factors outside a school's control when examining the school's effectiveness, such as how prepared students are coming into high school and other background characteristics.

Pittsburgh Board of Education on Changes in Academic Achievement During Remote Instruction cover Presentation to the Pittsburgh Board of Education on Changes in Academic Achievement During Remote Instruction (739 KB)
After shifting to remote learning at the beginning of the Covid-19 pandemic, Pittsburgh Public Schools sought assistance in analyzing how student achievement has changed during the pandemic and how this varies based on students' demographic characteristics. The district was also eager to understand how students' access and use online learning applications and how this varies across students, teachers, and schools. The REL helped analyze remote learning data to help school leaders and teachers assess which student groups and schools are struggling with accessing and using online learning applications and enable the district to target interventions to assist them. This presentation to the Pittsburgh Board of Public Education and methodology memo summarizes the findings. Watch a recording of the presentation from the committee meeting, available here: https://www.pghschools.org/Page/4744

Infographic on haracteristics of High School Equivalency Exams cover New Infographic on Fostering Social and Emotional Learning Competencies (1.5 MB)
Check out a new infographic shares findings from a REL Mid-Atlantic study that explains why schools or districts should measure SEL competencies, what they should measure, and how to get started measuring and developing these competencies for their students.

Infographic on haracteristics of High School Equivalency Exams cover Infographic on Characteristics of High School Equivalency Exams (1.9 MB)
High school equivalency exams are tests which students who do not finish high school can take to demonstrate their skills and knowledge and earn a high school equivalency credential. This infographic shares findings from a REL Mid-Atlantic study that investigated characteristics of high school equivalency exams in New Jersey and the students who take them. States considering changing the tests they offer or the passing scores for these tests can use this information to understand how these exams compare.

Report on Characteristics and Performance of High School Equivalency cover Report on Characteristics and Performance of High School Equivalency Exam Takers in New Jersey (1.0 MB)
Since 2014 the New Jersey Department of Education has offered three high school equivalency (HSE) exams for nongraduates seeking credentials. This study used data on exam takers who had been grade 8 students in a New Jersey public school between 2008/09 and 2013/14 and who had attempted at least one HSE exam in New Jersey between March 2014 and December 2018. It analyzed how the characteristics of exam takers differ across exams and from the characteristics of non–exam takers, how the performance of exam takers with similar backgrounds varies, and how a recent reduction in the passing threshold for two of the exams affected passing rates.

Using School Climate Index Data to Improve Social-Emotional Learning, Student Engagement and Support, and School Safety cover Using School Climate Index Data to Improve Social-Emotional Learning, Student Engagement and Support, and School Safety (611 KB)
Students, their families, and educators all desire a safe and supportive school climate. Research has demonstrated that school climate can influence student outcomes including academic achievement, school success, effective violence prevention, students' healthy development, and teacher retention. Check out this new fact sheet to learn about the recent findings in a REL project with Pennsylvania schools.

Using a survey of social and emotional learning and school climate to inform decisionmaking cover Using a survey of social and emotional learning and school climate to inform decisionmaking
The REL worked with the District of Columbia Public Schools (DCPS) to conduct analyses to enhance its School Leader IMPACT system. One of these recommendations was to consider incorporating an additional measure of principal performance based on perceptions of school staff. This study will provide DCPS timely information about whether and how staff input could be incorporated into the School Leader IMPACT measures.

Using experiments to design report cards on school performance cover Using experiments to design report cards on school performance (770 KB)
Well-designed report cards on school performance can effectively communicate information and support parents in engaging with their children's schools. Experiments are one way to gather feedback on report card designs. Here we highlight some considerations for those interested in this approach.

The Effect of School Report Card Design on Usability, Understanding, and Satisfaction cover The Effect of School Report Card Design on Usability, Understanding, and Satisfaction
Education policymakers view transparency and accountability as critical to the success of schools. To support these goals, the District of Columbia Office of the State Superintendent of Education (OSSE) has developed an online school report card for communicating information about the characteristics and performance of schools. To support OSSE's interest in making report cards more usable, this study assessed the effect of different designs on how easy the report cards are to use and understand, how easy it is to find information in them, and whether users would recommend the site to others.

The Reliability of Shorter Assessments in New Jersey for Group-Level Inferences cover The Reliability of Shorter Assessments in New Jersey for Group-Level Inferences (4 MB)
Education policymakers must balance the reliability of assessments to measure academic knowledge and skills with the burdens that assessments place upon students, teachers, and schools. In 2019, New Jersey began using the New Jersey Student Learning Assessments (NJSLA), shorter assessments based on the Partnership for Assessment of Readiness for College and Careers (PARCC). Regional Educational Laboratory researchers examined the reliability of test results for the NJSLA by comparing results at the school, test, and subgroup levels from 2016 to 2019. The findings indicated a high degree of reliability across most measures the researchers examined; during the transition to the NJSLA, the reliability did not decrease for any test results—except the Algebra 2 test—reported by the New Jersey Department of Education. The instability of the Algebra 2 results was most likely not attributable to changes in the assessment but instead to changes in the student population that was required to take the test following a change in the state's testing requirements.

Development of Pennsylvania Department of Education School Climate Index Summary cover Development of Pennsylvania Department of Education School Climate Index Summary (6 MB)
This project explores the properties of Pennsylvania Department of Education school climate surveys with the aim of developing a summary index of school climate that could be calculated for each school using the surveys. Data to develop measures of school climate were gathered from surveys of students, teachers, other staff, parents, and community members. Survey results are intended to provide Pennsylvania schools and educators with information that can be used for needs assessments, program development, and short- and long-term improvement planning. The measures of school climate resulting from surveys can also be used to compare schools in the state.

Schools Targeted for Improvement icon Why school accountability systems disproportionately identify middle schools' SWD subgroups for TSI (1.15 MB)
The purpose of this study was to understand why middle schools in two mid-Atlantic states were apparently disproportionately identified for targeted support and improvement (TSI) based on the performance of their students with disabilities (SWD) subgroups. The study used publicly available data to examine differences across school levels – elementary, middle, and high – and between all students and SWDs. It is relevant to compare all students to SWDs because accountability systems designate TSI schools as those with subgroups of students that perform poorly relative to all students. The study focused on two aspects of school accountability systems: (1) the number of schools in each school level in which enough SWDs took state assessments for the school to be held accountable for the academic proficiency of its SWD subgroup, and (2) the average performance on accountability indicators, by school level and subgroup. In both states, SWDs in middle schools were over 20 percentage points more likely to take state assessments than were SWDs in elementary or high schools, meaning that middle schools’ SWD subgroups were substantially more likely to meet states’ minimum sample size requirements for including academic performance in accountability scores. Also, SWD subgroups across school levels consistently performed worse than all students on academic proficiency accountability indicators. Taken together, these findings suggest that middle schools’ SWD subgroups are more likely than elementary or high school SWD subgroups to be identified for TSI because their accountability scores are more likely to include academic proficiency indicator scores – which tend to be substantially lower than the all students group’s academic proficiency indicator scores. This research suggests that when designing school accountability systems, state education agencies may wish to consider how sample size affects estimates of subgroups’ performance, and in particular how sample size exclusions may mask poor performance for small subgroups.

Development of a School Survey and Index as a School Performance Measure in Maryland: A REL–MSDE Research Partnership Development of a School Survey and Index as a School Performance Measure in Maryland: A REL–MSDE Research Partnership (803 KB)
This project expanded on previous work with Maryland on measuring school climate. In this follow-on work, the REL worked with MSDE to build its internal capacity to effectively employ the newly developed measure of school climate and its associated index by analyzing data from the spring 2019 survey. REL staff worked with MSDE staff to build skills to independently validate and revise the survey, develop a school climate index, and develop school-level reports on the student and staff surveys.

REL Mid-Atlantic is helping the District of Columbia DC PCSB update its Performance Management Framework (PMF) to make it a more effective lever for improving students' outcomes and public transparency. DC PCSB first created the PMF to have a universal set of standards for DC public charter schools serving similar grades. The REL will support activities to gather the community's input and design research-based metrics for the PMF.
After shifting to remote learning at the beginning of the Covid-19 pandemic, Pittsburgh Public Schools is seeking assistance in analyzing how student achievement has changed during the pandemic and how this varies based on students' demographic characteristics. The district is also eager to understand how students' access and use online learning applications and how this varies across students, teachers, and schools. The REL is helping analyze remote learning data to help school leaders and teachers assess which student groups and schools are struggling with accessing and using online learning applications and enable the district to target interventions to assist them.
The REL will facilitate a community of practice with state accountability officials to discuss the development of exit rules for Comprehensive School Improvement (CSI) and Targeted Support and Improvement (TSI) schools. The REL will review ESSA plans from states across the country and facilitate virtual meetings with state education agency staff in the mid-Atlantic region to examine the various options for establishing exit rules.
This study aims to develop measures of the effectiveness of individual high schools in helping their students achieve long-term success. Specifically, the REL will help the District of Columbia (DC) Office of the State Superintendent for Education (OSSE) measure each DC high school’s power to promote high school graduation and college enrollment. Similar to growth models, these “promotion power” models generate a relative performance index on which schools can be compared. Rather than simply examining college enrollment rates of high school graduates, these models will allow OSSE to measure the impact of each high school on college enrollment.

REL researchers will explore the properties of Pennsylvania’s school climate surveys with the aim of developing a summary index of school climate that could be calculated for each school. This project will provide diagnostic information that is useful to local educators in their school improvement efforts.

OSSE Promotion Power cover Using Promotion Power to Identify High School Effectiveness (1.3 MB)
Check out a new infographic on our study to examine the promotion power of public high schools in DC- an approach that accounts for factors outside a school's control when examining the school's effectiveness, such as how prepared students are coming into high school and other background characteristics.

Infographic on haracteristics of High School Equivalency Exams cover New Infographic on Fostering Social and Emotional Learning Competencies (1.5 MB)
Check out a new infographic shares findings from a REL Mid-Atlantic study that explains why schools or districts should measure SEL competencies, what they should measure, and how to get started measuring and developing these competencies for their students.

Infographic on haracteristics of High School Equivalency Exams cover Infographic on Characteristics of High School Equivalency Exams (1.9 MB)
High school equivalency exams are tests which students who do not finish high school can take to demonstrate their skills and knowledge and earn a high school equivalency credential. This infographic shares findings from a REL Mid-Atlantic study that investigated characteristics of high school equivalency exams in New Jersey and the students who take them. States considering changing the tests they offer or the passing scores for these tests can use this information to understand how these exams compare.

Describing Characteristics and Outcomes of High School Equivalency Exam Takers in New Jersey cover Describing Characteristics and Outcomes of High School Equivalency Exam Takers in New Jersey (1.0 MB)
Since 2014 the New Jersey Department of Education has offered three high school equivalency (HSE) exams for nongraduates seeking credentials: the GED, the High School Equivalency Test (HiSET), and the Test Assessing Secondary Completion (TASC). This study used data on exam takers who had been grade 8 students in a New Jersey public school between 2008/09 and 2013/14 and who had attempted at least one HSE exam in New Jersey between March 2014 and December 2018. It analyzed how the characteristics of exam takers differ across exams and from the characteristics of non–exam takers, how the performance of exam takers with similar backgrounds varies, and how a recent reduction in the passing threshold for two of the exams affected passing rates. Among all students who had been grade 8 students in a New Jersey public school during the study years, HSE exam takers completed fewer years of school, were more likely to have been eligible for the national school lunch program in grade 8, and were more likely to identify as Black or Hispanic than non–exam takers. GED takers had received higher grade 8 standardized test scores, were more likely to identify as White, and were less likely to have been eligible for the national school lunch program in grade 8 than HiSET and TASC takers. Under the New Jersey Department of Education's original passing thresholds, exam takers in the study sample were more likely to pass the HiSET and TASC than the GED on the first attempt (after grade 8 standardized test scores were controlled for). However, after the reduction in passing thresholds, the first-attempt passing rate was similar across the three exams. Under the new passing thresholds, two-thirds of GED takers and more than half of HiSET and TASC takers passed on the first attempt, and—when all exam attempts are included—three-quarters of all exam takers ever passed each exam.

Using School Climate Index Data to Improve Social-Emotional Learning, Student Engagement and Support, and School Safety cover Using School Climate Index Data to Improve Social-Emotional Learning, Student Engagement and Support, and School Safety (611 KB)
Students, their families, and educators all desire a safe and supportive school climate. Research has demonstrated that school climate can influence student outcomes including academic achievement, school success, effective violence prevention, students' healthy development, and teacher retention. Check out this new fact sheet to learn about the recent findings in a REL project with Pennsylvania schools.

Using experiments to design report cards on school performance cover Using experiments to design report cards on school performance (770 KB)
Well-designed report cards on school performance can effectively communicate information and support parents in engaging with their children's schools. Experiments are one way to gather feedback on report card designs. Here we highlight some considerations for those interested in this approach.

This study assisted DCPS by exploring the properties and uses of data from their districtwide survey of social and emotional learning (SEL). The survey assigned a range of SEL competencies, including perseverance, self-management, self-efficacy, social awareness, rigorous expectations, and sense of belonging. The study analyzed existing administrative and survey data for students, parents, and teachers who were eligible to complete the survey. The results helped inform how DCPS can effectively use its data to reach and track progress towards its time-sensitive strategic goals.

The study's key research questions are:

  1. How do SEL competencies evolve with age and grade for different types of students?
  2. To what extent do SEL competencies and individual students' growth in SEL competencies vary within and across schools?
  3. How do SEL measures complement other data that DCPS has access to for predicting future outcomes related to its strategic goals, such as test scores, graduation, absences, and successful re-enrollment?
  4. How do survey-based SEL measures align between various types of respondents (students, parents, and teachers)?

The design of information displays like school report cards can have a substantial effect on users. The REL worked with the District of Columbia Office of the State Superintendent of Education to test the impacts of different design factors. In this experiment, participants viewed report cards that varied in the factors we investigated. Each participant viewed up to two visually distinct report card designs. Across the entire sample, we examined all possible combinations of design choices. After displaying each report card, we measured parents' comprehension of information presented in the report card and their self-reported evaluations of the designs. The findings from this study provided evidence of how different school report card designs affect usability.

NJDOE requested the REL's assistance to support and help develop the department's capacity to interpret measures of school-level performance before and after the shortening of its statewide student assessments. The REL examined school-level variation in school performance measures over time to assess the extent to which the change in assessment is related to the reliability of school-level measures.

Development of Pennsylvania Department of Education School Climate Index Summary cover Development of Pennsylvania Department of Education School Climate Index Summary (6 MB)
This project explores the properties of Pennsylvania Department of Education school climate surveys with the aim of developing a summary index of school climate that could be calculated for each school using the surveys. Data to develop measures of school climate were gathered from surveys of students, teachers, other staff, parents, and community members. Survey results are intended to provide Pennsylvania schools and educators with information that can be used for needs assessments, program development, and short- and long-term improvement planning. The measures of school climate resulting from surveys can also be used to compare schools in the state.

Schools Targeted for Improvement icon Why school accountability systems disproportionately identify middle schools' SWD subgroups for TSI (1.15 MB)
The purpose of this study was to understand why middle schools in two mid-Atlantic states were apparently disproportionately identified for targeted support and improvement (TSI) based on the performance of their students with disabilities (SWD) subgroups. The study used publicly available data to examine differences across school levels – elementary, middle, and high – and between all students and SWDs. It is relevant to compare all students to SWDs because accountability systems designate TSI schools as those with subgroups of students that perform poorly relative to all students. The study focused on two aspects of school accountability systems: (1) the number of schools in each school level in which enough SWDs took state assessments for the school to be held accountable for the academic proficiency of its SWD subgroup, and (2) the average performance on accountability indicators, by school level and subgroup. In both states, SWDs in middle schools were over 20 percentage points more likely to take state assessments than were SWDs in elementary or high schools, meaning that middle schools' SWD subgroups were substantially more likely to meet states' minimum sample size requirements for including academic performance in accountability scores. Also, SWD subgroups across school levels consistently performed worse than all students on academic proficiency accountability indicators. Taken together, these findings suggest that middle schools' SWD subgroups are more likely than elementary or high school SWD subgroups to be identified for TSI because their accountability scores are more likely to include academic proficiency indicator scores – which tend to be substantially lower than the all students group's academic proficiency indicator scores. This research suggests that when designing school accountability systems, state education agencies may wish to consider how sample size affects estimates of subgroups' performance, and in particular how sample size exclusions may mask poor performance for small subgroups.

Development of a School Survey and Index as a School Performance Measure in Maryland: A REL–MSDE Research Partnership Development of a School Survey and Index as a School Performance Measure in Maryland: A REL–MSDE Research Partnership (803 KB)
This project expanded on previous work with Maryland on measuring school climate. In this follow-on work, the REL worked with MSDE to build its internal capacity to effectively employ the newly developed measure of school climate and its associated index by analyzing data from the spring 2019 survey. REL staff worked with MSDE staff to build skills to independently validate and revise the survey, develop a school climate index, and develop school-level reports on the student and staff surveys.

The REL conducted a descriptive analysis to examine how specific aspects of the accountability system rules in two mid-Atlantic states affect the types of schools identified for support for student subgroups. The report found that SWDs in middle schools were over 20 percentage points more likely to take state assessments than were SWDs in elementary or high schools, meaning that middle schools’ SWD subgroups were substantially more likely to meet states’ minimum sample size requirements for including academic performance in accountability scores.

Schools Targeted for Improvement icon Schools Targeted for Improvement: Are Small Sample Sizes Masking Poor School Performance? (479 KB)
This infographic summarizes the findings from the report Why School Accountability Systems Disproportionately Identify Middle Schools' SWD Subgroups for TSI. It highlights how specific aspects of the accountability system rules can affect the types of schools identified for support for student subgroups.

Using Chronic Absenteeism for School Accountability Using Chronic Absenteeism for School Accountability (798 KB)
The REL provided support to the Pennsylvania Department of Education (PDE) to develop its capacity on analytic issues related to chronic absenteeism, one of PDE’s School Quality and Student Success indicators identified in the state’s ESSA plan. This project improved PDE staff’s knowledge about how other state ESSA plans measure chronic absenteeism; about how well Pennsylvania’s measure usefully differentiates among schools; and about how potential variations in the planned measure of chronic absenteeism would alter the list of identified schools. The REL taught PDE staff to conduct their own analyses of their measures, enhancing their ability to inform potential decisions on the refinement of the state’s measures in future years.

Like other states across the country, Maryland is including additional measures of school performance, such as a measure of school climate, in its ESSA plan. The REL assisted the Maryland State Department of Education (MSDE) with selecting and refining school climate surveys for school staff, parents, and students. The REL provided coaching and technical support towards analyzing pilot survey data and interpreting survey results.

Infographic on the Importance of School Climate School Climate Infographic (3.15 MB)
School climate data can drive school improvement. This REL infographic discusses why school climate is important and how school climate can be measured.

The REL assisted the District of Columbia Public Schools (DCPS) in its analysis of data from a survey designed to assess SEL competencies and helped DCPS develop reliable and valid scales and indices of the resulting data. Review a recent presentation to learn more about this work.

During the workshop, the REL consolidated and disseminated evidence on best practices for communicating school performance data to parents, taxpayers, educators, and other stakeholders through online school report cards. The workshop focused on three themes: (1) content, (2) design, and (3) process. A recording and highlights of the workshop can be found here.

Research Alliance Facilitator: Brian Gill, PhD, JD
Name Title Organizational affiliation Organization location
Chandra Haislet Director of Accountability and Data Systems Maryland State Department of Education Maryland
John Iko Planning Associate II, Office of Performance Management New Jersey Department of Education New Jersey
Chantel Janiszewski Accountability and Performance Management Delaware Department of Education Delaware
Donna Johnson Strategic Planning & Performance Officer/ Director of Accountability Office of the State Superintendent of Education District of Columbia
Erin Kupferberg Senior Manager of the Financial and Academic Quality (FAQ) Team DC Public Charter School Board District of Columbia
Jessica Merville Director of Performance Management New Jersey Department of Education New Jersey
Gene Pinkard, Jr. Assistant Director, Practice and Leadership The Aspen Institute District of Columbia
Adam Schott Special Assistant to the Secretary Pennsylvania Department of Education Pennsylvania
Dara Shaw Executive Director, Research and Strategic Data Use Maryland State Department of Education Maryland
Interested in joining us or learning more? Contact us.

Goals

  • Help develop, refine, analyze, and interpret school climate measures.
  • Assess the extent to which high school equivalency measures predict student outcomes.
  • Develop thoughtful and strategic ways of using school-climate measures in accountability frameworks and school improvement.
  • Explore school performance measures to enhance the breadth and depth of accountability frameworks.
  • Provide a forum to share lessons about the design, development, and implementation of accountability systems and measures to improve their efficacy.

Upcoming Events

Recent Events