Skip Navigation
Skip Navigation

Back to Ask A REL Archived Responses

REL Midwest Ask A REL Response

Educator Effectiveness

October 2019

Question:

What research is available on incorporating equity into school accountability?



Response:

Following an established Regional Educational Laboratory (REL) Midwest protocol, we conducted a search for research reports, descriptive studies, and policy overviews on incorporating equity into school accountability. In addition, we searched for research-based resources on “equity walks.” For details on the databases and sources, keywords, and selection criteria used to create this response, please see the Methods section at the end of this memo.

Below, we share a sampling of the publicly accessible resources on this topic. References are listed in alphabetical order, not necessarily in order of relevance. The search conducted is not comprehensive; other relevant references and resources may exist. For each reference, we provide an abstract, excerpt, or summary written by the study’s author or publisher. We have not evaluated the quality of these references, but provide them for your information only.

Research References

American Institutes for Research. (2014). Moving toward equity data review tool: Getting started with equitable access data. Washington, DC: Author. Retrieved from https://eric.ed.gov/?id=ED580425.

From the ERIC abstract: “This tool and its supporting resources are intended to help education leaders understand and assess equitable access data to support a root-cause analysis and, ultimately, draft a State Plan to Ensure Equitable Access to Excellent Educators. The activities in this tool introduce metrics, address staff capacity for analyzing equitable access data, and provide tools to communicate what the data suggest about the equitable access challenges in your state. The tool is intended to help states analyze the policy implications of data, and present the findings to a variety of audiences effectively.”

Bae, S. (2018). Redesigning systems of school accountability: A multiple measures approach to accountability and support. Education Policy Analysis Archives, 26(8). Retrieved from https://eric.ed.gov/?id=EJ1169485.

From the ERIC abstract: “The challenges facing our children in the 21st century are rapidly changing. As a result, schools bear a greater responsibility to prepare students for college, career, and life and must be held accountable for more than just testing and reporting on a narrow set of outcomes aimed at minimum levels of competency. Thus, scholars, educators, and reform advocates are calling for a more meaningful next phase of school accountability, one that promotes continuous support and improvement rather than mere compliance and efforts to avoid punishment (Center for American Progress & CCSSO, 2014; Darling-Hammond, Wilhoit, & Pittenger, 2014). This paper reviews state and district level accountability systems that incorporate a multiple measures approach to accountability and highlights the following features that represent redesigned systems of accountability: 1) broader set of outcome measures, 2) mix of state and local indicators, 3) measures of opportunities to learn, 4) data dashboards, and 5) School Quality Reviews. The paper concludes with guidance for policymakers and practitioners on ways to support the development and implementation of a multiple measures system of accountability so that school accountability becomes synonymous with responsibility for deeper learning and support for continuous improvement.”

Baker, B. D., Di Carlo, M., & Weber, M. (2019). The adequacy and fairness of state school finance systems. Findings from the school finance indicators database, School Year 2015–2016. Washington, DC: Albert Shanker Institute. Retrieved from https://eric.ed.gov/?id=ED596199.

From the ERIC abstract: “When it comes to American education, few policy areas are as misunderstood—or as crucial—as school finance. Over the past several years, a political and empirical consensus has emerged about the importance of equitable and adequate school funding for high-quality K-12 education. Certainly, there are plenty of contentious debates about how education funds should be spent. But regardless of one’s opinions on specific education policies, virtually all of the options for improving America’s schools require investment—particularly for disadvantaged students. We introduce in this report an updated, public database of state school finance measures, and present results for three key measures in this system: effort, adequacy, and progressivity. Our results indicate, as would be expected, that states vary widely on all three measures. There are several states in which educational resources are comparatively adequate and distributed equitably. In general, however, resources in most states tend to be allocated non-progressively or even regressively. That is, higher-poverty districts do not receive more funds—and in some cases receive substantially less—than do lower-poverty districts, even controlling for factors that affect costs, such as regional wage variation, district size, and population density. Moreover, using models that estimate the spending levels required to achieve common outcome goals, we find that the vast majority of states spend well under the levels that would be necessary for their higher-poverty districts to achieve national average test scores. We do not provide state rankings or grades in this report, as the interplay between effort, adequacy, and progressivity is complex. We do, however, include recommendations on how researchers, policymakers, and the public can use our findings, as well as our database, to evaluate state systems and inform debates about improving school finance in the U.S.”

Best, J., & Winslow, E. (2015). Educational equity: Challenges for educator effectiveness. Denver, CO: McREL International. Retrieved from https://eric.ed.gov/?id=ED557602.

From the ERIC abstract: “With increasingly diverse student populations, educational equity is a bigger challenge than ever for public schools across the United States. While federal government, states, and school districts work to identify and address the root causes of equity gaps, efforts are often hampered by a limited body of research-based strategies and approaches that work best. This brief provides an overview for policymakers on addressing equity gaps for vulnerable student populations through educator effectiveness. Specifically, the authors examine three components: (1) recruiting and distributing effective educators; (2) supporting and retaining teachers through targeted professional development; and (3) improving educator evaluation practices to address equity gaps for vulnerable student populations. Included are essential questions to consider and policy recommendations.”

Cowen, J., Goldhaber, D., & Theobald, R. (2017). Teacher equity gaps in Massachusetts (ESE policy brief). Malden, MA: Massachusetts Department of Elementary and Secondary Education. Retrieved from https://eric.ed.gov/?id=ED589944.

From the ERIC abstract: “Effective teachers make a real difference for student learning. But research shows that both in Massachusetts and nationwide, academically struggling students and those from historically low performing subgroups are less likely to be assigned to the teachers who are most likely to generate strong results. This results in missed opportunities to close achievement gaps and increase educational outcomes for all students. This policy brief provides an overview of how effective teachers are identified, summarizes research from around the nation, and analyzes Massachusetts data to address several important questions. It also provides connections to resources available to Massachusetts schools and districts working to eliminate equity gaps.”

Darling-Hammond, L., Bae, S., Cook-Harvey, C. M., Lam, L., Mercer, C., Podolsky, A., et al. (2016). Pathways to new accountability through the Every Student Succeeds Act. Palo Alto, CA: Learning Policy Institute. Retrieved from https://eric.ed.gov/?id=ED574647. Full text available at https://edpolicy.stanford.edu/library/publications/1418.

From the ERIC abstract: “This paper examines the options available to states to redefine their accountability systems as they begin to implement the Every Student Succeeds Act (ESSA). The new law provides the possibility that states can create more balanced systems of support and accountability focused on educating young people so they can become productive, engaged citizens who are prepared for 21st century college and careers. We examine these possibilities, beginning with an overview of the law’s requirements, including its allowances for indicators of school progress, methods of identifying schools for support and intervention, and requirements for the use of evidence-based interventions. We then look more closely at the range of indicators that might be considered in a multiple measures accountability system as evidence of learning, opportunities to learn, and student engagement. Next we discuss how these indicators might be combined to identify schools for intervention and support, and how they could be used within a continuous improvement system that also examines school practices through school visits and observations. We close with a discussion of research supporting evidence-based interventions that may be worth considering to support school improvement in a new accountability system.”

Della Sala, M. R., & Knoeppel, R. C. (2015). Measuring the alignment between states’ finance and accountability policies: The opportunity gap. Education Policy Analysis Archives, 23(61). Retrieved from https://eric.ed.gov/?id=EJ1070464.

From the ERIC abstract: “The research described in this paper expands on attempts to conceptualize, measure, and evaluate the degree to which states have aligned their finance systems with their respective accountability policies. State education finance and accountability policies serve as levers to provide equal educational opportunities for all students--scholars have called for the alignment of education finance and accountability policies as a means for states to meet the demands of educational adequacy. A metric titled the ‘opportunity gap’ was developed, calculated, and tested to represent the degree of misalignment between the equity of states’ finance systems and the intended equity of student performance outcomes defined in accountability policies. School finance and student performance data from nine states were collected for this analysis. Findings indicated that none of the states were delivering simultaneous equity in finance and accountability systems- none of the states provided both equity of finance inputs and equity of student performance outputs. Implications for future research on measuring the alignment between finance and accountability policies are provided by the authors.”

Edley, C., Jr., Koenig, J., Nielsen, N., & Citro, C. (Eds.). (2019). Monitoring educational equity. Washington, DC: National Academies Press. Retrieved from https://eric.ed.gov/?id=ED595713.

From the ERIC abstract: “Disparities in educational attainment among population groups have characterized the United States throughout its history. Education is sometimes characterized as the ‘great equalizer,’ but to date, the country has not found ways to successfully address the adverse effects of socioeconomic circumstances, prejudice, and discrimination that suppress performance for some groups. To ensure that the pursuit of equity encompasses both the goals to which the nation aspires for its children and the mechanisms to attain those goals, a revised set of equity indicators is needed. Measures of educational equity often fail to account for the impact of the circumstances in which students live on their academic engagement, academic progress, and educational attainment. Some of the contextual factors that bear on learning include food and housing insecurity, exposure to violence, unsafe neighborhoods, adverse childhood experiences, and exposure to environmental toxins. Consequently, it is difficult to identify when intervention is necessary and how it should function. A revised set of equity indicators should highlight disparities, provide a way to explore potential causes, and point toward possible improvements. ‘Monitoring Educational Equity’ proposes a system of indicators of educational equity and presents recommendations for implementation. This report also serves as a framework to help policy makers better understand and combat inequity in the United States’ education system. Disparities in educational opportunities reinforce, and often amplify, disparities in outcomes throughout people’s lives. Thus, it is critical to ensure that all students receive comprehensive supports that level the playing field in order to improve the well-being of underrepresented individuals and the nation.”

Epstein, D. (2011). Measuring inequity in school funding. Washington, DC: Center for American Progress. Retrieved from https://eric.ed.gov/?id=ED535988.

From the ERIC abstract: “Low-income children tend to be concentrated in low-income school districts, and these children often attend schools that receive far fewer resources per pupil despite their greater need. Since education is primarily a state responsibility, more than 90 percent of school funding comes from state and local sources, and the federal government provides the rest. Districts have traditionally drawn much of their revenue from local property taxes, which means districts in high-wealth parts of a state are often funded more generously than districts in low-wealth areas. Over time, some states have moved to school finance models in which districts receive more funding from state sources and rely less on local revenue streams. The shift to higher proportions of state funding is aimed at ensuring districts in lower-wealth areas have access to additional resources so funding across districts is more equitable. In other states, however, the level of school funding is still largely driven by local taxes. This paper discusses the differences in per pupil funding across states by highlighting measures of spending and effort. It then examines the problem of intrastate fiscal inequity and surveys some of the different measures that are used to characterize a state’s level of funding equity among districts within a state. It then compares and contrasts the different measures and presents data on states’ fiscal equity using a variety of measures. The data demonstrate that many states are not fairly funding their school districts.”

Jackson, S. A., & Lunenburg, F. C. (2010). School performance indicators, accountability ratings, and student achievement. American Secondary Education, 39(1), 27–44. Retrieved from https://eric.ed.gov/?id=EJ906436.

From the ERIC abstract: “Researchers have been challenged to find school-level characteristics that make a difference in student achievement. This study focused on a diverse sample of 24 middle schools to examine differences between schools rated Exemplary, Recognized, Academically Acceptable, and Academically Unacceptable on four performance indicator dimensions: academic excellence, developmental responsiveness, social equity, and organizational structures. The study also examined relationships between performance indicators and student achievement. A random sample of teachers from each school and the principal provided data on the school’s performance indicator dimensions. School accountability ratings, student achievement scores, and demographic characteristics were obtained from the state department of education. Significant differences were found on all four performance indicator dimensions based on school accountability ratings, as well as relationships between performance indicators and student achievement.”

Note: REL Midwest was unable to locate a link to the full-text version of this resource. Although REL Midwest tries to provide publicly available resources whenever possible, it was determined that this resource may be of interest to you. It may be found through university or public library systems.

Kaufman, J. H., Cannon, J. S., Culbertson, S., Hannan, M. Q., Hamilton, L. S., & Meyers, S. (2018). Raising the bar: Louisiana’s strategies for improving student outcomes. Santa Monica, CA: RAND Corporation. Retrieved from https://eric.ed.gov/?id=ED588644.

From the ERIC abstract: “Louisiana has received recent attention for some of its new education policies and promising early results. In this report, RAND researchers provide an in-depth description of Louisiana’s approach to improving student outcomes since 2012. The authors examine how the state has used various policy levers—including mandates, resource alignment, incentives, and communication and planning processes—to pursue reform strategies in four areas: early childhood education, K-12 academics, K-12 teacher preparation, and graduation pathways. This report sets the stage for more in-depth analyses of Louisiana’s policy implementation, challenges, and student outcomes that might be linked to its reform efforts. Reports detailing these analyses will also provide recommendations on how Louisiana’s and similar efforts might be improved. Key findings from the report include: (1) In each area of the Louisiana’s reform efforts, there is a clarity of vision around what constitutes ‘high-quality’ teaching and learning; (2) When reforms are not part of federal mandates, reform strategies follow a theory of implementation and action that might be more expected to lead to change; (3) State leaders engage in close communication with educators to gather feedback and promote buy-in; (4) LDOE has established partnerships with external stakeholders who have embraced the state’s vision; (5) LDOE has undertaken efforts to align its mandates with incentives, resources, and tools; (6) The resources and incentives provided by the state are intended to promote equity; and (7) LDOE has created an accountability system that uses indicators of both processes and outcomes, with increasing emphasis on outcomes as students get older.”

Knoeppel, R. C., & Della Sala, M. R. (2013). Measuring equity: Creating a new standard for inputs and outputs. Educational Considerations, 40(2), 45–53. Retrieved from https://eric.ed.gov/?id=EJ1006217. Full text available from https://newprairiepress.org/edconsiderations/vol40/iss2/.

From the ERIC abstract: “The purpose of this article is to introduce a new statistic to capture the ratio of equitable student outcomes given equitable inputs. Given the fact that finance structures should be aligned to outcome standards according to judicial interpretation, a ratio of outputs to inputs, or ‘equity ratio,’ is introduced to discern if conclusions can be drawn with regard to the equity of both the financial resources and educational opportunity. In developing this ratio, the authors were interested in knowing if educational outcomes were equitable given equitable inputs. Previous analyses of the equity of finance systems made use of measures of dispersion; yet a more complete understanding of the equity of the system must also include measures of distribution. As such, part of the discussion of the equity ratio will include both an analysis of both the dispersion and the distribution of the results.”

Knoeppel, R. C., & Della Sala, M. R. (2015). Education funding and student outcomes: A conceptual framework for measurement of the alignment of state education finance and academic accountability policies. Educational Considerations, 42(2), 13–19. Retrieved from https://eric.ed.gov/?id=EJ1073925.

From the ERIC abstract: “The conceptualization and measurement of education finance equity and adequacy has engaged researchers for more than three decades. At the same time, calls for increased academic accountability and higher student achievement in K-12 public education have reached new levels at both the national and state levels. Aligning these represents an emerging area of research with many challenges. For example, recent efforts by the authors to measure the alignment of fiscal equity and student outcomes using an equity ratio faced challenges, particularly because traditional education finance statistical measures do not fully account for factors that either impeded or contributed to their alignment. In this article, the authors first review the process they used to create an equity ratio used to measure alignment. They then turn to their subsequent and related research to identify relevant contextual factors. Based upon these studies, they propose a conceptual framework that illustrates the interrelationship of factors associated with the alignment of education finance and accountability policies.”

Kostyo, S., Cardichon, J., & Darling-Hammond, L. (2018). Making ESSA’s equity promise real: State strategies to close the opportunity gap. Palo Alto, CA: Learning Policy Institute. Retrieved from https://learningpolicyinstitute.org/product/essa-equity-promise-report.

From the abstract: “It is well-documented that students of color, and other historically underserved students, have had less access to an equitable and supportive learning environment, perpetuating school failure and, too often, a school-to-prison pipeline that is difficult to escape. The inequalities have included exclusionary and discriminatory discipline practices that have pushed students out of school and on a pathway to dropping out, unsupportive school environments, and less access to high-quality curriculum that would prepare students for college and productive careers. This report shows how a number of states are taking advantage of opportunities in the federal Every Student Succeeds Act (ESSA) to address these disparities, make schools more inclusive, and help all students succeed. It provides information on which states have committed in their ESSA accountability plans and school improvement efforts to use one or more of five measures (or ‘indicators’) to diagnose and address sources of inequity and school failure and to support the success of all students. They are:

  1. suspension rates;
  2. school climate;
  3. chronic absenteeism;
  4. extended-year graduation rate; and
  5. access to a college- and career-ready curriculum

The report explains how the data can be used to inform efforts to improve schools, highlights selected state approaches, and is accompanied by a series of online interactive maps that show which states are using which indicators and how they are measuring student performance on that indicator.”

Mellor, M., & Griffith, D. (2015). Multimetric accountability systems: A next-generation vision of student success. Alexandria, VA: ASCD. Retrieved from http://www.ascd.org/public-policy/multimetric-accountability.aspx.

From the description: “Find out how five different accountability systems at multiple levels (district, state, and provincial) are using multiple measures to better understand and support student learning and development.”

Noguera, P., Darling-Hammond, L., & Friedlaender, D. (2015). Equal opportunity for deeper learning. Boston, MA: Jobs for the Future. Retrieved from https://eric.ed.gov/?id=ED560802.

From the ERIC abstract: “Out of concern that the nation’s schools—particularly those working with traditionally underserved populations—are not adequately preparing all students to succeed in college and careers, education policymakers have launched a series of major reform efforts in recent years. To help students meet the new standards, schools will need to provide regular opportunities to practice high-level skills such as solving complex problems, conducting research, communicating in multiple forms, and using new technologies to find, analyze, and evaluate information. This report addresses the issue of equity in the crucial dimensions of teaching and learning. The authors argue that to ensure equity in access to deeper learning, practices and policies must address the context for education both outside and inside of schools. To inform efforts to prepare greater numbers of students for college, careers, and civic life, first are describe the obstacles that currently prevent schools from delivering high-quality instruction. Next, are examined educational models, structures, and practices that facilitate deeper learning. Finally, the authors take a wider systemic perspective to consider how policy, practice, and research can be aligned to support the development of pedagogy for deeper learning in schools serving students who have been placed at risk of school failure.”

Penuel, W., Meyer, E., & Valladares, M. R. (2016). Making the most of the Every Student Succeeds Act (ESSA): Helping states focus on school equity, quality and climate. Boulder, CO: National Education Policy Center. Retrieved from https://eric.ed.gov/?id=ED578780.

From the ERIC abstract: “Staff in State Departments of Education are diligently reviewing and revising their state accountability systems to meet the new requirements and opportunities of the Every Student Succeeds Act (ESSA). ESSA is the latest reauthorization of the 1965 Elementary and Secondary Education Act, the primary federal bill guiding K-12 education policy. As a democracy requires equity and adequate educational opportunities, this policy memo provides guidance to states for selecting more inclusive school quality and student success indicators for accountability systems. The authors do not prescribe a fixed list of indicators that each state should adopt, or even a fixed list of categories for states to consider. Rather, they use indicators of opportunities to learn and school climate to describe the risks and possibilities that states should consider when deciding on indicators. This memo concludes by recommending approaches for selecting indicators that address the importance of equity and of students’ perceptions of support, safety, and respect in the classroom.”

Wiener, R., Goldstein, M., & Gonzales, D. (2016). Advancing equity through ESSA: Strategies for state leaders. Washington, DC: Aspen Institute. Retrieved from https://eric.ed.gov/?id=ED577041.

From the ERIC abstract: “With the implementation of the Every Student Succeeds Act (ESSA), states have a new opportunity to lead on educational equity. Public education is fundamentally a state responsibility that is explicitly addressed by each state’s constitution, and states provide the largest share of funding for public schools, which positions states to ensure equity in education remains a priority—and becomes a reality. To help states use ESSA to close opportunity and achievement gaps, this document identifies eight equity priorities—closing funding gaps, improving low-performing schools, increasing access to effective teachers and leaders, supporting English learners, increasing access to advanced coursework, addressing disproportionate discipline practices, addressing students’ social-emotional learning needs, and improving access to high-quality instructional materials—and illustrates how states can use ESSA to improve equity in opportunities and outcomes.”

Additional Organizations to Consult

CORE Districts – https://coredistricts.org/about-us/

From the website: “In 2013, this shared focus on innovation, collaboration and local control helped the CORE Districts secure a No Child Left Behind waiver from the federal government to use more than just test scores to measure strengths and weaknesses in schools and to identify those in need of improvement. The CORE Districts built and maintain a comprehensive school improvement and accountability system that is nationally recognized because it provides educators a clear view of progress by including data on student-level academic growth, high school readiness, students’ social-emotional skills and schools’ culture-climate, along with traditional measures of test scores, graduation rates and absenteeism. Today, an unwavering belief in equity and access for all students continues to drive the CORE Districts’ work, which is evident in the lessons shared within and across districts and schools, and with state and federal policymakers.”

PACE Reports and Findings – https://coredistricts.org/our-data-research/research-report-findings/

National Equity Project – https://nationalequityproject.org/about

From the website: “Our mission is to dramatically improve educational experiences, outcomes, and life options for students and families who have been historically underserved by their schools and districts. We work to build culture, conditions, and competencies for excellence and equity in districts, schools, classrooms, nonprofits, and communities. We work with partners across the U.S. and are always seeking new partners who are committed to achieving equity in education.”

Methods

Keywords and Search Strings

The following keywords and search strings were used to search the reference databases and other sources:

  • “Classroom observation techniques” equity

  • EdEquity

  • Equity

  • Equity walk

  • K-12 equity indicators

  • Measurement equity

  • “Measurement techniques” equity

  • Performance equity

  • “School performance rating” equity

Databases and Search Engines

We searched ERIC for relevant resources. ERIC is a free online library of more than 1.6 million citations of education research sponsored by the Institute of Education Sciences (IES). Additionally, we searched IES and Google Scholar.

Reference Search and Selection Criteria

When we were searching and reviewing resources, we considered the following criteria:

  • Date of the publication: References and resources published over the last 15 years, from 2004 to present, were included in the search and review.

  • Search priorities of reference sources: Search priority is given to study reports, briefs, and other documents that are published or reviewed by IES and other federal or federally funded organizations.

  • Methodology: We used the following methodological priorities/considerations in the review and selection of the references: (a) study types—randomized control trials, quasi-experiments, surveys, descriptive data analyses, literature reviews, policy briefs, and so forth, generally in this order, (b) target population, samples (e.g., representativeness of the target population, sample size, volunteered or randomly selected), study duration, and so forth, and (c) limitations, generalizability of the findings and conclusions, and so forth.
This memorandum is one in a series of quick-turnaround responses to specific questions posed by educational stakeholders in the Midwest Region (Illinois, Indiana, Iowa, Michigan, Minnesota, Ohio, Wisconsin), which is served by the Regional Educational Laboratory (REL Midwest) at American Institutes for Research. This memorandum was prepared by REL Midwest under a contract with the U.S. Department of Education’s Institute of Education Sciences (IES), Contract ED-IES-17-C-0007, administered by American Institutes for Research. Its content does not necessarily reflect the views or policies of IES or the U.S. Department of Education nor does mention of trade names, commercial products, or organizations imply endorsement by the U.S. Government.