Skip Navigation
archived information
Skip Navigation

Back to Ask A REL Archived Responses

REL Midwest Ask A REL Response

Educator Effectiveness

October 2019

Question:

What does research say about school performance rating systems?



Response:

Following an established Regional Educational Laboratory (REL) Midwest protocol, we conducted a search for research reports, descriptive studies, and policy overviews on school performance rating systems. In particular, we looked for research on the design, use, utility, and impact of school performance rating systems for formative and summative purposes. For details on the databases and sources, keywords, and selection criteria used to create this response, please see the Methods section at the end of this memo.

Below, we share a sampling of the publicly accessible resources on this topic. References are listed in alphabetical order, not necessarily in order of relevance. The search conducted is not comprehensive; other relevant references and resources may exist. For each reference, we provide an abstract, excerpt, or summary written by the study’s author or publisher. We have not evaluated the quality of these references, but provide them for your information only.

Research References

Auty, B., & Brockmann, F. (2012). Growth model comparison study: A summary of results. Washington, DC: Council of Chief State School Officers. Retrieved from https://eric.ed.gov/?id=ED542760

From the ERIC abstract: “School accountability is subject to considerable scrutiny. It generates sharp political debate, policy challenges, and continuous discussion. Growth models are now a part of that discussion. To many practitioners the sheer volume of ‘important to know’ information is daunting. The members of the Technical Issues in Large Scale Assessment (TILSA) and Accountability Systems & Reporting (ASR) state collaboratives have recognized the challenge of communicating detailed, technically-oriented measurement issues. In response, these groups have produced a series of publications aimed at providing technical and non-technical users with practical guidance about growth models. This publication continues the series by summarizing the ‘Growth Model Comparison Study: Practical Implications of Alternative Models for Evaluating School Performance’ conducted by Pete Goldschmidt, Kilchan Choi, and J.P. Beaudoin which implemented statistical growth models using real student data from four states. The study is important because it is the first designed to make it possible to see whether results are different or similar when (a) the same growth models are estimated in different states and (b) different models are estimated in the same state. As a summary, this document is intended to provide timely and comprehensible information for practitioners and other stakeholders who may not have a technical background in using assessment data for educational accountability.”

Castellano, K. E., & Ho, A. D. (2013). A practitioner’s guide to growth models. Washington, DC: Council of Chief State School Officers. Retrieved from https://eric.ed.gov/?id=ED551292

From the ERIC abstract:“ This ‘Practitioner’s Guide to Growth Models,’ commissioned by the Technical Issues in Large-Scale Assessment (TILSA) and Accountability Systems & Reporting (ASR), collaboratives of the ‘Council of Chief State School Officers,’ describes different ways to calculate student academic growth and to make judgments about the adequacy of that growth. It helps to clarify the questions that each model answers best, as well as the limitations of each model. This document is intended to support states as they address the challenges of evolving assessment and accountability systems. This guide does not promote one type of interpretation over another. Rather, it describes growth models in terms of the interpretations they best support and, in turn, the questions they are best designed to answer. The goal of this guide is thus to increase alignment between user interpretations and model function in order for models to best serve their desired purposes: increasing student achievement, decreasing achievement gaps, and improving the effectiveness of educators and schools.”

Darling-Hammond, L., & Plank, D. N. (2015). Supporting continuous improvement in California’s education system. Berkeley, CA: Policy Analysis for California Education. Retrieved from https://eric.ed.gov/?id=ED564346

From the ERIC abstract: “California’s new accountability system originated in the radical decentralization of power and authority from Sacramento to local schools and their communities brought about by the Legislature’s adoption of the Local Control Funding Formula (LCFF) in 2013. Under California’s previous accountability policies and the federal ‘No Child Left Behind’ law, the state set performance targets for schools and districts based almost entirely on students’ standardized test scores. Schools that fell short of their targets were subject to a variety of increasingly harsh sanctions, ranging from designation as a ‘failing’ school to reconstitution or closure. California’s new accountability system is different from the previous system in nearly every important respect. The new system is grounded in the concept of reciprocal accountability: that is, every actor in the system—from the Capitol to the classroom—must be responsible for the aspects of educational quality and performance that it controls. This publication presents some key elements of California’s new accountability system and the state has made three fundamental commitments: (1) To pursue ‘meaningful learning’ for students—through the adoption of new standards and curriculum frameworks more focused on higher order thinking and performance abilities; (2) To give schools and districts the ‘resources’ and flexibility they need to serve their communities effectively—through the new LCFF, which allocates funds based on student needs and allows communities to determine where the funds should be spent to achieve the best results; and (3) To provide ‘professional learning’ and supports for teachers and administrators—through stronger preparation and ongoing professional development. At the same time, the state has adopted three complementary mechanisms to hold schools and districts accountable: (1) Political accountability, operationalized through Local Control Accountability Plans (LCAPs), created by districts with their communities, updated annually, and reviewed by county agencies. The LCAPs, intended to ensure that resources are used wisely and effectively, articulate local goals for schooling and report outcomes; (2) Professional accountability, through effective licensure, professional development, and productive evaluation, to ensure that educators deliver high-quality instructional and other services to their students; and (3) Performance accountability, to ensure continuous improvement in the performance of schools across the state’s eight priority areas, plus other priorities identified locally. The eight priority areas include student achievement, student engagement, school climate, parent involvement, provision of basic services, curriculum access, and implementation of the state’s new standards. This kind of unified long-term strategy could enable California to move successfully from a compliance-driven system to one that is capable of system learning and continuous improvement.”

Duffy, M., & Jenkins, D. (2015). Creating a comprehensive picture of school performance. A PACER policy brief. Philadelphia, PA: Research for Action. Retrieved from https://eric.ed.gov/?id=ED570124

From the ERIC abstract: “For more than two decades, states have been required to report publicly on the academic performance of schools and districts. These school rating systems have received increased public attention amid growing concerns about the prevalence and cost of standardized testing in schools. According to a recent Gallup Poll, 64 percent of the public overall, and 67 percent of public school parents, said there is too much emphasis on standardized testing in education. These perceptions have been widely acknowledged by policymakers at both the state and federal levels. As deliberations on the future of Pennsylvania’s school rating system, the School Performance Profile, gather steam, Research for Action (RFA) is pleased to provide background on the existing research related to best practices in reporting on school performance. We also offer examples of reporting systems from neighboring and high-performing states.”

Education Commission of the States. (2018). 50-state comparison: States’ school accountability systems. Denver, CO: Author. Retrieved from https://www.ecs.org/50-state-comparison-states-school-accountability-systems/

From the description: “This resource captures an important transition period in state accountability systems by providing a national overview of these systems as described in current state statute and regulation (as of December 2017), where available, and in states’ ESSA plans (as of May 31, 2018). State statute and regulation often outline or provide a foundation for accountability systems. In some cases, states may operate multiple systems to ensure school quality, not all of which are described in statute and regulation. To help fill in the blanks that are not described in state policies, other resources (where publicly available) have been provided.”

Evans, C. M. (2019). Effects of New Hampshire’s innovative assessment and accountability system on student achievement outcomes after three years. Education Policy Analysis Archives, 27(10). Retrieved from https://eric.ed.gov/?id=EJ1204567

From the ERIC abstract: “New Hampshire’s Performance Assessment of Competency Education (PACE) pilot received a waiver from federal statutory requirements related to state annual achievement testing starting in the 2014-15 school year. PACE is considered an ‘innovative’ assessment and accountability system because performance assessments are used to help determine student proficiency in most federally required grades and subjects instead of the state achievement test. One key criterion for success in the early years of the PACE innovative assessment system is ‘no harm’ on the statewide accountability test. This descriptive study examines the effect of PACE on Grades 8 and 11 mathematics and English language arts student achievement during the first three years of implementation (2014-15, 2015-16, and 2016-17 school years) and the extent to which those effects vary for certain student subgroups using results from the state’s accountability tests (Smarter Balanced and SATs). Findings suggest that students in PACE schools tend to exhibit small positive effects on the Grades 8 and 11 state achievement tests in both subjects in comparison to students attending non-PACE comparison schools. Lower achieving students tended to exhibit small positive differential effects, whereas male students tended to exhibit small negative differential effects. Implications for research, policy, and practice are discussed.”

Goldschmidt, P., Choi, K., & Beaudoin, J. P. (2012). Growth model comparison study: Practical implications of alternative models for evaluating school performance. Washington, DC: Council of Chief State School Officers. Retrieved from https://eric.ed.gov/?id=ED542761

From the ERIC abstract: “The Elementary and Secondary Education Act (ESEA) has had several tangible effects on education and the monitoring of education. There have been both intended and unintended consequences. ESEA’s newer generation of federal programs, such as Race to the Top, and the recent ESEA flexibility guidelines, have continued to push development of methods to accurately and fairly monitor school (and more recently teacher) performance. The purpose of this study is to compare several different growth models and examine empirical characteristics of each. This study differs from previous research comparing various models for accountability purposes in that the focus is broader--it is based on large scale assessment results from four states (Delaware, Hawaii, North Carolina, and Wisconsin) across two cohorts of students (each with three consecutive years of assessment results), and explicitly considers model results with respect to elementary and middle schools. This study addresses the following research questions regarding the performance of the different growth models: (1) Overall, does the model matter?; (2) Do different models lead to different inferences about schools?; (3) How accurately do models classify schools into performance categories?; (4) Are models consistent in classifying schools from one year to the next?; (5) How are models influenced by school intake characteristics (percent ELL, FRL, etc.)?; (6) Do models perform similarly for elementary and middle schools?; and (7) Do models behave similarly across states? The results of these analyses confirm that no single model can unequivocally be assumed to provide the best results. This is not possible for two reasons: one, different models address different questions about schools; and two, the empirical results indicate that context matters when examining models. By context the authors mean that the state in which the model will be run affects how the model may work. State affects include several pieces that are confounded. These include tests scales, testing procedures, student characteristics, and school characteristics. An accountability model should not be unduly influenced by factors outside of schools’ control and models clearly differ in this respect. Distinguishing between a school’s ability to facilitate learning and a school’s performance as a function of advantageous (or challenging) student enrollment characteristics is where statistical machinery provides its biggest benefit.”

Hamilton, L. S., Stecher, B. M., & Yuan, K. (2012). Standards-based accountability in the United States: Lessons learned and future directions. Education Inquiry, 3(2), 149–170. Retrieved from https://eric.ed.gov/?id=EJ1211880

From the ERIC abstract: “Standards-based accountability (SBA) has been a primary driver of education policy in the United States for several decades. Although definitions of SBA vary, it typically includes standards that indicate what students are expected to know and be able to do, measures of student attainment of the standards, targets for performance on those measures, and a set of consequences for schools or educations based on performance. Research on SBA indicates that these policies have led to some of the consequences its advocates had hoped to achieve, such as an emphasis on equity and alignment of curriculum within and across grade levels, but that it has also produced some less desirable outcomes. This article summarizes the research on SBA in three areas: quality of standards, ways in which SBA has shaped educators’ practices, and effects on student achievement. The article identifies lessons learned from the implication of SBA in the United States and provides guidance for developing SBA systems that could promote beneficial outcomes for students.”

Hough, H., Byun, E., & Mulfinger, L. (2018). Using data for improvement: Learning from the CORE Data Collaborative (Technical Report, Getting Down to Facts II). Berkeley, CA: Policy Analysis for California Education. Retrieved from https://eric.ed.gov/?id=ED594744

From the ERIC abstract: “Under emerging policy structures in California, the responsibility for school improvement is increasingly placed upon local school districts, with County Offices of Education (COEs) playing a critical support role. In this system, districts are responsible for school improvement, with counties in charge of ensuring quality across districts and providing feedback and support where necessary. Underlying this major policy shift is the idea that local leaders are in the best position to drive real educational improvement and ensure quality across multiple schools and contexts. As California supports districts and counties statewide to embark on this improvement journey, there are important lessons to be learned from the CORE districts, six of which developed an innovative accountability system under a waiver from No Child Left Behind (NCLB). The CORE districts are early adopters of the new accountability paradigm: local leaders using multiple measures of school performance and working together to figure out collectively what works best for struggling schools. Now deepening their work together as a Networked Improvement Community (NIC), the CORE districts have simultaneously expanded access to their multiple-measures data and learning system by inviting other California districts to join their ‘Data Collaborative.’ Districts who join contribute their own student data to the CORE measurement system and are then able to benchmark student performance against other schools in the state. Data Collaborative districts engage in structured network learning activities that enhance their capacity to use data to drive system improvements. Currently, over 50 California school districts representing nearly a million students have joined the Data Collaborative. This report first provides a framework for how data use for improvement is different from data use for accountability and how data should be used by actors at different levels of the system. Next, it discusses the policy context in California and the current state of data use based on interviews conducted in the summer of 2017 with 41 leaders from state education agencies, COEs, school districts, technical assistance providers, education advocacy organizations, and education associations. Finally, the report shares lessons from the CORE Data Collaborative about how data can be used for improvement within networked structures, including what data is needed and how learning and collaboration can be facilitated.”

Hough, H., Penner, E., & Witte, J. (2016). Identity crisis: Multiple measures and the identification of schools under ESSA (Policy Memo 16-3). Berkeley, CA: Policy Analysis for California Education. Retrieved from https://eric.ed.gov/?id=ED574851

From the ERIC abstract: “The Every Student Succeeds Act (ESSA) makes sweeping changes to the way school performance is measured. Using the innovative measurement system developed by the CORE Districts in California, the authors explore how schools can be identified for support and improvement using a multiple measures framework. They show that 1) Different academic indicators measure very different aspects of school performance, suggesting that states should be allowed and encouraged to make full use of multiple measures to identify schools in the way they see fit instead of reporting a summative rating; 2) The ESSA regulations effectively restrict the weighting of the non-academic ‘School Quality and Student Success’ indicators to zero, which is not in the spirit of the expanded measurement; and 3) The majority of schools will be identified for targeted support under the current regulations, suggesting the need for a clarification in federal policy.”

Isenberg, E., & Hock, H. (2011). Measuring school and teacher value added in DC, 2011-2012 School Year: Final report. Princeton, NJ: Mathematica Policy Research, Inc. Retrieved from https://eric.ed.gov/?id=ED566131

From the ERIC abstract: “This report describes the value-added models used as part of teacher evaluation systems in the District of Columbia Public Schools (DCPS) and in eligible DC charter schools participating in ‘Race to the Top.’ The authors estimated: (1) teacher effectiveness in DCPS and eligible DC charter schools during the 2011-2012 school year; and (2) school effectiveness in DCPS during the same year. This report updates earlier technical reports (Isenberg and Hock 2010; Isenberg and Hock 2011). The 2011-2012 school year is the third year of IMPACT, a teacher evaluation system for DCPS that relies in part on value-added estimates of teacher and school effectiveness. Under IMPACT, teachers who earn a highly effective rating have received performance pay, and those who earn an ineffective or a minimally effective rating for two consecutive years have been dismissed. This report provides an overview of value-added methods in nontechnical terms (Chapter I); updates the earlier technical reports by describing the data used to estimate teacher and school value added, including the new data on charter school students and teachers (Chapter II); and then describes the statistical methods used to estimate teacher and school value added (Chapter III). Chapters II and III include tables of diagnostic information that summarize the population of students and teachers on which the value-added estimates are based, as well as the results from the statistical model used to produce those estimates.”

Koon, S., Petscher, Y., & Hughes, J. (2015). Development and examination of an alternative school performance index in South Carolina (REL 2015-097). Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Southeast. Retrieved from https://eric.ed.gov/?id=ED560729

From the ERIC abstract: “The purpose of this study was to examine the extent to which the measures that make up each of the three separate accountability indices of school performance in South Carolina could be used to create an overall, reliable index of school performance. Data from public elementary, middle, and high schools in 2012/13 were used in confirmatory factor analysis models designed to estimate the relations between the measures under different specifications. Four different factor models were compared at each school level, beginning with a one-factor model and ending with a bi-factor model. Results from the study suggest that the measures which currently are combined into three separate indices of school performance can instead be combined into a single index of school performance using a bi-factor model. The reliability of the school performance general factor estimated by the bi-factor model ranged from 0.89 to 0.95. Using this alternative school performance rating, the study found that approximately 3 percent of elementary schools, 2 percent of middle schools, and 3 percent of high schools were observed to statistically outperform their predicted performance when accounting for the school’s demographic characteristics. These schools, referred to as schools beating the odds, were found in most of the demographic profiles which represent South Carolina schools. The results of this study can inform decisions related to the development of new accountability indices in South Carolina and other states with similar models.”

Marsh, J. A., Bush-Mecenas, S., Hough, H. J., Park, V., Allbright, T., Hall, M., et al. (2016). At the forefront of the new accountability era: Early implementation findings from the CORE waiver districts. Berkeley, CA: Policy Analysis for California Education. Retrieved from https://eric.ed.gov/?id=ED574787

From the ERIC abstract: “California and the nation are at the crossroads of a major shift in school accountability policy. At the state level, California’s Local Control and Accountability Plan (LCAP) encourages the use of multiple measures of school performance used locally to support continuous improvement and strategic resource allocation. Similarly, the federal Every Student Succeeds Act (ESSA) reinforces this local control, requiring more comprehensive assessment of school performance and a less prescriptive, local approach to school support. These changes represent a major cultural shift for California schools and districts. Ahead of the curve, six California districts, known as the California Office to Reform Education (CORE) waiver districts, have implemented an innovative measurement system and supports for school and district improvement under an NCLB waiver, and thus provide a unique opportunity to examine and learn from the enactment of a system supported by accountability policy in this new era. This report examines the early implementation and effects of the CORE reform and seeks to inform the ongoing efforts within CORE as well as the development and implementation of future accountability policy in other states and districts. Additional details about the research methods are appended.”

Martin, M. (2016). School accountability systems and the Every Student Succeeds Act. Durham, NC: Hunt Institute. Retrieved from https://eric.ed.gov/?id=ED569952

From the ERIC abstract: “The ‘Every Student Succeeds Act’ (ESSA) replaced the ‘No Child Left Behind Act of 2001’ (NCLB) in December 2015, substantially changing the federal role in education and how schools across the country will be held accountable. For state policymakers, designing new ESSA-compliant accountability systems is a significant opportunity and a serious responsibility. Both NCLB and ESSA are reauthorizations of the ‘Elementary and Secondary Education Act’ of 1964 (ESEA) and have important similarities and striking differences in their approaches to school accountability. Under NCLB, states complied with rigid federal requirements for setting performance goals on grade-level standardized tests for students from all backgrounds and implemented mandated interventions in schools if those goals were not met. ESSA—like its predecessor—continues to require states to conduct grade-level testing, report results by subgroup, and implement school accountability systems; however, ESSA gives states greater authority to determine the specifics of what is measured and how those measures are used in school accountability. The new law does not give ‘carte blanche’ to the states, but it does give significantly increased flexibility. ESSA opens the door for innovation in school accountability systems, while the responsibility for maintaining high expectations for all students rests squarely on the shoulders of the states. This issue of ‘re:VISION,’ designed to inform state accountability policy, pairs the specific ESSA accountability requirements with important design considerations in light of these requirements and the opportunity afforded by the new flexibility.”

McEachin, A., & Polikoff, M. S. (2012). We are the 5%: Which schools would be held accountable under a proposed revision of the Elementary and Secondary Education Act? Educational Researcher, 41(7), 243–251. Retrieved from https://eric.ed.gov/?id=EJ979650

From the ERIC abstract: “This article uses data from California to analyze the results of the proposed accountability system in the Senate’s Harkin-Enzi draft Elementary and Secondary Education Act reauthorization. The authors analyze existing statewide school-level data from California, applying the accountability criteria proposed in the draft law. Comparing the proposed system to the No Child Left Behind Act’s Adequate Yearly Progress provisions, they draw conclusions about the stability of the proposed identification schemes and the types of schools likely to be identified. They conclude with several policy recommendations that could be easily incorporated into the law, based on their analysis and the existing literature.”

Note: REL Midwest was unable to locate a link to the full-text version of this resource. Although REL Midwest tries to provide publicly available resources whenever possible, it was determined that this resource may be of interest to you. It may be found through university or public library systems.

Ni, X., Bowers, A. J., & Esswein, J. (2016). What counts in calculating school and district level performance index scores: A summary and analysis of academic performance index metrics across the 50 states. New York, NY: Teachers College, Columbia University. Retrieved from https://academiccommons.columbia.edu/doi/10.7916/D86973RR

From the abstract: “The purpose of this report is to summarize the key elements of school and district level Performance Index scores (PI scores) for the 50 states and the District of Columbia (D.C.) across the United States. PI scores are partial or overall summative ratings of schools or districts currently used across US state accountability systems to assess organizational performance. In this study, we first extracted 14 elements from 49 PI calculation metrics for states in the U.S and conducted a descriptive analysis to provide an overview of which data elements are used across the different calculation metrics for each state and what role PI scores play in state accountability systems. Second, we categorized the fourteen elements into seven categories proposed by the most recent ESSA regulations (81 FR 34539 §200.14-16, 2016) and examined how each state integrated each element in their PI score calculations. Third, we conducted a multidimensional scaling (MDS) analysis to compare the similarities and differences of PI calculation metrics across the states. The results indicate that there are few commonalities in PI score calculation metrics across the states, as each state has its own methods in addressing the requirements of NCLB and now ESSA. The goal of this report is to inform decisions across states on PI score calculations through summarizing overall ratings and metrics nationally used to hold schools and districts accountable as states move toward implementing the recent Every Child Succeeds Act (ESSA) regulations.”

Portz, J. (2017). “Next-generation” accountability? Evidence from three school districts. Urban Education. Retrieved from https://doi.org/10.1177/0042085917741727

From the abstract: “Educational accountability is dominated by a focus on test scores to assess academic achievement. An emerging trend toward ‘next-generation’ accountability includes a broader conception of student learning and multiple metrics. A policy design approach is used to analyze this trend. Four design elements—goals, actors, metrics, and consequences—are used to compare accountability systems in New York City, Los Angeles, and Chicago. Key design trends include broader accountability goals, multiple metrics with different data types, and an emphasis on school improvement. Although facing challenges, urban school districts have opportunities to play a role in shaping this accountability debate.”

Note: REL Midwest was unable to locate a link to the full-text version of this resource. Although REL Midwest tries to provide publicly available resources whenever possible, it was determined that this resource may be of interest to you. It may be found through university or public library systems.

Schwartz, H. L., Hamilton, L. S., Stecher, B. M., & Steele, J. L. (2011). Expanded measures of school performance (Technical report). Santa Monica, CA: RAND Corporation. Retrieved from https://eric.ed.gov/?id=ED521079

From the ERIC abstract: “The upcoming reauthorization of the Elementary and Secondary Education Act provides an opportunity to reconsider what factors school performance-reporting systems should include. Critics of No Child Left Behind (NCLB) have pointed to the narrowing effects of the law’s focus on mathematics and reading achievement, and they have called for efforts to broaden the measures used to rate schools. This report poses and addresses questions regarding expanded measures of school quality to reflect the multiple goals of schooling. The authors convened a panel of five experts on school accountability policies, scanned published research about expanded measures of school performance, conducted ten semistructured phone interviews with staff from local or state education agencies and research institutions, and reviewed the measures employed in each state that publishes its own school ratings in addition to those required under NCLB. After classifying the measures state education agencies use to develop their own school ratings, they then describe categories of measures that research indicates are the most rapidly growing in usage by state and local education agencies. They supplement categories of measures with more detailed examples of localities that have adopted them, examining why they adopted the measures and how the measures are employed. This report describes promising directions for expanding the set of measures that schools have at their disposal while acknowledging the need for more research on how the availability of such measures affects educational practice and student achievement.”

Warren, P. (2014). Designing California’s next school accountability program. San Francisco, CA: Public Policy Institute of California. Retrieved from https://eric.ed.gov/?id=ED561978

From the ERIC abstract: “California is in the midst of a major K-12 reform effort. In 2010, the state adopted the Common Core State Standards (CCSS), which outline what students should know in mathematics and English. In 2013, it adopted tests of the new standards developed by the Smarter Balanced Assessment Collaborative (SBAC). These tests will be administered beginning in 2015, replacing the California Standards Tests (CSTs). In addition, the state revamped its school-finance system in 2013, creating the Local Control Funding Formula (LCFF) to streamline local funding and increase support for disadvantaged students. The LCFF also requires districts to set performance targets on a range of school and student success indicators as part of a district Local Control Accountability Plan (LCAP). This report reviews the state’s options for the next generation of K-12 school accountability programs. The analysis of the strengths and weaknesses of the current programs leads to the proposal of several steps that merge state and local accountability programs and create a more straightforward approach to improving schools and student outcomes: (1) California should create a new state measure that would align with the LCAP program; (2) The state should develop and fund a larger program of technical assistance to school districts; and (3) The legislature and governor need to address governance arrangements of accountability programs. The report provides an outline of a state accountability measure and program that includes a broader range of student outcomes than just test scores and also aligns with the design guidelines.”

Yatsko, S., Opalka, A., Sutter, J., Weeldreyer, L., & Stewart, D. (2016). Apples to apples: Common school performance frameworks as a tool for choice and accountability. Seattle, WA: Center on Reinventing Public Education. Retrieved from https://eric.ed.gov/?id=ED570669

From the ERIC abstract: “Many districts are expanding and diversifying the school options available to parents—a trend that shows no signs of reversing. While all public schools are required to test and publicly report results, it remains nearly impossible for families and education and civic leaders to make school-to-school comparisons, especially across district-run and charter public schools. To address the need for parents to have reliable performance information across school type, and/or for districts and charter authorizers to hold schools accountable to the same set of measures, some cities are developing unified accountability systems. This paper focuses on developing a common school performance framework (CSPF), a tool for measuring performance of an individual school using a defined set of metrics that is common to schools across different agencies or governing bodies. This report is made up of three sections: (1) An Overview (Sarah Yatsko and Alice Opalka); (2) Lessons from Chicago (Jessica Sutter); and (3) How-To Guide: Developing a CSPF (Laura Weeldreyer and David Stewart). The ‘Overview’ draws on examples of cities who have effectively developed CSPFs, such as Los Angeles and Denver. ‘Lessons from Chicago’ discusses a Chicago Public Schools case study and offers ‘pearls of wisdom’ for other cities looking to implement their own CSPFs. The ‘How-To Guide’ outlines the major steps and key considerations a city should take into account when designing a CSPF.”

Additional Organizations to Consult

Council of Chief State School Officers: State Collaborative on Assessment and Student Standards – http://programs.ccsso.org/projects/scass/

From the website: “The State Collaboratives on Assessment and Student Standards (SCASS) System was created in 1991 by the Council to encourage and facilitate state collaboration on assessment development and implementation issues. SCASS groups are formed by state education agency career service professionals in response to specific project needs. Some SCASS groups focus on specific content areas like science, math, social studies, or health. Others focus on policy, psychometric problems, or technology. The goal is to develop and implement high standards and valid assessment systems that maximize educational achievement for all children.”

Performance Assessment Resource Bank – http://performanceassessmentresourcebank.org/about

From the website: “Our goal is to increase the capacity of leaders, policymakers, and school-based educators to develop systems that support more meaningful forms of student learning and prepare students for success in college, career, and life. We believe performance assessment is a powerful educative experience for students and teachers and its effective use can lead to deeper student learning, increased opportunities to learn, and more equitable systems of assessment and accountability.”

Policy & Research: “Learn about state and district policies that support performance assessment & evidence from research to support these policies” – http://performanceassessmentresourcebank.org/bin/policy-resources

Methods

Keywords and Search Strings

The following keywords and search strings were used to search the reference databases and other sources:

  • “Educational improvement” “measurement techniques”

  • Measurement “school performance”

  • “School effectiveness” “measurement techniques”

  • “School performance”

  • “School performance rating”

Databases and Search Engines

We searched ERIC for relevant resources. ERIC is a free online library of more than 1.6 million citations of education research sponsored by the Institute of Education Sciences (IES). Additionally, we searched IES and Google Scholar.

Reference Search and Selection Criteria

When we were searching and reviewing resources, we considered the following criteria:

  • Date of the publication: References and resources published over the last 15 years, from 2004 to present, were included in the search and review.

  • Search priorities of reference sources: Search priority is given to study reports, briefs, and other documents that are published or reviewed by IES and other federal or federally funded organizations.

  • Methodology: We used the following methodological priorities/considerations in the review and selection of the references: (a) study types—randomized control trials, quasi-experiments, surveys, descriptive data analyses, literature reviews, policy briefs, and so forth, generally in this order, (b) target population, samples (e.g., representativeness of the target population, sample size, volunteered or randomly selected), study duration, and so forth, and (c) limitations, generalizability of the findings and conclusions, and so forth.
This memorandum is one in a series of quick-turnaround responses to specific questions posed by educational stakeholders in the Midwest Region (Illinois, Indiana, Iowa, Michigan, Minnesota, Ohio, Wisconsin), which is served by the Regional Educational Laboratory (REL Midwest) at American Institutes for Research. This memorandum was prepared by REL Midwest under a contract with the U.S. Department of Education’s Institute of Education Sciences (IES), Contract ED-IES-17-C-0007, administered by American Institutes for Research. Its content does not necessarily reflect the views or policies of IES or the U.S. Department of Education nor does mention of trade names, commercial products, or organizations imply endorsement by the U.S. Government.