Effect of Accountability Policies
November 2019
Question
What does the research say about the effect of accountability policies such as high stakes tests and standards on K-12 schools?
Ask A REL Response
Thank you for your request to our Regional Educational Laboratory (REL) Reference Desk. Ask A REL is a collaborative reference desk service provided by the 10 RELs that, by design, functions much in the same way as a technical reference library. Ask A REL provides references, referrals, and brief responses in the form of citations in response to questions about available education research.
Following an established REL Northwest research protocol, we conducted a search for evidence- based research. The sources included ERIC and other federally funded databases and organizations, research institutions, academic research databases, Google Scholar, and general Internet search engines. For more details, please see the methods section at the end of this document.
The research team has not evaluated the quality of the references and resources provided in this response; we offer them only for your reference. The search included the most commonly used research databases and search engines to produce the references presented here. References are listed in alphabetical order, not necessarily in order of relevance. The research references are not necessarily comprehensive and other relevant research references may exist. In addition to evidence-based, peer-reviewed research references, we have also included other resources that you may find useful. We provide only publicly available resources, unless there is a lack of such resources or an article is considered seminal in the topic area.
References
Chakrabarti, R., & Schwartz, N. (2013). Unintended consequences of school accountability policies: Evidence from Florida and implications for New York. Economic Policy Review, 19(1), 19–43. Retrieved from https://www.newyorkfed.org/
From the Abstract:
"Over the past two decades, state and federal education policies have increasingly emphasized school accountability. This approach focuses on the assignment of rewards and sanctions for schools based on measurable outcomes, usually student performance on standardized tests. A common criticism of accountability policies is that they may induce schools to "game the system" along with—or instead of—making genuine educational improvements. This article investigates whether schools resorted to such strategic behavior in response to the Florida Opportunity Scholarship Program (FOSP), an influential accountability policy that made students from low-performing schools eligible for vouchers to transfer to better ones. Our findings have important implications for New York City's Progress Reports program and New York's implementation of the federal No Child Left Behind (NCLB) Act, which were modeled on the Florida program but contain crucial design changes."
Figlio, D. N., & Loeb, S. (2011). School accountability. In E. A. Hanushek, S. Machin & L. Woessmann (Eds.), Handbook of the economics of education (Vol. 3, pp. 383-421). North-Holland, The Netherlands: Elsevier. Retrieved from https://cepa.stanford.edu/
From the Abstract:
"School accountability—the process of evaluating school performance on the basis of student performance measures—is increasingly prevalent around the world. In the United States, accountability has become a centerpiece of both Democratic and Republican federal administrations' education policies. This chapter reviews the theory of school-based accountability, describes variations across programs, and identifies key features influencing the effectiveness and possible unintended consequences of accountability policies. The chapter then summarizes the research literature on the effects of test-based accountability on students and teachers, concluding that the preponderance of evidence suggests positive effects of the accountability movement in the United States during the 1990s and early 2000s on student achievement, especially in math. The effects on teachers and on students' long-run outcomes are more difficult to judge. It is also clear that school personnel respond to accountability in both positive and negative ways, and that accountability systems run the risk of being counter-productive if not carefully thought out and monitored."
Holbein, J. B., & Ladd, H. F. (2015). Accountability pressure and non-achievement student behaviors. (CALDER Working Paper 122). Washington, DC: CALDER https://eric.ed.gov/
From the Abstract:
"In this paper we examine how failing to make adequate yearly progress under No Child Left Behind (NCLB), and the accountability pressure that ensues, affects various non-achievement student behaviors. Using administrative data from North Carolina and leveraging a discontinuity in the determination of school failure, we examine the causal impact of accountability pressure both on student behaviors that are incentivized by NCLB and on those that are not. We find evidence that, as NCLB intends, pressure encourages students to show up at school and to do so on time. Accountability pressure also has the unintended effect, however, of increasing the number of student misbehaviors such as suspensions, fights, and offenses reportable to law enforcement. Further, this negative response is most pronounced among minorities and low performing students, who are the most likely to be left behind. The following are appended: (1) Descriptive Statistics; (2) Offense Measures Used; (3) Supplemental Analyses; and (4) Alternate Model Specifications."
Saw, G., Schneider, B., Frank, K., & Chen, I. (2017). The impact of being labeled as a persistently lowest achieving school: Regression discontinuity evidence on consequential school labeling. American Journal of Education, 123(4), 585–613. Retrieved from https://www.researchgate.net/
From the Abstract:
"Since the No Child Left Behind Act was enacted, school sanction policies have been increasingly used as a means to incentivize failing schools to raise student achievement. Using state-wide high school data from Michigan, our regression discontinuity analyses show that the bottom 5% schools identified as Persistently Lowest Achieving (PLA), which are accompanied by threats of sanctions, increased their student achievement in reading and writing. No immediate impact on student performance in mathematics was detected after year one but its effects become noticeable in year two. We find no improvement in student achievement for those bottom 6-20% schools labeled as "watch list", which received no actual penalties, suggesting that labeling per se appears to have limited effects on school performance."
Whitney, C. R., & Candelaria, C. A. (2017). The effects of No Child Left Behind on children's socioemotional outcomes. AERA Open, 3(3), 1–21. https://eric.ed.gov/
From the Abstract:
"Many people have worried about possible adverse effects of high-stakes testing on socioemotional outcomes. This article uses a difference-in-differences approach to investigate the effects of the introduction of high-stakes testing via the No Child Left Behind Act on socioemotional outcomes. Data are from the Early Childhood Longitudinal Survey--Kindergarten Cohort of 1998-1999, a nationally representative longitudinal survey. The 10 outcomes that we examine are from the children's Self-Descriptive Questionnaire, including externalizing problems related to paying attention and behavior in school, internalizing problems related to feeling sad and lonely and academic anxiety, as well as interest and competence in math, reading, and school in general. We find that the introduction of high-stakes test accountability did not have consistent significant effects on these socioemotional outcomes. These findings can help states address concerns and motivate further research on potential unintended consequences of revised accountability systems under the Every Student Succeeds Act."
Other Resources
Brill, F, Grayson, H and Kuhn, L. (2018). What impact does accountability have on curriculum, standards and engagement in education? A literature review. Slough: National Foundation for Educational Research. https://eric.ed.gov/
From the Abstract:
"The authors define accountability broadly as a government's mechanism for holding educational institutions to account for the delivery of high quality education. The idea that the practice of accountability can contribute directly to improvements in education is a powerful one that underpins policy. Paradoxically, though, some hold that accountability systems can also produce negative impacts on education, making it more difficult for schools to deliver the sought after quality. They believe it is critical that research evidence should inform any rationales for policy change. The literature review aims to evaluate a small body of international research evidence on the impact of accountability on three key areas: (1) curriculum; (2) standards; and (3) engagement."
David, J. L. (2011). Research says …/high stakes testing narrows the curriculum. Educational Leadership, 68(6), 78–80. Retrieved from http://www.ascd.org/
From the Article:
"Are science, social studies, the arts, and physical education really disappearing from elementary schools? Are critical thinking and deep reading of literature fading from the high school curriculum?"
Elish-Piper, L., Matthews, M. W., & Risko, V. J. (2013). Invisibility: An unintended consequence of standards, tests, and mandates. Journal of Language and Literacy Education, 9(2), 4–23. https://eric.ed.gov/
From the Abstract:
"As elementary and middle school teachers and students face standards, high-stakes testing, accountability, and one-size-fits all curricula, concerns have arisen that these practices limit the relevance and efficacy of teaching and learning. In this paper, we argue that such practices exact personal costs on students and the teachers expected to implement them. With data from a series of studies implemented across several years, we show how such practices too often create an instructional climate that, in effect, renders teachers and students invisible and nonessential to the literacy instruction that occurs in the classroom. First, we discuss the research that grounds our thinking. Then, we describe three approaches that can overcome invisibility for both students and teachers: teaching with students' hearts and heads in mind, promoting culturally responsive pedagogy, and creating a productive literacy environment. We conclude with portraits of three teachers, who in spite of external pressures create literacy instruction that makes their students' capabilities visible in their classroom instruction."
Marsh, J. A., Bush-Mecenas, S., Hough, H. J., Park, V., Allbright, T., Hall, M., & Glover, H. (2016). At the forefront of the new accountability era: Early implementation findings from the CORE waiver districts. Policy Analysis for California Education. https://eric.ed.gov/
From the Abstract:
"California and the nation are at the crossroads of a major shift in school accountability policy. At the state level, California's Local Control and Accountability Plan (LCAP) encourages the use of multiple measures of school performance used locally to support continuous improvement and strategic resource allocation. Similarly, the federal Every Student Succeeds Act (ESSA) reinforces this local control, requiring more comprehensive assessment of school performance and a less prescriptive, local approach to school support. These changes represent a major cultural shift for California schools and districts. Ahead of the curve, six California districts, known as the California Office to Reform Education (CORE) waiver districts, have implemented an innovative measurement system and supports for school and district improvement under an NCLB waiver, and thus provide a unique opportunity to examine and learn from the enactment of a system supported by accountability policy in this new era. This report examines the early implementation and effects of the CORE reform and seeks to inform the ongoing efforts within CORE as well as the development and implementation of future accountability policy in other states and districts."
Supovitz, J. (2009). Can high stakes testing leverage educational improvement? Prospects from the last decade of testing and accountability reform. Journal of Educational Change, 10(2-3), 211–227. Retrieved from https://cpre.org/
From the Abstract:
"This article examines major trends in testing and accountability reform in the United States over the past decade. The review covers the apex and decline of the national experimentation with a range of alternative assessments and the rise of test-based accountability as a central policy initiative. These trends signify that testing has become a widely utilized instrument for educational reform in America. Research on these trends indicates that high stakes testing does motivate teachers and administrators to change their practices, yet the changes they motivate tend to be more superficial adjustments in content coverage and test preparation activities rather than promoting deeper improvements in instructional practice. Further, the information provided by large scale assessments is primarily useful to measure school and system progress, but of more limited utility for instructional guidance. Most problematic is that the high stakes testing system in America has been repeatedly promoted as a substantive reform in itself. However, high stakes testing is a relatively weak intervention because, while it reveals shortcomings, it does not contain the guidance and expertise to inform response. The article concludes with suggestions on how to capitalize on the strengths of high stakes testing while minimizing its shortcomings."
Methods
Keywords and Search Strings: The following keywords, subject headings, and search strings were used to search reference databases and other sources: School, High stakes, Effects, Standardized, Accountability
Databases and Resources: We searched ERIC for relevant resources. ERIC is a free online library of more than 1.6 million citations of education research sponsored by the Institute of Education Sciences (IES). Additionally, we searched Google Scholar and EBSCO databases (Academic Search Premier, Education Research Complete, and Professional Development Collection).
Reference Search and Selection Criteria
When we were searching and reviewing resources, we considered the following criteria:
Date of publications: This search and review included references and resources published in the last 10 years.
Search priorities of reference sources: Search priority was given to study reports, briefs, and other documents that are published and/or reviewed by IES and other federal or federally funded organizations, as well as academic databases, including ERIC, EBSCO databases, and Google Scholar.
Methodology: The following methodological priorities/considerations were given in the review and selection of the references:
- Study types: randomized control trials, quasi experiments, surveys, descriptive data analyses, literature reviews, and policy briefs, generally in this order
- Target population and samples: representativeness of the target population, sample size, and whether participants volunteered or were randomly selected
- Study duration
- Limitations and generalizability of the findings and conclusions
This memorandum is one in a series of quick-turnaround responses to specific questions posed by stakeholders in Alaska, Idaho, Montana, Oregon, and Washington, which is served by the Regional Educational Laboratory (REL) Northwest. It was prepared under Contract ED-IES-17-C-0009 by REL Northwest, administered by Education Northwest. The content does not necessarily reflect the views or policies of IES or the U.S. Department of Education, nor does mention of trade names, commercial products, or organizations imply endorsement by the U.S. Government.