The Effects of School Accountability Ratings in California under ESSA on Student Graduation and College and Career Readiness
Drew Atchison; Kerstin Carlson Le Floch; Umut Özek; Justin Luu; Steven Hurlburt; Andrea Boyle (2025). AERA Open, v11 n1. Retrieved from: https://eric.ed.gov/?id=EJ1494460
-
examining992,911Students, grade12
Single Study Review
Review Details
Reviewed: April 2026
- Single Study Review (findings for California school accountability ratings - Atchison et al. (2025))
- Regression Discontinuity Design
- Meets WWC standards with reservations because it is a cluster randomized controlled trial with high cluster-level attrition, but the analytic intervention and comparison groups satisfy the baseline equivalence requirement.
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Findings
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
||
|---|---|---|---|---|---|---|---|---|---|
|
High School Graduation |
California school accountability ratings - Atchison et al. (2025) vs. Business as usual |
0 Years |
Red Designation for High School Graduation (Overall Sample);
|
N/A |
N/A |
Yes |
|
||
|
High School Graduation |
California school accountability ratings - Atchison et al. (2025) vs. Business as usual |
0 Years |
Orange Designation for High School Graduation (Overall Sample);
|
N/A |
N/A |
Yes |
|
||
| Show Supplemental Findings | |||||||||
|
High School Graduation |
California school accountability ratings - Atchison et al. (2025) vs. Business as usual |
0 Years |
Red Designation for High School Graduation (Year 3 Sample);
|
78.50 |
73.10 |
Yes |
|
||
|
High School Graduation |
California school accountability ratings - Atchison et al. (2025) vs. Business as usual |
0 Years |
Red Designation for High School Graduation (Year 2 Sample);
|
80.30 |
76.10 |
Yes |
|
||
|
High School Graduation |
California school accountability ratings - Atchison et al. (2025) vs. Business as usual |
0 Years |
Red Designation for High School Graduation (Year 4 Sample);
|
82.70 |
80.40 |
No |
-- | ||
|
High School Graduation |
California school accountability ratings - Atchison et al. (2025) vs. Business as usual |
0 Years |
Orange Designation for High School Graduation (Year 2 Sample);
|
89.20 |
87.80 |
No |
-- | ||
|
High School Graduation |
California school accountability ratings - Atchison et al. (2025) vs. Business as usual |
0 Years |
Orange Designation for High School Graduation (Year 4 Sample);
|
91.30 |
90.00 |
No |
-- | ||
|
High School Graduation |
California school accountability ratings - Atchison et al. (2025) vs. Business as usual |
0 Years |
Red Designation for High School Graduation (Year 1 Sample);
|
78.90 |
76.80 |
No |
-- | ||
|
High School Graduation |
California school accountability ratings - Atchison et al. (2025) vs. Business as usual |
0 Years |
Orange Designation for High School Graduation (Year 1 Sample);
|
89.20 |
88.60 |
No |
-- | ||
|
High School Graduation |
California school accountability ratings - Atchison et al. (2025) vs. Business as usual |
0 Years |
Orange Designation for High School Graduation (Year 3 Sample);
|
88.60 |
87.90 |
No |
-- | ||
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
||
|---|---|---|---|---|---|---|---|---|---|
|
Completed courses (A-G requirements) required for admission to the University of California |
California school accountability ratings - Atchison et al. (2025) vs. Business as usual |
0 Years |
Red Designation for College and Career Readiness (Overall Sample);
|
N/A |
N/A |
Yes |
|
||
|
College credit attainment in high school |
California school accountability ratings - Atchison et al. (2025) vs. Business as usual |
0 Years |
Red Designation for College and Career Readiness (Overall Sample);
|
N/A |
N/A |
Yes |
|
||
| Show Supplemental Findings | |||||||||
|
Completed courses (A-G requirements) required for admission to the University of California |
California school accountability ratings - Atchison et al. (2025) vs. Business as usual |
0 Years |
Red Designation for College and Career Readiness (Year 4 Sample);
|
22.00 |
13.70 |
Yes |
|
||
|
Completed courses (A-G requirements) required for admission to the University of California |
California school accountability ratings - Atchison et al. (2025) vs. Business as usual |
0 Years |
Red Designation for College and Career Readiness (Year 2 Sample);
|
19.70 |
14.50 |
Yes |
|
||
|
College credit attainment in high school |
California school accountability ratings - Atchison et al. (2025) vs. Business as usual |
0 Years |
Red Designation for College and Career Readiness (Year 2 Sample);
|
8.90 |
5.70 |
No |
-- | ||
|
Completed courses (A-G requirements) required for admission to the University of California |
California school accountability ratings - Atchison et al. (2025) vs. Business as usual |
0 Years |
Red Designation for College and Career Readiness (Year 3 Sample);
|
18.00 |
13.60 |
No |
-- | ||
|
Completed courses (A-G requirements) required for admission to the University of California |
California school accountability ratings - Atchison et al. (2025) vs. Business as usual |
0 Years |
Red Designation for College and Career Readiness (Year 1 Sample);
|
17.50 |
13.70 |
No |
-- | ||
|
College credit attainment in high school |
California school accountability ratings - Atchison et al. (2025) vs. Business as usual |
0 Years |
Red Designation for College and Career Readiness (Year 3 Sample);
|
6.10 |
4.10 |
No |
-- | ||
|
College credit attainment in high school |
California school accountability ratings - Atchison et al. (2025) vs. Business as usual |
0 Years |
Red Designation for College and Career Readiness (Year 4 Sample);
|
8.70 |
6.00 |
No |
-- | ||
|
College credit attainment in high school |
California school accountability ratings - Atchison et al. (2025) vs. Business as usual |
0 Years |
Red Designation for College and Career Readiness (Year 1 Sample);
|
5.20 |
4.60 |
No |
-- | ||
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Sample Characteristics
Characteristics of study sample as reported by study author.
-
15% English language learners -
Female: 49%
Male: 51% -
Rural, Suburban, Town, Urban
-
- B
- A
- C
- D
- E
- F
- G
- I
- H
- J
- K
- L
- P
- M
- N
- O
- Q
- R
- S
- V
- U
- T
- W
- X
- Z
- Y
- a
- h
- i
- b
- d
- e
- f
- c
- g
- j
- k
- l
- m
- n
- o
- p
- q
- r
- s
- t
- u
- v
- x
- w
- y
California
-
Race Asian 10% Black 6% Other or unknown 60% White 24% -
Ethnicity Hispanic 53% Not Hispanic or Latino 47% -
Eligible for Free and Reduced Price Lunch Free or reduced price lunch (FRPL) 68% No FRPL 32%
Study Details
Setting
The study was conducted in all public high schools in California in school years 2018–19 through 2021–22.
Study sample
The study sample includes approximately 3,500 high schools and over 900,000 students. Twenty-four percent of students were White, 10 percent were Asian, and six percent were Black. Fifty-three percent of students were Hispanic. Forty-nine percent of students were female. Fifteen percent of students were English learners, 11 percent were students with disabilities, and 68 percent of students were economically disadvantaged.
Intervention Group
The study examined the impact of several school performance designations under the Every Student Succeeds Act (ESSA). California assigns color designations to schools based on a range of school quality indicators. These color ratings are public and are available for every public school in California. The color designations are assigned based on current year performance status and change from the previous year on each quality indicator. Color designations include red, orange, yellow, green, and blue designations, with red being the lowest and blue being the highest. This review focused on three designations: schools with red designations on the high school graduation indicator, schools with orange designations on the high school graduation indicator, and schools with red designations on the college and career readiness indicator. The study examined the impact of these color designations on student outcomes using a regression discontinuity design in which outcomes for students in schools just below the color threshold were compared to outcomes for students in schools just above the respective color threshold.
Comparison Group
Schools in the comparison condition fell just above the threshold for each color designation. Specifically, for schools with red designations for high school graduation, the comparison condition was composed of schools who were just above the red cutoff for high school graduation. For schools with orange designations for high school graduation, the comparison condition was composed of schools just above the orange cutoff for high school graduation. For schools with red designations for college and career readiness, the comparison condition fell just above the red cutoff for college and career readiness.
Support for implementation
In California, schools may be designated as Comprehensive Support and Improvement (CSI) schools based on their performance, including color ratings. CSI schools are entitled to receive additional federal school improvement funding. A portion of school improvement funding is provided to county offices of education to provide technical assistance for CSI schools. The study did not report the nature, intensity, or duration of technical assistance or other supports actually received by schools (for example, frequency of coaching, specific planning activities, or how funds were spent within schools).
An indicator of the effect of the intervention, the improvement index can be interpreted as the expected change in percentile rank for an average comparison group student if that student had received the intervention.
For more, please see the WWC Glossary entry for improvement index.
An outcome is the knowledge, skills, and attitudes that are attained as a result of an activity. An outcome measures is an instrument, device, or method that provides data on the outcome.
A finding that is included in the effectiveness rating. Excluded findings may include subgroups and subscales.
The sample on which the analysis was conducted.
The group to which the intervention group is compared, which may include a different intervention, business as usual, or no services.
The timing of the post-intervention outcome measure.
The number of students included in the analysis.
The mean score of students in the intervention group.
The mean score of students in the comparison group.
The WWC considers a finding to be statistically significant if the likelihood that the finding is due to chance alone, rather than a real difference, is less than five percent.
The WWC reviews studies for WWC products, Department of Education grant competitions, and IES performance measures.
The name and version of the document used to guide the review of the study.
The version of the WWC design standards used to guide the review of the study.
The result of the WWC assessment of the study. The rating is based on the strength of evidence of the effectiveness of the intervention. Studies are given a rating of Meets WWC Design Standards without Reservations, Meets WWC Design Standards with Reservations, or >Does Not Meet WWC Design Standards.
A related publication that was reviewed alongside the main study of interest.
Study findings for this report.
Based on the direction, magnitude, statistical significance, and sample size of the findings within a domain, the WWC assigns effectiveness ratings as one of the following: Tier 1 (strong evidence), Tier 2 (moderate evidence), Tier 3 (promising evidence), uncertain effects, and negative effects. For more detail, please see the WWC Handbook.
The WWC may review studies for multiple purposes, including different reports and re-reviews using updated standards. Each WWC review of this study is listed in the dropdown. Details on any review may be accessed by making a selection from the drop down list.
Tier 1 Strong indicates strong evidence of effectiveness,
Tier 2 Moderate indicates moderate evidence of effectiveness, and
Tier 3 Promising indicates promising evidence of effectiveness,
as defined in the
non-regulatory guidance for ESSA
and the regulations for ED discretionary grants (EDGAR Part 77).