Project Activities
Researchers will conduct a historical analysis of exit exams, up to and including the graduating class of 2019, in years 1-3 of the project (2019-2021). Beginning in fall 2019, the team will track implementation and impacts for the new HSEE policy; this analysis will be ongoing for the duration of the project.
Structured Abstract
Setting
The research will take place across the state of Massachusetts.
Sample
Researchers will analyze data on all high school test takers in the state, approximately 70,000 per year, from the class of 2004 through the class of 2023. The total sample of approximately 1,400,000 students will include significant numbers of students who have had difficulty with HSEEs including English learners, students with disabilities, and low-income students.
The intervention includes three different generations of the Massachusetts HSEE: the 2002 version, which was the first test students were required to pass; the 2010 version, which increased the math and ELA pass scores and added a science test requirement; and the 2020 version, aligned to the state's College and Career Readiness framework.
Initial research
Researchers will use a regression discontinuity design (RDD) to evaluate the impacts of test failure. One of the primary interests is the difference in subsequent outcomes for students who fail a test by a narrow margin. The RDD design will show the comparison of outcomes between students with scores just above and below the threshold passing score. Researchers will use a difference-in-differences (DiD) design to evaluate the effect of historical changes to the HSEE policy, such as increases to the threshold passing scores in 2010, as well as an entirely new test beginning in spring 2020. In these models, researchers will assess the effects of a policy change for students with test scores in a range affected by a given policy change. In both sets of models, the researchers will assess whether impacts differ across key demographic groups. In the implementation analysis, researchers will obtain a representative sample of schools across the state and will employ quasi-experimental and descriptive research designs to provide rapid feedback to the state as it fine-tunes implementation of the new HSEE policy.
Control condition
In the RDD models, the control condition will be students who narrowly passed an HSEE. In the DiD models, the control condition will be students unaffected by a policy change.
Key measures
The impact evaluation will focus on key student education outcomes, including educational attainments (high school graduation, college-going, college completion), intermediate outcomes (attendance, college remediation), and longer-term life outcomes (criminal justice involvement, labor market participation, earnings). The implementation study will gather data on district and school procedures for supporting students who fail an HSEE in tenth grade.
Data analytic strategy
Researchers will useregression models with fixed effects for cohorts and, in some analyses, for high schools.
Cost analysis strategy
Researchers will conduct a cost analysis (using the ingredients method) to take account of costs associated with implementing HSEEs and their related supports. They will then compare these costs to the benefits of the policy.
People and institutions involved
IES program contact(s)
Project contributors
Partner institutions
Brown University
Massachusetts Department of Elementary and Secondary Education
Products and publications
Products: Researchers will generate historical evidence on the impacts of HSEEs for key student groups and real-time evidence on the short-term effects and implementation procedures for new HSEEs. The team will share their findings through research briefs targeted toward policymakers, and through working papers and peer-reviewed publications.
Related projects
Supplemental information
Co-Principal Investigators: Wulfson, Jeff; Murnane, Richard
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
