Project Activities
Researchers will carry out a series of secondary data analyses using a comparative regression discontinuity design to explore relationships between test performance and feedback. They will also complete and examine a set of interviews and case studies.
Structured Abstract
Setting
Researchers will use data drawn from all districts and high schools in Delaware's public and charter education system and five large colleges/universities across the state: University of Delaware, Delaware State University, Wilmington University, Delaware Technical and Community College, and Wesley University.
Sample
The study sample is a longitudinal sample of more than 90,000 students enrolled in 32 Delaware high schools from 19 school districts with an expected graduation year between 2014 and 2025 (that is, 12 cohorts consisting of 8,000 to 9,000 students each).
Factors
Researchers will examine Delaware's implementation of the Smarter Balanced Assessments (SBA) in 8th grade, the PSAT/SAT in 10th and 11th grades, and associated feedback and support mechanisms that are intended to act as a feedback mechanism for students and educators and to influence students' expectations, choices, and coursework trajectories during high school and into college.
Research design and methods
Researchers are blending elements of regression discontinuity (RD) and comparative interrupted time-series (CITS) with a mixed methods approach using a sequential explanatory design with an emphasis on quantitative methods. The comparative RD model will allow the researchers to test for discontinuities in outcomes for students near the SBA "proficiency" cut score and PSAT/SAT "college-ready" cut scores, while also confirming that a discontinuity near that same point in the achievement distribution did not exist for prior cohorts of students taking the previous state test. The data used in this study will come from (a) Delaware's iMart-EdInsight statewide longitudinal data system (SLDS), (b) the National Student Clearinghouse database, (c) an annual survey of high schools and districts, (d) interviews with a statewide sample of 9th and 12th grade students, and (e) case studies of 5 high schools, including interviews with students, teachers, school counselors, and district administrators.
Control condition
There are two comparison conditions in this study. One comparison consists of Delaware high school students from the 2014 through 2018 graduating cohorts who never experienced the 8th grade Smarter Balanced Assessments or the associated feedback. The other consists of Delaware high school students from the 2016 through 2022 graduating cohorts who scored just above or just below the "college-ready" cut score on the SAT in 11th grade.
Key measures
Key outcome measures include students' high school course-taking patterns, PSAT/SAT scores, college enrollment, remedial courses, and college persistence.
Data analytic strategy
Researchers will carry out multi-cohort hierarchical linear and non-linear models of education outcomes for students scoring near assessment cut scores. They will use moderation analyses to explore variation in impacts by student subgroups and district curricula and supports. They will analyze the qualitative data via concurrent-mixed analysis to inform interpretation of quantitative results and design of subsequent analyses.
People and institutions involved
IES program contact(s)
Products and publications
Products: The research team will produce preliminary evidence of the relationship between feedback and support mechanisms provided in response to student performance on the Smarter Balanced Assessments (SBA) in 8th grade, the PSAT/SAT in 10th and 11th grades, and students' educational trajectories through high school and into college. The researchers will also produce conference presentations, peer reviewed publications, and policy briefs.
Supplemental information
Co-Principal Investigator: Klein, Jeff
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
