Co-Principal Investigator: Klein, Jeff

Purpose: The purpose of this project is to identify the mechanisms through which local curricula and course offerings, statewide assessments, and feedback on predicted college-readiness might be better leveraged to influence students' educational trajectories through high school and into college.

Project Activities: Researchers will carry out a series of secondary data analyses using a comparative regression discontinuity design to explore relationships between test performance and feedback. They will also complete and examine a set of interviews and case studies.

Products: The research team will produce preliminary evidence of the relationship between feedback and support mechanisms provided in response to student performance on the Smarter Balanced Assessments (SBA) in 8th grade, the PSAT/SAT in 10th and 11th grades, and students' educational trajectories through high school and into college. The researchers will also produce conference presentations, peer reviewed publications, and policy briefs.

Structured Abstract

Setting: Researchers will use data drawn from all districts and high schools in Delaware's public and charter education system and five large colleges/universities across the state: University of Delaware, Delaware State University, Wilmington University, Delaware Technical and Community College, and Wesley University.

Sample: The study sample is a longitudinal sample of more than 90,000 students enrolled in 32 Delaware high schools from 19 school districts with an expected graduation year between 2014 and 2025 (that is, 12 cohorts consisting of 8,000 to 9,000 students each).

Factors: Researchers will examine Delaware's implementation of the Smarter Balanced Assessments (SBA) in 8th grade, the PSAT/SAT in 10th and 11th grades, and associated feedback and support mechanisms that are intended to act as a feedback mechanism for students and educators and to influence students' expectations, choices, and coursework trajectories during high school and into college.

Research Design and Methods: Researchers are blending elements of regression discontinuity (RD) and comparative interrupted time-series (CITS) with a mixed methods approach using a sequential explanatory design with an emphasis on quantitative methods. The comparative RD model will allow the researchers to test for discontinuities in outcomes for students near the SBA "proficiency" cut score and PSAT/SAT "college-ready" cut scores, while also confirming that a discontinuity near that same point in the achievement distribution did not exist for prior cohorts of students taking the previous state test. The data used in this study will come from (a) Delaware's iMart-EdInsight statewide longitudinal data system (SLDS), (b) the National Student Clearinghouse database, (c) an annual survey of high schools and districts, (d) interviews with a statewide sample of 9th and 12th grade students, and (e) case studies of 5 high schools, including interviews with students, teachers, school counselors, and district administrators.

Comparison Condition: There are two comparison conditions in this study. One comparison consists of Delaware high school students from the 2014 through 2018 graduating cohorts who never experienced the 8th grade Smarter Balanced Assessments or the associated feedback. The other consists of Delaware high school students from the 2016 through 2022 graduating cohorts who scored just above or just below the "college-ready" cut score on the SAT in 11th grade.

Key Measures: Key outcome measures include students' high school course-taking patterns, PSAT/SAT scores, college enrollment, remedial courses, and college persistence.

Data Analytic Strategy: Researchers will carry out multi-cohort hierarchical linear and non-linear models of education outcomes for students scoring near assessment cut scores. They will use moderation analyses to explore variation in impacts by student subgroups and district curricula and supports. They will analyze the qualitative data via concurrent-mixed analysis to inform interpretation of quantitative results and design of subsequent analyses.