Co-Principal Investigators: Maroulis, Spiro; Saw, Guan; Xu, Ran; Rosenberg, Joshua

The purpose of this project is to advance, extend, and apply existing sensitivity analysis techniques to make them most useful for education research and practice. Specifically, the researchers will expand the Impact Threshold for a Confounding Variable (ITCV) and the Robustness of Inference to Replacement (RIR) approaches to a variety of designs and design features, such as differential attrition, moderation analyses, and regression discontinuity designs. In doing so, the researchers hope to provide a more precise language for researchers advocating a causal inference and challengers of that inference to interpret impacts by debating the strength of the evidence relative to concerns about potential violations of the assumptions of the inference.

The research team will first extend the theoretical frameworks for sensitivity analysis and test the extensions via Monte Carlo simulation. They will then add the new analyses to existing software in R, Shiny (KonFound-It!), and Stata, along with user's guides to help applied researchers use the new functions in the software packages. The team will also create a searchable database that generates a reference distribution for commonly used sensitivity analysis measures for studies that meet the What Works Clearinghouse standards (with and without reservations). As additional means of dissemination, the researchers will conduct workshops, present at conferences, and publish in peer-reviewed journals.