Skip to main content

Breadcrumb

Home arrow_forward_ios Information on IES-Funded Research arrow_forward_ios Quantifying the Robustness of Causa ...
Home arrow_forward_ios ... arrow_forward_ios Quantifying the Robustness of Causa ...
Information on IES-Funded Research
Grant Open

Quantifying the Robustness of Causal Inferences: Extensions and Application to Existing Databases

NCER
Program: Statistical and Research Methodology in Education
Program topic(s): Core
Award amount: $899,319
Principal investigator: Kenneth Frank
Awardee:
Michigan State University
Year: 2022
Award period: 3 years (08/16/2022 - 08/15/2025)
Project type:
Methodological Innovation
Award number: R305D220022

Purpose

The purpose of this project is to advance, extend, and apply existing sensitivity analysis techniques to make them most useful for education research and practice. Specifically, the researchers will expand the Impact Threshold for a Confounding Variable (ITCV) and the Robustness of Inference to Replacement (RIR) approaches to a variety of designs and design features, such as differential attrition, moderation analyses, and regression discontinuity designs. In doing so, the researchers hope to provide a more precise language for researchers advocating a causal inference and challengers of that inference to interpret impacts by debating the strength of the evidence relative to concerns about potential violations of the assumptions of the inference.

Project Activities

The research team will first extend the theoretical frameworks for sensitivity analysis and test the extensions via Monte Carlo simulation. They will then add the new analyses to existing software in R, Shiny (KonFound-It!), and Stata, along with user's guides to help applied researchers use the new functions in the software packages.

People and institutions involved

IES program contact(s)

Charles Laurin

Education Research Analyst
NCER

Project contributors

Guan Saw

Co-principal investigator

Products and publications

The team will also create a searchable database that generates a reference distribution for commonly used sensitivity analysis measures for studies that meet the What Works Clearinghouse standards (with and without reservations). As additional means of dissemination, the researchers will conduct workshops, present at conferences, and publish in peer-reviewed journals.

Publications:

Frank, K., Lin, Qi., Maroulis, S., Dai, S., Jess, N., Lin, H.-C. Liu, Y., Maestrales, S., Searle, E., and Tait, J., (2022). Improving Oster's d*: Exact Calculation for the Coefficient of Proportionality Without Subjective Specification of a Baseline Model. SSRN. Available at SSRN: https://ssrn.com/abstract=4305243 or http://dx.doi.org/10.2139/ssrn.4305243

Frank, K.A., Lin, Q., Xu, R., Maroulis, S., and Mueller, M. (2023). Quantifying the robustness of causal inferences: Sensitivity analysis for pragmatic social science, Social Science Research, 110, 102815.

Lin, Q., Nuttall, A. K., Zhang, Q., & Frank, K. A. (2023). How do unobserved confounding mediators and measurement error impact estimated mediation effects and corresponding statistical inferences? Introducing the R package ConMed for sensitivity analysis. Psychological Methods, 28(2), 339-358.

Supplemental information

Co-Principal Investigators: Maroulis, Spiro; Saw, Guan; Xu, Ran; Rosenberg, Joshua

Questions about this project?

To answer additional questions about this project or provide feedback, please contact the program officer.

 

Tags

Data and Assessments

Share

Icon to link to Facebook social media siteIcon to link to X social media siteIcon to link to LinkedIn social media siteIcon to copy link value

Questions about this project?

To answer additional questions about this project or provide feedback, please contact the program officer.

 

You may also like

Zoomed in IES logo
Workshop/Training

Data Science Methods for Digital Learning Platform...

August 18, 2025
Read More
Zoomed in IES logo
Workshop/Training

Meta-Analysis Training Institute (MATI)

July 28, 2025
Read More
Zoomed in Yellow IES Logo
Workshop/Training

Bayesian Longitudinal Data Modeling in Education S...

July 21, 2025
Read More
icon-dot-govicon-https icon-quote