Skip to main content

Breadcrumb

Home arrow_forward_ios Information on IES-Funded Research arrow_forward_ios Methods for Investigating Causal Me ...
Home arrow_forward_ios ... arrow_forward_ios Methods for Investigating Causal Me ...
Information on IES-Funded Research
Grant Open

Methods for Investigating Causal Mechanisms in Single-Case Experimental Designs

NCER
Program: Statistical and Research Methodology in Education
Program topic(s): Core
Award amount: $896,181
Principal investigator: Matthew Valente
Awardee:
University of South Florida
Year: 2024
Award period: 3 years (09/01/2024 - 08/31/2027)
Project type:
Methodological Innovation
Award number: R305D240044

Purpose

The goals of this project are to provide single-case researchers with statistical methodology to help understand how interventions produce change in behaviors and/or academic outcomes and how to synthesize this information across cases in a Single-Case Experimental Design (SCED). Interventions to encourage positive classroom behavior and academic outcomes are critical to providing quality education to the American public. Often classroom interventions are implemented on individual students in the form of SCEDs which consist of repeated measurements of an outcome over time for a single case and are considered a type of interrupted time series design. SCEDs are useful when a target population is comprised of a small percentage of individuals thus making a randomized controlled trial infeasible for assessing intervention effectiveness.

Project Activities

An important step in evaluating the effectiveness of interventions requires an understanding of the mechanisms by which interventions bring about change. While the study of mediating mechanisms is largely unexplored in single-case research, the theories that guide the expected behavior change often make note of why, or through which mechanisms, the intervention is expected to produce change. Little attention has been devoted to the definition, identification, and estimation of causal mediation effects in SCEDs. Additionally, there are no studies describing or evaluating optimal approaches for synthesizing causal mediation effects across cases in a SCED. Quantitative data synthesis of treatment effects in SCEDs results in summarizing information about the population treatment effect, variability between individuals, and potential moderators of treatment effects. Therefore, an important step is to combine causal mediation methods and data synthesis methods to estimate and synthesize causal mediation effects across SCED participants.

This project will bridge the gulf between methods traditional in SCEDs research and cutting-edge methods for estimating causal mediation effects by completing three aims. First, the team will analytically derive and empirically validate the one-stage multilevel model for synthesis of causal mediation effects. Second, the team will empirically validate the two-stage multilevel model for synthesis of causal mediation effects. Finally, the team will empirically compare the performance of the one-stage and two-stage approach for commonly encountered between-case heterogeneity in mediating processes, trends, and serial dependencies. The project team will derive new multilevel models for SCEDs, implement them in R and R Shiny, and will share their results in academic publications and workshop presentations. The team will also release documentation and tutorials for the software.

Structured Abstract

Research design and methods

This project team will complete its aims, first through analytical derivations then through empirical evaluation using Monte Carlo simulations. In these simulations, hierarchically structured SCED data will be generated under realistic design conditions and population values chosen based on review studies summarizing characteristics of SCEDs and previous simulation studies.

User Testing: The project team will develop and release a R Shiny application to implement the multilevel models for SCEDs. To increase the likelihood of researchers using the application, the project team will write a user-friendly tutorial paper that will demonstrate the application of the proposed methods in this project and the interpretation of results from the proposed methods in the context of empirical datasets. The  team will also evaluate the R shiny application based on its ease of use for educational researchers. First, the application will be tested on simulated datasets to ensure the R application produces the correct effect estimates, standard errors, and confidence intervals. Second, the application will be applied to empirical data examples to demonstrate the application and interpretation of the newly developed methods for methodological and empirical researchers. Third, the application will be tested by all key personnel and funded graduate students to assess the user friendliness of the application and to identify and resolve any programming errors.

Use in Applied Education Research: The findings will benefit two groups of education researchers. The first group is empirical educational researchers who make use of cutting-edge statistical methods to answer research questions in SCEDs. For empirical researchers, the statistical methods that will be developed in this proposal will help address questions of why an intervention resulted in changes in educational outcomes. With the tools developed in this proposal, educational researchers will be able to better answer questions such as "did the intervention reduce anxiety through its effect on increasing coping?"  The second group is methodologists and statisticians who conduct theoretical research on single-case experiments and causal mediation analysis and who serve as analysts on substantive projects requiring such models. The researchers may need statistical models and causal assumptions to estimate and interpret causal mediation effects.

People and institutions involved

IES program contact(s)

Charles Laurin

Education Research Analyst
NCER

Project contributors

Mariola Moeyaert

Co-principal investigator

Products and publications

Products: Products include analytically derived and empirically validated causal mediation effect definitions using the one-stage multilevel model for data synthesis and empirically validated causal mediation effect definitions using the two-stage multilevel model for data synthesis.  Additionally, the project team will produce an empirical comparison and recommendations for using either the one-stage or two-stage approach for data synthesis in the presence of heterogeneity in mediating processes, trends, and serial dependencies. The team will also produce an R shiny application for estimating and synthesizing causal mediation effects using one-stage and two-stage data synthesis approaches. The research team will produce publications accessible to both methodological and applied education researchers in quantitative and applied research journals, tutorial papers, and conference presentations and workshops. Further, education researchers will have access to all code and the R shiny application through file sharing websites. Specifically, the R application will be demonstrated in a workshop or symposium at a national conference (e.g., AERA) and shared via the project team's OSF repositories and ResearchGate sites.

ERIC Citations: Find available citations in ERIC for this award here.

Supplemental information

Co-Principal Investigators: Moyaert, Mariola S.; Miocevic, Milicia

 

Questions about this project?

To answer additional questions about this project or provide feedback, please contact the program officer.

 

Tags

DisabilitiesSocial/Emotional/Behavioral

Share

Icon to link to Facebook social media siteIcon to link to X social media siteIcon to link to LinkedIn social media siteIcon to copy link value

Questions about this project?

To answer additional questions about this project or provide feedback, please contact the program officer.

 

You may also like

Zoomed in Yellow IES Logo
Grant

Longitudinal Relations Among Social Contexts, Bull...

Award number: R305A230406
Read More
Rectangle Blue 1 Pattern 1
Descriptive Study

Exploring Implementation of Trauma-Engaged Practic...

Author(s): Ashley Boal, Shannon McCullough, Angela Chin
Read More
Zoomed in IES logo
Request for Applications

Special Education Dissertation Research Fellowship...

November 14, 2024
Read More
icon-dot-govicon-https icon-quote