Project Activities
An important step in evaluating the effectiveness of interventions requires an understanding of the mechanisms by which interventions bring about change. While the study of mediating mechanisms is largely unexplored in single-case research, the theories that guide the expected behavior change often make note of why, or through which mechanisms, the intervention is expected to produce change. Little attention has been devoted to the definition, identification, and estimation of causal mediation effects in SCEDs. Additionally, there are no studies describing or evaluating optimal approaches for synthesizing causal mediation effects across cases in a SCED. Quantitative data synthesis of treatment effects in SCEDs results in summarizing information about the population treatment effect, variability between individuals, and potential moderators of treatment effects. Therefore, an important step is to combine causal mediation methods and data synthesis methods to estimate and synthesize causal mediation effects across SCED participants.
This project will bridge the gulf between methods traditional in SCEDs research and cutting-edge methods for estimating causal mediation effects by completing three aims. First, the team will analytically derive and empirically validate the one-stage multilevel model for synthesis of causal mediation effects. Second, the team will empirically validate the two-stage multilevel model for synthesis of causal mediation effects. Finally, the team will empirically compare the performance of the one-stage and two-stage approach for commonly encountered between-case heterogeneity in mediating processes, trends, and serial dependencies. The project team will derive new multilevel models for SCEDs, implement them in R and R Shiny, and will share their results in academic publications and workshop presentations. The team will also release documentation and tutorials for the software.
Structured Abstract
Research design and methods
This project team will complete its aims, first through analytical derivations then through empirical evaluation using Monte Carlo simulations. In these simulations, hierarchically structured SCED data will be generated under realistic design conditions and population values chosen based on review studies summarizing characteristics of SCEDs and previous simulation studies.
User Testing: The project team will develop and release a R Shiny application to implement the multilevel models for SCEDs. To increase the likelihood of researchers using the application, the project team will write a user-friendly tutorial paper that will demonstrate the application of the proposed methods in this project and the interpretation of results from the proposed methods in the context of empirical datasets. The team will also evaluate the R shiny application based on its ease of use for educational researchers. First, the application will be tested on simulated datasets to ensure the R application produces the correct effect estimates, standard errors, and confidence intervals. Second, the application will be applied to empirical data examples to demonstrate the application and interpretation of the newly developed methods for methodological and empirical researchers. Third, the application will be tested by all key personnel and funded graduate students to assess the user friendliness of the application and to identify and resolve any programming errors.
Use in Applied Education Research: The findings will benefit two groups of education researchers. The first group is empirical educational researchers who make use of cutting-edge statistical methods to answer research questions in SCEDs. For empirical researchers, the statistical methods that will be developed in this proposal will help address questions of why an intervention resulted in changes in educational outcomes. With the tools developed in this proposal, educational researchers will be able to better answer questions such as "did the intervention reduce anxiety through its effect on increasing coping?" The second group is methodologists and statisticians who conduct theoretical research on single-case experiments and causal mediation analysis and who serve as analysts on substantive projects requiring such models. The researchers may need statistical models and causal assumptions to estimate and interpret causal mediation effects.
People and institutions involved
IES program contact(s)
Project contributors
Products and publications
Products: Products include analytically derived and empirically validated causal mediation effect definitions using the one-stage multilevel model for data synthesis and empirically validated causal mediation effect definitions using the two-stage multilevel model for data synthesis. Additionally, the project team will produce an empirical comparison and recommendations for using either the one-stage or two-stage approach for data synthesis in the presence of heterogeneity in mediating processes, trends, and serial dependencies. The team will also produce an R shiny application for estimating and synthesizing causal mediation effects using one-stage and two-stage data synthesis approaches. The research team will produce publications accessible to both methodological and applied education researchers in quantitative and applied research journals, tutorial papers, and conference presentations and workshops. Further, education researchers will have access to all code and the R shiny application through file sharing websites. Specifically, the R application will be demonstrated in a workshop or symposium at a national conference (e.g., AERA) and shared via the project team's OSF repositories and ResearchGate sites.
ERIC Citations: Find available citations in ERIC for this award here.
Supplemental information
Co-Principal Investigators: Moyaert, Mariola S.; Miocevic, Milicia
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
