Co-Principal Investigators: Kang, Hyeon-Ah; Sales, Adam

Unlike their more traditional counterparts, computer-based interventions typically allow researchers and administrators to collect implementation data automatically in the form of log or clickstream data. Log data from technology RCTs present an unprecedented opportunity for researchers to use the fine-grained and rich data to help understand how, why, and when online interventions work. Log data, however, present a challenge, in that they differ in structure and size from data commonly encountered in studies of causal mechanisms. Computer log data are highly multivariate, multilevel, and messy. Statistical methods developed for computer log data do not adapt easily to causal models.

The purpose of this grant is to develop fully latent principal stratification (FLPS), a data science tool for modeling large complex multivariate implementation data from randomized controlled trials (RCTs) in education. FLPS combines principal stratification and structural equation modeling techniques to examine treatment effect heterogeneity as a function of implementation constructs measured by longitudinal, multidimensional, and mixed-type data, such as computer log data from educational technology applications. The researchers will formulate FLPS models for a variety of measurement scenarios and study their feasibility, estimation properties, robustness, and associated model-checking techniques. They will also develop and disseminate a set of vignettes with worked examples and open-source code in R that will make it possible for practitioners to fit and make inferences from multivariate intermediate data using FLPS models. The research team will publish in peer-reviewed journals, present at conferences, and provide workshops at conferences and universities on FLPS models and the software for estimating them.