|Title:||Bayesian Inference for Experimental and Observational Studies in Education|
|Principal Investigator:||Kaplan, David||Awardee:||University of Wisconsin, Madison|
|Program:||Statistical and Research Methodology in Education [Program Details]|
|Award Period:||3 years||Award Amount:||$566,397|
|Goal:||Methodological Innovation||Award Number:||R305D110001|
The purpose of this project is to develop, apply, and disseminate Bayesian statistical tools for designs and analytic strategies used in empirical education research. This will include: (1) randomized experimental designs; (2) quasi-experimental/observational designs; and (3) longitudinal studies. In addition, the project will compare different types of prior information that may be encountered in practical circumstances, and provide guidance into best practices in Bayesian statistical modeling for the education sciences. Finally, this project will use or develop open-source software to carry out this work and this software will be made readily available for future researchers.
The project has three parts. In Part I, the researchers will examine the utility of Bayesian inference for randomized experiments in education. From a Bayesian perspective, experiments are dynamic, involving decisions that often rest on prior knowledge gleaned from previous studies (e.g., meta-analyses) or subjective judgments of experts. The project will review a select set of randomized experiments in education and reanalyze previous studies within a Bayesian framework. Special attention will be paid to the use of Bayesian informative hypotheses, which allow for tests of directional hypotheses based on prior research and can mitigate the classical concern over Type I error control. Bayesian analysis of variance falls into the class of procedures that can be used to test informative hypotheses. A focused set of simulation studies will be done to provide direct comparisons between Bayesian and classical approaches to issues of design and hypothesis testing. In addition, Bayesian analysis of variance will be extended to cluster randomized designs. . Existing data from a randomized field trial of a large scale reform that provided professional development in Los Angeles and from an evaluation of the Success-for-All intervention will be used to supplement the simulation studies.
In Part II, the researchers will examine a Bayesian approach to quasi-experimental/observational studies. The researchers will focus particularly on the use of the propensity score adjustment procedures for addressing nonequivalence in such studies. Classical approaches to propensity score adjustment cannot account for uncertainties in model parameters or model choice, either of which can affect causal inferences after propensity score adjustment. The project will compare the Bayesian propensity score adjustment with varying prior probabilities on model parameters that reflect reasonable choices found in education settings. This will include situations in which little or no prior information is available. Specifically, a Bayesian approach to propensity score stratification, weighting, and optimal matching will be developed. The researchers will also examine Bayesian model averaging as a means of drawing strength from various realistic models for the propensity score. Analysis of data from the Early Childhood Longitudinal Study will be used to supplement simulation studies.
Part III will extend recent developments in Bayesian growth mixture modeling to the general growth mixture modeling case. In particular, researchers will examine piecewise growth mixture modeling. They will then examine the properties of the Bayesian piecewise growth mixture model under a variety of realistic conditions through simulation studies. Next, the Bayesian piecewise growth mixture modeling will be applied to samples of data drawn from the World-class Instructional Design and Assessment Consortium that addresses response to intervention for English language learners.
Publications from this project:
Kaplan, D., and Chen, J. (2012). A Two-Step Bayesian Approach for Propensity Score Analysis: Simulations and Case Study. Psychometrika, 77: 581–609.
Kaplan, D., and Depaoli, S. (2012). Bayesian Structural Equation Modeling. In R. Hoyle (ed.), Handbook of Structural Equation Modeling. (pp 650–673), New York: Guilford Publications, Inc.
Kaplan, D., and Depaoli, S. (2013). Bayesian Statistical Methods. In T.D. Little (Ed.), The Oxford Handbook of Quantitative Methods (Vol 1): Foundations (pp. 407–437). New York, NY US: Oxford University Press.
Kaplan, D., and Park, S. (in press). Analyzing International Large-Scale Assessment Data Within a Bayesian Framework. In Rutkowski, L., von Davier, M., and Rutkowski, D. (eds.) A Handbook Of International Large-Scale Assessment: Background, Technical Issues, and Methods Of Data Analysis. London: Chapman Hall/CRC Press.