Co-Principal Investigator: Chen, Jianshen

Purpose: In education research, long-standing, large-scale educational assessments, such as the National Assessment of Educational Progress (NAEP ), the Programme for International Student Assessment (PISA), and the Early Childhood Longitudinal Program (ECLS), provide historical data that can be used to inform policy-relevant research. However, statistical analyses of current large-scale assessment data may not make use of this historical information, and it may be difficult to do so when applying conventional statistical methods. This project developed and adapted the method of Bayesian dynamic borrowing (Viele et al., 2014) as a means of systematically incorporating prior historical data arising from large-scale education assessments into current analyses. Bayesian dynamic borrowing allows a researcher to account for the fact that not all historical data, even from the same survey program, are of equal quality. As such, prior information can be systematically adjusted to reflect the analyst’s degree-of-confidence in the importance and/or quality of sources of prior data.

Project Activities: The project team implemented Bayesian dynamic borrowing under both the single-level and multilevel settings characteristic of large-scale assessment data and extended Bayesian dynamic borrowing to longitudinal designs focusing on growth over time.

Key Outcomes

• The researchers developed an application software program to conduct Bayesian dynamic borrowing called Shiny BHP App, which is available at https://github.com/Bayesian-Methods-for-Education-Research/ShinyBHB
• Details about applying Bayesian dynamic borrowing to large scale and longitudinal assessment were released in peer reviewed publications (Kaplan et al., 2022; Kaplan et al., 2023)

Structured Abstract

Statistical/Methodological Product: This project developed an application software program to conduct Bayesian dynamic borrowing for complex sampling designs for cross-sectional and longitudinal data found in education research.

Development/Refinement Process: Bayesian dynamic borrowing has been primarily developed in the clinical trials literature, where information is borrowed from control groups of previous studies in order to increase power. This project expanded Bayesian dynamic borrowing to complex sampling designs for cross-sectional and longitudinal data found in education research. The work required development of statistical models for the complexity of the data encountered in education and the use of new probabilistic modeling tools, such as the Stan programming environment, which underlies the application program.

Related IES Projects: Bayesian Inference for Experimental and Observational Studies in Education (R305D110001)

Products and Publications

ERIC Citations:  Find available citations in ERIC for this award here.

Project Website: http://bise.wceruw.org/index.html

Additional Online Resources and Information:

• Shiny BHP App: https://github.com/Bayesian-Methods-for-Education-Research/ShinyBHB

Select Publications:

Journal articles

Kaplan, D. & Chen, J. & Yavuz, S. & Lyu, W. (2022). Bayesian dynamic borrowing of historical information with applications to the analysis of large-scale assessments. Psychometrika., 88(1)1-30. doi.org/10.1007/s11336-022-09869-3

Kaplan, D., Chen, J., Lyu, W. and Yavuz, S. (2023) Bayesian historical borrowing with longitudinal large-scale assessments Large-scale Assessments in Education, volume 11(1),2 doi.org/10.1186/s40536-022-00140-w

Viele, K., Berry, S., Neuenschwander, B., Amzal, B., Chen, F., Enas, N., ... & Thompson, L. (2014). Use of historical control data for assessing treatment effects in clinical trials. Pharmaceutical statistics, 13(1), 41–54.