|Title:||Consequences of Selective Reporting Bias in Education Research|
|Principal Investigator:||Citkowicz, Martyna||Awardee:||American Institutes for Research (AIR)|
|Program:||Statistical and Research Methodology in Education [Program Details]|
|Award Period:||3 years (09/01/2022 – 08/31/2025)||Award Amount:||$896,931|
|Type:||Methodological Innovation||Award Number:||R305D220026|
Co-Principal Investigators: Polanin, Joshua; Pustejovsky, James; Williams, Ryan;
The most popular meta-analytic methods have serious limitations in diagnosing and adjusting for selective reporting, especially when there are dependencies among multiple effects from primary studies, a widespread occurrence. For meta-analysis of independent effects, selection models have shown promise in flexibly capturing complex reporting patterns while providing adjusted meta-analytic estimates, but no existing model also simultaneously addresses effect size dependencies. The purpose of this project is to develop two selection models that will simultaneously account for selective reporting and effect size dependencies.
This model is based on the beta-density distribution and the other model is based on theoretically important p-value cut points. After these models are derived, their performance will be tested via Monte Carlo simulation studies under varying conditions, such as different sample sizes and different degrees of selection bias. The research team will then develop an R software package and a Shiny app for meta-analysis researchers to apply the new models to their own research. In order to help researchers use the new software, the project team will prepare several tutorials, prepare a webinar, and hold in-person trainings. They will also conduct and publish the results of a second-order meta-analysis.