Project Activities
The team will produce a practice guide, a primer, and a handbook for the use of these methods, which are used to improve the quality of statistical inferences made in quasi-experimental studies, where randomized assignment is not possible.
Structured Abstract
Research design and methods
The practice guide will cover propensity score matching for two-level and three-level multisite designs, two-level and three-level cluster designs, and three-level multisite cluster designs. Both the primer and handbook will be structured to walk users through the different stages of the propensity score matching process for each design. The primer will be based on a synthesis of the relevant methodological literature to provide users with foundational knowledge about propensity score matching and a review of key considerations when applying propensity score matching to multilevel educational settings. To help researchers implement the most appropriate matching method for their study, the handbook will use real-world data and simulated data to illustrate how to conduct matching for different multilevel designs, including supporting R code.
User Testing: The project includes the following user testing to determine whether the product works as intended and whether education researchers can use the product: (a) internal usability trials, (b) feedback from a technical advisory group, and (c) end-user testing.
Use in Applied Education Research: While methodological advances over the past decade have improved our understanding of the issues researchers must consider when using matching in multilevel settings, that information has not been consolidated and communicated for applied education researchers. The proposed practice guide will provide education researchers with a resource that can improve the design of their studies and interpretation of the findings. In particular, the practice guide can help researchers address threats to internal validity and more explicitly consider treatment effect heterogeneity and place-based contextual factors as part of the study design process.
People and institutions involved
IES program contact(s)
Project contributors
Products and publications
The team will produce a practice guide, a primer and a handbook. The products will address the need for more comprehensive and education-focused guidance regarding the use of matching methods in multilevel educational settings. Education researchers will have access to the practice guide through Quarto Pub, and accompanying R code and data will be available on GitHub. The project team will create a dedicated project page on AIR's website that provides an overview of the project and directs users to the Quarto Pub and GitHub sites. To facilitate education researcher use of the practice guide, the project team will host a virtual training and an in-person workshop (at an education research conference) that provides an overview of the practice guide's main features and walks participants through the illustrative examples. The virtual training will be recorded and edited into a series of instructional videos that cover each section of the practice guide. The instructional videos will be hosted on AIR's website.
Publications:
ERIC Citations: Find available citations in ERIC for this award here.
Supplemental information
Co-Principal Investigators: Joshi, Megha; Zhang, Qi
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
