Skip Navigation
Funding Opportunities | Search Funded Research Grants and Contracts

IES Grant

Title: Investigating the Potential of Machine Learning Methods for Identifying Impact Variation in Randomized Control Trials
Center: NCER Year: 2022
Principal Investigator: Zhu, Pei Awardee: MDRC
Program: Statistical and Research Methodology in Education      [Program Details]
Award Period: 3 years (07/1/2022 – 06/30/2025) Award Amount: $899,995
Type: Methodological Innovation Award Number: R305D220028

In research on the effects of education interventions, there is interest not only in the overall average treatment effect but also in effects for subgroups of sample members and whether those effects differ. The purpose of this grant is to investigate the value and limitations of using machine learning to detect the presence of heterogeneous subgroup impacts in randomized control trials (RCTs) in education and other policy domains. The machine learning approach does not require researchers to pre-specify subgroups, as is typically important when using standard multiple comparison procedures, and machine learning can be used when there are a large number of candidate characteristics to examine.

The research team will use machine learning to conduct a secondary analysis of data from Career Academies, Growth Mindset, and the Accelerated Studies in Associate Program at CUNY and Ohio. With the secondary analysis, the researchers will determine whether machine learning replicates the findings of the original analyses and whether it identifies additional and theoretically meaningful subgroup effects not identified by the original analyses. The team will also conduct a Monte Carlo simulation study to investigate the circumstances under which the machine learning approach would be potentially more useful in multi-site RCTs than conventional subgroup analyses. The products of the grant include user-friendly R software for conducting machine learning subgroup analysis for RCTs, an instructional webinar for using the software, a conference presentation, and two research papers.