Getting SMART About Adaptive Interventions in Education

Co-Principal Investigator: Nahum-Shani, Inbal

1. Establish a website with training materials and related resources;
2. Develop the content of the training institute to cover a variety of topics, including an introduction to AIs, experimental designs for evaluating AIs, and techniques for analyzing data from SMARTs;
3. Recruit and select institute trainees;
4. Engage trainees in accessing foundational materials on the website;
5. Hold two training institutes during the summers of 2019 and 2020;
6. Providing online training materials and related resources (e.g., video-taped institute sessions);
7. Meet with trainees and their team in one follow-up videoconference; and
8. Evaluate the project website, training materials, training institute, and follow-up mentoring annually.

Through this approach, the training team intends to improve the knowledge and skills needed for the development of effective AIs in education, including knowledge and skills pertaining to the design and conduct of SMARTs. The specific learning objectives include an understanding of (a) the nature and utility of AIs in educational settings, (b) various experimental designs for evaluating and optimizing AIs, (c) how to apply data analytic strategies for common primary and secondary aims in a SMART, (d) the utility and application of SMARTs in pilot studies, and (e) various ways in which SMARTs optimize AIs in education and other fields.

The training team will evaluate specific aspects of the training program (i.e., project website, training institute, mentoring) annually to inform continuous improvement. To evaluate the website, they will track the total number and number of unique visits. To evaluate the effectiveness of the training institute, trainees will complete a questionnaire to rate various aspects of the training and make suggestions for improvement. They will complete a similar questionnaire following the videoconference designed to promote ongoing learning.

Trainee outcomes will also be evaluated to determine the extent to which they apply the knowledge and skills learned as part of the training program. Trainees will complete a survey once per year, for up to 2 years following training completion to determine the extent to which the training program facilitated scholarly activities (e.g., research studies, grant submissions, peer-reviewed publications, presentations, teaching activities) related to AIs. The training team will also track the total number of IES grant submissions that focus on AIs and, whenever possible, this data will be linked to trainees as an objective measure of grant submissions. The training team will also calculate the total training cost per person and the cost-effectiveness.