Project Activities
Through ongoing collaboration with local infant-toddler programs, including Part C and Early Head Start, the MMS was developed and tested across four phases. In phases 1–3, the team developed, tested, and refined the system based on feedback from service providers and their families through usability and feasibility testing. In phase 4, they pilot tested custom MOD units deployed to infant-toddler agencies using a small-scale randomized controlled trial to evaluate the effects of the system on service providers' data-based decision-making practices and infant-toddler growth in cognitive problem-solving skills using the Early Problem Solving Indicator (EPSI).
Structured Abstract
Setting
The research took place in center- and home-based infant-toddler programs that serve children with identified disabilities (Part C programs) or are mandated to serve a proportion of children with disabilities (Early Head Start) in Kansas.
Sample
The target population included center-based staff in Early Head Start or Part C programs and the infants and toddlers in their classrooms with or at risk for a disability. For usability testing, there were 9 administrators, 16 service providers, and 6 parents. For feasibility testing, there were 12 administrators, 18 service providers, and 18 child-parent dyads. For pilot testing, there were 22 classrooms participating with 3 eligible children per classroom, leading to a total of 44 service providers and 66 child-parent dyads.
The MMS is a web-based system to develop custom MODs informed by child outcome data from the Infant-Toddler IGDIs (Early Problem Solving Indicator, Early Movement Indicator, Early Communication Indicator, or Early Social Indicator) to make recommendations for intervention/curriculum decisions for individual children. These IGDIs are 6-minute play-based assessments normed for children aged 6-42 months. The recommendations provided by MODs developed through the MMS are driven by each child's assessment scores across the sub-domains of each IGDI. For example, the EPSI sub-domains include Looking, Exploring, Functions, and Solutions. The MMS has the ability to be customized to each agency's service-delivery model and curriculum.
Research design and methods
During the first 4 years of the project, the research team used design-based research methods to develop and test iterations of the MMS using feedback and usability and feasibility data from center-based infant-toddler staff, interventionists, administrators, and parents. During the fifth year, the team conducted a randomized controlled trial pilot study, assigning classrooms within center-based programs to the treatment or control condition. The pilot study focused on one domain of development – problem solving as measured by the Early Problem Solving Indicator (EPSI) – to assess the impact of the MODs deployed through the MMS on educators' knowledge and self-efficacy in using data to make curriculum decisions, their data-based decision-making practices, and children's growth in cognitive problem solving. The pilot study experienced high provider attrition due to educators leaving their classroom/agency (not just dropping out of the study), primarily caused by the impacts of the pandemic and the increasing shortages in staffing in the field more broadly.
Control condition
Classrooms assigned to the comparison condition assessed children's problem-solving skills quarterly with the EPSI and used their existing curriculum, similar to the experimental group; however, the control group did not have data-based decision-making support from the MOD.
Key measures
During iterative development, the research team measured usability, feasibility, and fidelity of the system using researcher-developed surveys and direct observation protocols, including think-aloud procedures. Early educator progress monitoring and decision-making practices were measured using the Examining Data Informing Teaching (EDIT) measure. Knowledge and self-efficacy of data-based decision-making practices were measured with an adapted version of the Data-Driven Decision-Making Efficacy Inventory (3D-ME) survey originally designed for K-12 educators. Child growth in cognitive problem solving was measured using the EPSI. Moderators included educator and child demographics.
Data analytic strategy
Usability and feasibility data were analyzed descriptively. The research team addressed limitations on the system based on the severity, frequency, and consistency of problems encountered by users. For the randomized controlled trial, multivariate analyses and growth curve modeling were used to examine educator outcomes and children's problem-solving skills.
Key outcomes
The main findings of this project, as reported by the principal investigator, are as follows:
Educator outcomes
- Although at baseline the MOD group performed lower on some measures of self-reported data-driven decision-making practices and self-efficacy in using data to make curriculum decisions than the comparison group, at posttest, the MOD group increased their scores more than the comparison group, resulting in non-significant differences at posttest.
- On both the EDIT and 3D-ME assessments, the MOD group made greater gains in their total scores of knowledge and self-reported data driven decision-making practices (EDIT) and self-efficacy in using data to inform practice (3D-ME); however, the differences at posttest were not statistically significant with this small sample size.
- MOD implementation, as measured by database logs within the MOD web application, varied. Overall, educators rated the MOD components high on usability and helpfulness.
Child outcomes
- Children served by MOD educators had observed growth greater than children served by comparison educators on the EPSI’s Solutions key skill element, which is the most advanced key skill of the EPSI’s four key skills.
- Although the weighted total EPSI and the other key skills showed greater growth for children served by MOD educators, this growth was not significantly different.
People and institutions involved
IES program contact(s)
Project contributors
Products and publications
Project website:
Publications:
ERIC Citations: Find available citations in ERIC for this award here.
Select Publications:
Buzhardt, J., Leonard, J., Ai, J., Higgins, S., Greenwood, C., Consolver, K., ... & Carta, J. (2023). Technology to Facilitate Progress Monitoring of Infant–Toddler Growth and Development: Measuring Implementation in Community-Based Agencies. Journal of Special Education Technology, 38(2), 198-212.
Greenwood, C. R., Higgins, S., McKenna, M., Buzhardt, J., Walker, D., Ai, J., ... & Grasley-Boy, N. (2021). Remote Use of Individual Growth and Development Indicators (IGDIs) for Infants and Toddlers. Journal of Early Intervention, 10538151211057552.
Additional project information
Additional online resources and information:
Related projects
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
