Resources
Newsflash
Contact Us
Facebook
Twitter
YouTube
Web-Based Support for Data-Based Decision Making: Effect of Intervention Implementation on Infant-Toddler Communication
Buzhardt, Jay; Greenwood, Charles R.; Walker, Dale; Jia, Fan; Schnitz, Alana G.; Higgins, Susan; Montagna, Debra; Muehe, Christine (2018). Journal of Early Intervention v40 n3 p246-267. Retrieved from: https://eric.ed.gov/?id=ED598358
-
examining146Students, gradePK
Department-funded evaluation
Review Details
Reviewed: February 2022
- Department-funded evaluation (findings for Making Online Decisions (MOD))
- Randomized Controlled Trial
- Meets WWC standards with reservations because it is a cluster randomized controlled trial with high individual-level non-response, but provides evidence of effects on individuals by satisfying the baseline equivalence requirement for the individuals in the analytic intervention and comparison groups.
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Findings
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Early Communication Indicator (ECI) |
Making Online Decisions (MOD) vs. Other intervention |
0 Days |
Full sample;
|
14.28 |
11.53 |
No |
-- |
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Sample Characteristics
Characteristics of study sample as reported by study author.
-
16% English language learners -
Female: 40%
Male: 60% -
- B
- A
- C
- D
- E
- F
- G
- I
- H
- J
- K
- L
- P
- M
- N
- O
- Q
- R
- S
- V
- U
- T
- W
- X
- Z
- Y
- a
- h
- i
- b
- d
- e
- f
- c
- g
- j
- k
- l
- m
- n
- o
- p
- q
- r
- s
- t
- u
- v
- x
- w
- y
Midwest
Study Details
Setting
This intervention is delivered through home visiting. The intervention was implemented in the homes of families and children in 13 Early Head Start programs in four midwestern states.
Study sample
The families in the study were at, or below, the federal poverty level. Only 1 of the 146 families had a monthly income of more than $3,000. Thirty-two percent of the families were headed by a single parent and 68 percent of caregivers were married. Thirty-seven percent of the families' primary caregiver had some college (up to a BA degree). Eighty-eight percent of the families spoke English as their primary language while 14 percent spoke Spanish. About 60 percent of the children were male and about 40 percent were female. Their average age was 21.3 months. All children scored at least one standard deviation below their age-based benchmark on the ECI. All but one of the home visitors were women, and all but six were White. Sixty percent of the home visitors had at least some college experience, up to a BA degree, and 11 percent had some experience in graduate school, up to an MA degree.
Intervention Group
The Making Online Decisions (MOD) intervention is an online adaptive system that guides data-based intervention decision-making of home visitors who work with infants and toddlers at risk for language delay. The MOD system is one of the few adaptive interventions designed specifically to help home visitors individualize intervention services for infants and toddlers. The home visitors in the intervention group implemented the MOD to guide their use of the ECI assessment with the children and their use of the Promoting Communication for Accelerating Language in Kids (PC TALK) strategies with the parents. The home visitors followed five steps that addressed five questions: 1) Is there a problem? 2) What is causing the problem? 3) What intervention should be used? 4) Is the intervention being done? 5) Is the intervention working? Home visitors implemented these steps to varying degrees, which the researchers organized into three implementation stages. Twenty-nine home visitors implemented stage 1 (MOD steps 1 to 3), 15 implemented stage 2 (MOD steps 1 to 4), and 12 implemented stage 3 (MOD steps 1 to 5). The authors call this "natural variation."
Comparison Group
The comparison group was a business-as-usual group that implemented PC TALK, which the intervention group also implemented. PC TALK is a manualized set of strategies and tools designed to extend caregivers’ use of language-promoting strategies across daily routines. The strategies included (a) arranging the environment, (b) following the child’s lead, (c) commenting and labeling, (d) imitating and expanding, (e) open-ended questioning, (f) time delay, (g) positive attention, and (h) providing choices.
Support for implementation
The home visitors in the intervention group received a 1-hour training on how to use the MOD. The training included completing MOD steps, entering checklists, and printing MOD-recommended PC TALK strategies.
An indicator of the effect of the intervention, the improvement index can be interpreted as the expected change in percentile rank for an average comparison group student if that student had received the intervention.
For more, please see the WWC Glossary entry for improvement index.
An outcome is the knowledge, skills, and attitudes that are attained as a result of an activity. An outcome measures is an instrument, device, or method that provides data on the outcome.
A finding that is included in the effectiveness rating. Excluded findings may include subgroups and subscales.
The sample on which the analysis was conducted.
The group to which the intervention group is compared, which may include a different intervention, business as usual, or no services.
The timing of the post-intervention outcome measure.
The number of students included in the analysis.
The mean score of students in the intervention group.
The mean score of students in the comparison group.
The WWC considers a finding to be statistically significant if the likelihood that the finding is due to chance alone, rather than a real difference, is less than five percent.
The WWC reviews studies for WWC products, Department of Education grant competitions, and IES performance measures.
The name and version of the document used to guide the review of the study.
The version of the WWC design standards used to guide the review of the study.
The result of the WWC assessment of the study. The rating is based on the strength of evidence of the effectiveness of the intervention. Studies are given a rating of Meets WWC Design Standards without Reservations, Meets WWC Design Standards with Reservations, or >Does Not Meet WWC Design Standards.
A related publication that was reviewed alongside the main study of interest.
Study findings for this report.
Based on the direction, magnitude, and statistical significance of the findings within a domain, the WWC characterizes the findings from a study as one of the following: statistically significant positive effects, substantively important positive effects, indeterminate effects, substantively important negative effects, and statistically significant negative effects. For more, please see the WWC Handbook.
The WWC may review studies for multiple purposes, including different reports and re-reviews using updated standards. Each WWC review of this study is listed in the dropdown. Details on any review may be accessed by making a selection from the drop down list.
Tier 1 Strong indicates strong evidence of effectiveness,
Tier 2 Moderate indicates moderate evidence of effectiveness, and
Tier 3 Promising indicates promising evidence of effectiveness,
as defined in the
non-regulatory guidance for ESSA
and the regulations for ED discretionary grants (EDGAR Part 77).