Evaluating the Impact of the Investing in Innovation Fund (i3) UPSTART Project on Rural Preschoolers' Early Literacy Skills
Hobbs, L. Jon; Overby, Melanie (2017). Retrieved from: https://eric.ed.gov/?id=ED608232
-
examining491Students, gradePK
Department-funded evaluation
Review Details
Reviewed: September 2022
- Department-funded evaluation (findings for Utah Preparing Students Today for a Rewarding Tomorrow (UPSTART) Reading Program)
- Randomized Controlled Trial
- Meets WWC standards without reservations because it is a randomized controlled trial with low attrition.
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Findings
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Brigance Inventory of Early Development III (IEDIII): Academic Skills/Cognitive Development: Literacy: Alphabet Knowledge |
Utah Preparing Students Today for a Rewarding Tomorrow (UPSTART) Reading Program vs. Utah Preparing Students Today for a Rewarding Tomorrow (UPSTART) - Math Program |
0 Days |
Full sample;
|
20.53 |
16.19 |
Yes |
|
|
|
Brigance Inventory of Early Development III (EID-III) - Names Uppercase Letters Subscale |
Utah Preparing Students Today for a Rewarding Tomorrow (UPSTART) Reading Program vs. Utah Preparing Students Today for a Rewarding Tomorrow (UPSTART) - Math Program |
0 Days |
Full sample;
|
20.53 |
16.19 |
Yes |
|
|
|
Brigance Inventory of Early Development III (IEDIII): Phonemic Manipulation sub-scale |
Utah Preparing Students Today for a Rewarding Tomorrow (UPSTART) Reading Program vs. Utah Preparing Students Today for a Rewarding Tomorrow (UPSTART) - Math Program |
0 Days |
Full sample;
|
4.07 |
3.56 |
Yes |
|
|
|
Preschool Early Learning Indicators (PELI) - Phonological Awareness - Initial Word Sounds |
Utah Preparing Students Today for a Rewarding Tomorrow (UPSTART) Reading Program vs. Utah Preparing Students Today for a Rewarding Tomorrow (UPSTART) - Math Program |
0 Days |
Full sample;
|
10.53 |
9.19 |
Yes |
|
|
|
Brigance Inventory of Early Development III (EID-III) - Phonological Awareness |
Utah Preparing Students Today for a Rewarding Tomorrow (UPSTART) Reading Program vs. Utah Preparing Students Today for a Rewarding Tomorrow (UPSTART) - Math Program |
0 Days |
Full sample;
|
6.57 |
6.06 |
No |
-- |
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Preschool Early Literacy Indicators (PELI): Listening Comprehension |
Utah Preparing Students Today for a Rewarding Tomorrow (UPSTART) Reading Program vs. Utah Preparing Students Today for a Rewarding Tomorrow (UPSTART) - Math Program |
0 Days |
Full sample;
|
18.36 |
18.14 |
No |
-- | |
|
Preschool Early Literacy Indicators (PELI) : Vocabulary and Oral Language |
Utah Preparing Students Today for a Rewarding Tomorrow (UPSTART) Reading Program vs. Utah Preparing Students Today for a Rewarding Tomorrow (UPSTART) - Math Program |
0 Days |
Full sample;
|
22.48 |
22.68 |
No |
-- |
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Sample Characteristics
Characteristics of study sample as reported by study author.
-
Other or unknown: 100% -
Rural
-
- B
- A
- C
- D
- E
- F
- G
- I
- H
- J
- K
- L
- P
- M
- N
- O
- Q
- R
- S
- V
- U
- T
- W
- X
- Z
- Y
- a
- h
- i
- b
- d
- e
- f
- c
- g
- j
- k
- l
- m
- n
- o
- p
- q
- r
- s
- t
- u
- v
- x
- w
- y
Utah
-
Race Other or unknown 100% -
Ethnicity Other or unknown 100% -
Eligible for Free and Reduced Price Lunch Other or unknown 100%
Study Details
Setting
The UPSTART program was implemented in public schools in rural school districts in Utah. The intervention serves pre-kindergarten students.
Study sample
Information on sample demographics are not available; to note, program families were excluded from the evaluation if their child did not speak English or had a diagnosed learning disability.
Intervention Group
The intervention group had access to UPSTART Reading software at home. Recommended usage of the software was 15 minutes a day, five days a week. More generally, UPSTART)is a school readiness program that uses a home-based educational technology model with supportive resources that is well-suited to preschoolers and families residing in rural communities in Utah. The objective of UPSTART is to provide service to students in traditionally underserved rural areas, foster early cognitive growth, close reading achievement gaps, and increase school readiness at entry to kindergarten. While the UPSTART adaptive educational software is the core component of the program, the provision of technology in the form of computer hardware and/or high-speed internet access ensures equitable access for all. Additionally, UPSTART families are provided with technical, motivational, and curricular support to encourage program software usage and facilitate the academic supervision of children by parents
Comparison Group
The comparison group had access to UPSTART Math software at home. Recommended usage of the software is 15 minutes a day, five days a week.
Support for implementation
No further details provided.
An indicator of the effect of the intervention, the improvement index can be interpreted as the expected change in percentile rank for an average comparison group student if that student had received the intervention.
For more, please see the WWC Glossary entry for improvement index.
An outcome is the knowledge, skills, and attitudes that are attained as a result of an activity. An outcome measures is an instrument, device, or method that provides data on the outcome.
A finding that is included in the effectiveness rating. Excluded findings may include subgroups and subscales.
The sample on which the analysis was conducted.
The group to which the intervention group is compared, which may include a different intervention, business as usual, or no services.
The timing of the post-intervention outcome measure.
The number of students included in the analysis.
The mean score of students in the intervention group.
The mean score of students in the comparison group.
The WWC considers a finding to be statistically significant if the likelihood that the finding is due to chance alone, rather than a real difference, is less than five percent.
The WWC reviews studies for WWC products, Department of Education grant competitions, and IES performance measures.
The name and version of the document used to guide the review of the study.
The version of the WWC design standards used to guide the review of the study.
The result of the WWC assessment of the study. The rating is based on the strength of evidence of the effectiveness of the intervention. Studies are given a rating of Meets WWC Design Standards without Reservations, Meets WWC Design Standards with Reservations, or >Does Not Meet WWC Design Standards.
A related publication that was reviewed alongside the main study of interest.
Study findings for this report.
Based on the direction, magnitude, and statistical significance of the findings within a domain, the WWC characterizes the findings from a study as one of the following: statistically significant positive effects, substantively important positive effects, indeterminate effects, substantively important negative effects, and statistically significant negative effects. For more, please see the WWC Handbook.
The WWC may review studies for multiple purposes, including different reports and re-reviews using updated standards. Each WWC review of this study is listed in the dropdown. Details on any review may be accessed by making a selection from the drop down list.
Tier 1 Strong indicates strong evidence of effectiveness,
Tier 2 Moderate indicates moderate evidence of effectiveness, and
Tier 3 Promising indicates promising evidence of effectiveness,
as defined in the
non-regulatory guidance for ESSA
and the regulations for ED discretionary grants (EDGAR Part 77).