WWC review of this study

The impact of Achieve3000 on elementary literacy outcomes: Randomized control trial evidence, 2013-14 to 2014-15 (DRA Report No. 16.02).

Hill, D. V., & Lenard, M. A. (2016). Cary, NC: Wake County Public School System, Data and Accountability Department. Retrieved from: https://eric.ed.gov/?id=ED581879

  • Randomized Controlled Trial
     examining 
    22,583
     Students
    , grades
    2-5

Reviewed: February 2018

At least one statistically significant positive finding
Meets WWC standards with reservations
Comprehension outcomes—Indeterminate effects found
Outcome
measure
Comparison Period Sample Intervention
mean
Comparison
mean
Significant? Improvement
    index
ESSA
rating

North Carolina End-of-Grade Test

Achieve3000® vs. Business as usual

1 Year

2015 sample, grades 4-5;
6,307 students

991.77

985.11

No

--
More Outcomes

North Carolina End-of-Grade Test

Achieve3000® vs. Business as usual

1 Year

2014 sample, grades 4-5;
6,235 students

989.43

991.06

No

--
Show Supplemental Findings

North Carolina End-of-Grade Test

Achieve3000® vs. Business as usual

1 Year

2015 AIG sample, grades 4-5;
788 students

1221.25

1244.40

No

--
Literacy achievement outcomes—Statistically significant positive effects found
Outcome
measure
Comparison Period Sample Intervention
mean
Comparison
mean
Significant? Improvement
    index
ESSA
rating

LevelSet Lexile score

Achieve3000® vs. Business as usual

1 Year

2015 sample, grades 2-5;
12,851 students

577.94

520.72

Yes

 
 
8
More Outcomes

LevelSet Lexile score

Achieve3000® vs. Business as usual

1 Year

2014 sample, grades 2-5;
9,732 students

550.13

562.00

Yes

-1
 
 
Show Supplemental Findings

LevelSet Lexile score

Achieve3000® vs. Business as usual

1 Year

2014 AIG sample, grades 2-5;
889 students

985.28

999.20

No

--

LevelSet Lexile score

Achieve3000® vs. Business as usual

1 Year

2015 AIG sample, grades 4-5;
896 students

983.61

952.40

Yes

N/A

Characteristics of study sample as reported by study author.


  • 9% English language learners

  • 37% Free or reduced price lunch

  • Female: 49%
    Male: 51%
  • Race
    Black
    26%
    White
    51%
  • Ethnicity
    Hispanic
    19%
    Not Hispanic
    82%

  • Rural, Suburban, Urban
    • B
    • A
    • C
    • D
    • E
    • F
    • G
    • I
    • H
    • J
    • K
    • L
    • P
    • M
    • N
    • O
    • Q
    • R
    • S
    • V
    • U
    • T
    • W
    • X
    • Z
    • Y
    • a
    • h
    • i
    • b
    • d
    • e
    • f
    • c
    • g
    • j
    • k
    • l
    • m
    • n
    • o
    • p
    • q
    • r
    • s
    • t
    • u
    • x
    • w
    • y

    North Carolina

Setting

The study was conducted in the Wake County Public School System (WCPSS) in Raleigh, North Carolina. As a countrywide district, WCPSS has schools representing suburban, urban, and rural areas.

Study sample

The authors used a cluster randomized controlled trial design to study the effects of KidBiz3000® on reading achievement for students in grades 2–5. The study took place over two school years (from 2013–14 to 2014–15) in 32 elementary schools. In summer of 2013, the authors matched pairs of schools on the basis of their average 2013 end-of-grade (EOG) reading composite scores, and then from within each matched pair, randomly assigned one school to the intervention group and one school to the comparison group. In both study years, the same 16 KidBiz3000® schools and 16 comparison schools participated in the study. The WWC considers random assignment jeopardized because the analytic sample included students who enrolled in study schools after random assignment. For the general literacy domain, the 2-year combined analysis sample included 22,583 students in grades 2–5 in 24 schools: 11,802 students were in the Achieve3000® group, and 10,781 students were in the comparison group. For the comprehension domain, the 2-year combined analysis sample included 12,542 students in grades 4–5 in 32 schools: 6,585 students were in the Achieve3000® group, and 5,957 students were in the comparison group. Because some of the same students were analyzed in both years of the study, these reported sample sizes count some individual students more than once. No demographic data were available on the analytic study sample; however, in the 32 participating schools, the student population was 51% White, 26% African-American, and 19% Hispanic. Moreover, 12% of students had disabilities, 9% of students in study schools were English learners, and 7% of students were academically and intellectually gifted (AIG). Approximately, one-third of the district's students were certified for free or reduced-price lunch.

Intervention Group

KidBiz3000® was implemented in 30-minute sessions, two times per week over a school year. On initial use, students took a 30-minute test to measure their baseline reading achievement. For each lesson, students followed Achieve3000®’s five-step literacy routine. Across both years of the study, about 8% of intervention students completed at least 80 lessons (i.e., the developer’s recommended dosage), about 21% used 40-79 lessons, about 50% used 1-39 lessons, and 22% of students completed no activities. The Achieve3000® intervention was used to supplement a standard core reading curriculum; however, the authors did not identify which core curriculum was used.

Comparison Group

The comparison condition was business-as-usual reading instruction. Classrooms in comparison schools did not receive a supplemental curriculum.

Support for implementation

The study included professional development to train teachers, consisting of two 2.5-hour large group training sessions, and one 1-hour small group session. Teachers were able to obtain follow-up help if needed.

In the case of multiple manuscripts that report on one study, the WWC selects one manuscript as the primary citation and lists other manuscripts that describe the study as additional sources.

  • Hill, D. V., Lenard, M. A., & Page, L. C. (2016, March). The impact of Achieve3000 on elementary literacy outcomes: Evidence from a two-year randomized control trial. Paper presented at the Society for Research on Educational Effectiveness (SREE) Spring Conference, Washington, DC.

Reviewed: February 2018

No statistically significant positive
findings
Meets WWC standards with reservations
Reading fluency outcomes—Indeterminate effects found
Outcome
measure
Comparison Period Sample Intervention
mean
Comparison
mean
Significant? Improvement
    index
ESSA
rating

DIBELS Oral Reading Fluency

Achieve3000® vs. Business as usual

1 Year

2014 sample, grades 2-3;
7,197 students

674.88

656.43

No

--
More Outcomes

DIBELS Oral Reading Fluency

Achieve3000® vs. Business as usual

1 Year

2015 sample, grades 2-3;
7,296 students

663.89

650.85

No

--
Show Supplemental Findings

DIBELS Oral Reading Fluency

Achieve3000® vs. Business as usual

1 Year

2014 AIG sample, grades 2-3;
331 students

1049.40

1066.30

No

--

DIBELS Oral Reading Fluency

Achieve3000® vs. Business as usual

1 Year

2015 AIG sample, grades 2-3;
173 students

1063.60

1107.20

No

--

Characteristics of study sample as reported by study author.


  • 9% English language learners

  • 33% Free or reduced price lunch

  • Female: 49%
    Male: 51%
  • Race
    Black
    26%
    White
    51%
  • Ethnicity
    Hispanic
    19%
    Not Hispanic
    82%

  • Rural, Suburban, Urban
    • B
    • A
    • C
    • D
    • E
    • F
    • G
    • I
    • H
    • J
    • K
    • L
    • P
    • M
    • N
    • O
    • Q
    • R
    • S
    • V
    • U
    • T
    • W
    • X
    • Z
    • Y
    • a
    • h
    • i
    • b
    • d
    • e
    • f
    • c
    • g
    • j
    • k
    • l
    • m
    • n
    • o
    • p
    • q
    • r
    • s
    • t
    • u
    • x
    • w
    • y

    North Carolina

Setting

The study was conducted in the Wake County Public School System (WCPSS) in Raleigh, North Carolina. As a countywide district, WCPSS has schools representing suburban, urban, and rural areas.

Study sample

The authors used a cluster randomized controlled trial design to study the effects of KidBiz3000® on the reading achievement of students in grades 2-5 (only analyses for grades 2-3 were eligible for review under the Beginning Reading Protocol). The study took place over two school years (from 2013–14 to 2014–15) in 32 elementary schools. In summer of 2013, the authors matched pairs of schools based on their average 2013 End-of-Grade (EOG) reading composite scores, and then from within each matched pair, randomly assigned one school to the intervention group and one school to the comparison group. In both study years, the same 16 KidBiz3000® schools and 16 comparison schools participated in the study. The WWC considers random assignment to be jeopardized because the analytic sample included students who enrolled in study schools after random assignment. The 2-year combined analysis sample included 14,493 students: 7,540 students were in the Achieve3000® group, and 6,953 students were in the comparison group. The reported sample sizes count some students more than once because some second grade students in 2013–14 may also appear as third-grade students in 2014–15. No demographic data were available on the study sample in grades 2–3; however, in the 32 participating schools, the student population was 51% White, 26% African-American and 19% Hispanic. Moreover, 12% of students had disabilities, 9% of students in study schools were English learners, and 7% of students were academically and intellectually gifted (AIG). Approximately, one-third of the district's students were certified for free or reduced-price lunch.

Intervention Group

KidBiz3000® was implemented in 30-minute sessions, two times per week over a school year. On initial use, students took a 30-minute test to measure their baseline reading achievement. For each lesson, students followed Achieve3000®’s five-step literacy routine. Across both years of the study, about 8% of intervention students completed at least 80 lessons (i.e., the developer’s recommended dosage), about 21% used 40-79 lessons, about 50% used 1-39 lessons, and 22% of students completed no activities. The Achieve3000® intervention was used to supplement a standard core reading curriculum; however, the authors did not identify which core curriculum was used.

Comparison Group

The comparison condition was business-as-usual reading instruction. Classrooms in comparison schools did not receive a supplemental curriculum.

Support for implementation

The study included professional development to train teachers, consisting of two 2.5-hour large-group training sessions and one 1-hour small-group session. Teachers were able to obtain follow-up help if needed.

In the case of multiple manuscripts that report on one study, the WWC selects one manuscript as the primary citation and lists other manuscripts that describe the study as additional sources.

  • Hill, D. V., Lenard, M. A., & Page, L. C. (2016, March). The impact of Achieve3000 on elementary literacy outcomes: Evidence from a two-year randomized control trial. Paper presented at the Society for Research on Educational Effectiveness (SREE) Spring Conference, Washington, DC.

 

Your export should download shortly as a zip archive.

This download will include data files for study and findings review data and a data dictionary.

Connect With the WWC

loading
back to top