WWC review of this study

Scaling up an early reading program: Relationships among teacher support, fidelity of implementation, and student performance across different sites and years.

Stein, M. L., Berends, M., Fuchs, D., McMaster, K., Sáenz, L., Yen, L., & Compton, D. L. (2008). Educational Evaluation and Policy Analysis, 30(4), 368–388. Retrieved from: https://eric.ed.gov/?id=EJ951754

  • Randomized Controlled Trial
     examining 
    1,636
     Students
    , grade
    K
At least one statistically significant positive finding
Meets WWC standards without reservations

Reviewed: May 2012

Alphabetics outcomes—Statistically significant positive effects found
Outcome
measure
Comparison Period Sample Intervention
mean
Comparison
mean
Significant? Improvement
index

Rapid letter sound

Peer-Assisted Learning Strategies vs. Business as usual

Posttest

Kindergarten;
1,599 students

51.57

32.9

Yes

 
 
36
More Outcomes

Rapid letter sound

Peer-Assisted Learning Strategies vs. Business as usual

Posttest

Kindergarten;
1,272 students

49.01

32.9

Yes

 
 
31

Rapid letter sound

Peer-Assisted Learning Strategies vs. Business as usual

Posttest

Kindergarten;
1,636 students

44

32.9

Yes

 
 
23

Characteristics of study sample as reported by study author.


  • 24% English language learners

  • 62% Free or reduced price lunch

  • Female: 50%
    Male: 50%
  • Race
    Asian
    5%
    Black
    25%
    Not specified
    3%
    White
    26%
  • Ethnicity
    Hispanic
    40%
    Not Hispanic
    60%

  • Urban
    • B
    • A
    • C
    • D
    • E
    • F
    • G
    • I
    • H
    • J
    • K
    • L
    • P
    • M
    • N
    • O
    • Q
    • R
    • S
    • V
    • U
    • T
    • W
    • X
    • Z
    • Y
    • a
    • h
    • i
    • b
    • d
    • e
    • f
    • c
    • g
    • j
    • k
    • l
    • m
    • n
    • o
    • p
    • q
    • r
    • s
    • t
    • u
    • x
    • w
    • y

    Minnesota, Tennessee, Texas

Setting

The study took place in 71 schools in three sites over two years: 14 schools in Nashville, Tennessee; 36 schools in Minnesota; and 21 schools in south Texas. The final analytic sample included 67 schools.

Study sample

Project staff first recruited schools to obtain balanced samples on site-specific factors: Title I status in Nashville, Title I status and whether the school offered half-day or full-day kindergarten in Minnesota, and the proportion of limited English proficiency students in the schools in south Texas. Teachers were recruited within the selected schools, and 224 teachers participated over the two study years (55 teachers participated in both years, for a total of 279 teacheryears). Within each participating school, teachers were randomly assigned to one of the four conditions: control, workshop, workshop plus booster, or workshop plus booster and helper. The study does not report the number of teachers in each condition.Researchers obtained parental consent for more than 90% of the students in the classrooms of study teachers. These students were pretested, and 12 students were selected from each class: four children with the lowest reading scores, four children with the highest scores, and four children with scores in the middle of the score distribution. The consented study sample included 3,171 kindergarten students, with 668 in the control condition, 968 in the workshop condition, 931 in the booster condition, and 604 in the helper condition. The final hierarchical linear modeling (HLM) analysis sample included only 2,959 students and 259 teachers. The WWC could not calculate attrition by condition based on the information provided in the study. However, based on reasonable assumptions about how to attribute overall attrition to groups, the study is assumed to have low differential attrition. Twenty-four percent of the students in the study were English language learners, 62% were eligible for free or reduced-price lunch, 50% were female, and 5% had Individualized Education Plans. Of the students in the sample, 40% were Hispanic, 26% were non-Hispanic White, 25% were African American, 5% were Asian, and 3% were of other ethnicities.

Intervention Group

The study included three treatment conditions: (1) a day-long training workshop (K-PALS), (2) the workshop plus two follow-up booster sessions (K-PALS + Booster), and (3) the workshop and booster sessions plus weekly technical assistance provided by a graduate student (K-PALS + Booster + Helper). Although the treatment conditions vary by the amount of training and support received by teachers, the K-PALS intervention was the same in all three treatment conditions. Students were paired by their teachers and then worked through structured lessons during 35-minute sessions implemented four times per week in this study. Stronger readers were paired with weaker readers, and pairings were maintained for four to six weeks before being reorganized. Within each pair, students took turns acting as the reader and the coach. The classroom teacher monitored the pairs and provided feedback as necessary. Program materials, including a teacher manual and all student worksheets, were provided by K-PALS.

Comparison Group

The comparison was a business-as-usual counterfactual. Comparison teachers did not implement the intervention and did not receive any additional training.

Outcome descriptions

The primary outcome is Rapid Letter Sounds, an alphabetics measure developed by Levy and Lysunchuk (1997). All study students were tested approximately three weeks before the intervention began and again 20 weeks later. For a more detailed description of the outcome measure, see Appendix B.

Support for implementation

All teachers in the three treatment groups attended a day-long training workshop before the intervention began. For the K-PALS + Booster treatment group, two follow-up booster sessions were also provided to allow teachers to review program procedures and to identify and solve implementation issues. Teachers in the K-PALS + Booster + Helper treatment group attended the training workshop and booster sessions and also had weekly technical assistance provided by a trained graduate assistant. Average implementation fidelity, measured at two points during implementation by the project coordinator, was 86%.

Reviewed: January 2012

Study sample characteristics were not reported.
 

Your export should download shortly as a zip archive.

This download will include data files for study and findings review data and a data dictionary.

Connect With the WWC

newsflash icon contact icon facebook icon twitter icon
loading
back to top