
Scaling up an Early Reading Program: Relationships among Teacher Support, Fidelity of Implementation, and Student Performance across Different Sites and Years
Stein, Marc L.; Berends, Mark; Fuchs, Douglas; McMaster, Kristen; Saenz, Laura; Yen, Loulee; Fuchs, Lynn S.; Compton, Donald L. (2008). Educational Evaluation and Policy Analysis, v30 n4 p368-388. Retrieved from: https://eric.ed.gov/?id=EJ951754
-
examining1,636Students, gradeK
Peer-Assisted Learning Strategies Intervention Report - Beginning Reading
Review Details
Reviewed: May 2012
- Randomized Controlled Trial
- Meets WWC standards without reservations
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Please see the WWC summary of evidence for Peer-Assisted Learning Strategies.
Findings
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
---|---|---|---|---|---|---|---|---|
Rapid letter sound |
Peer-Assisted Learning Strategies vs. Business as usual |
Posttest |
Kindergarten;
|
51.57 |
32.90 |
Yes |
|
|
Rapid letter sound |
Peer-Assisted Learning Strategies vs. Business as usual |
Posttest |
Kindergarten;
|
49.01 |
32.90 |
Yes |
|
|
Rapid letter sound |
Peer-Assisted Learning Strategies vs. Business as usual |
Posttest |
Kindergarten;
|
44.00 |
32.90 |
Yes |
|
|
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Sample Characteristics
Characteristics of study sample as reported by study author.
-
24% English language learners -
Female: 50%
Male: 50% -
Urban
-
- B
- A
- C
- D
- E
- F
- G
- I
- H
- J
- K
- L
- P
- M
- N
- O
- Q
- R
- S
- V
- U
- T
- W
- X
- Z
- Y
- a
- h
- i
- b
- d
- e
- f
- c
- g
- j
- k
- l
- m
- n
- o
- p
- q
- r
- s
- t
- u
- v
- x
- w
- y
Minnesota, Tennessee, Texas
-
Race Asian 5% Black 25% Other or unknown 3% White 26% -
Ethnicity Hispanic 40% Not Hispanic or Latino 60%
Study Details
Setting
The study took place in 71 schools in three sites over two years: 14 schools in Nashville, Tennessee; 36 schools in Minnesota; and 21 schools in south Texas. The final analytic sample included 67 schools.
Study sample
Project staff first recruited schools to obtain balanced samples on site-specific factors: Title I status in Nashville, Title I status and whether the school offered half-day or full-day kindergarten in Minnesota, and the proportion of limited English proficiency students in the schools in south Texas. Teachers were recruited within the selected schools, and 224 teachers participated over the two study years (55 teachers participated in both years, for a total of 279 teacheryears). Within each participating school, teachers were randomly assigned to one of the four conditions: control, workshop, workshop plus booster, or workshop plus booster and helper. The study does not report the number of teachers in each condition.Researchers obtained parental consent for more than 90% of the students in the classrooms of study teachers. These students were pretested, and 12 students were selected from each class: four children with the lowest reading scores, four children with the highest scores, and four children with scores in the middle of the score distribution. The consented study sample included 3,171 kindergarten students, with 668 in the control condition, 968 in the workshop condition, 931 in the booster condition, and 604 in the helper condition. The final hierarchical linear modeling (HLM) analysis sample included only 2,959 students and 259 teachers. The WWC could not calculate attrition by condition based on the information provided in the study. However, based on reasonable assumptions about how to attribute overall attrition to groups, the study is assumed to have low differential attrition. Twenty-four percent of the students in the study were English language learners, 62% were eligible for free or reduced-price lunch, 50% were female, and 5% had Individualized Education Plans. Of the students in the sample, 40% were Hispanic, 26% were non-Hispanic White, 25% were African American, 5% were Asian, and 3% were of other ethnicities.
Intervention Group
The study included three treatment conditions: (1) a day-long training workshop (K-PALS), (2) the workshop plus two follow-up booster sessions (K-PALS + Booster), and (3) the workshop and booster sessions plus weekly technical assistance provided by a graduate student (K-PALS + Booster + Helper). Although the treatment conditions vary by the amount of training and support received by teachers, the K-PALS intervention was the same in all three treatment conditions. Students were paired by their teachers and then worked through structured lessons during 35-minute sessions implemented four times per week in this study. Stronger readers were paired with weaker readers, and pairings were maintained for four to six weeks before being reorganized. Within each pair, students took turns acting as the reader and the coach. The classroom teacher monitored the pairs and provided feedback as necessary. Program materials, including a teacher manual and all student worksheets, were provided by K-PALS.
Comparison Group
The comparison was a business-as-usual counterfactual. Comparison teachers did not implement the intervention and did not receive any additional training.
Outcome descriptions
The primary outcome is Rapid Letter Sounds, an alphabetics measure developed by Levy and Lysunchuk (1997). All study students were tested approximately three weeks before the intervention began and again 20 weeks later. For a more detailed description of the outcome measure, see Appendix B.
Support for implementation
All teachers in the three treatment groups attended a day-long training workshop before the intervention began. For the K-PALS + Booster treatment group, two follow-up booster sessions were also provided to allow teachers to review program procedures and to identify and solve implementation issues. Teachers in the K-PALS + Booster + Helper treatment group attended the training workshop and booster sessions and also had weekly technical assistance provided by a trained graduate assistant. Average implementation fidelity, measured at two points during implementation by the project coordinator, was 86%.
Peer-Assisted Learning Strategies Intervention Report - Adolescent Literacy
Review Details
Reviewed: January 2012
- The study is ineligible for review because it does not use a sample within the age or grade range specified in the protocol.
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Please see the WWC summary of evidence for Peer-Assisted Learning Strategies.
Findings
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Sample Characteristics
Study sample characteristics were not reported.An indicator of the effect of the intervention, the improvement index can be interpreted as the expected change in percentile rank for an average comparison group student if that student had received the intervention.
For more, please see the WWC Glossary entry for improvement index.
An outcome is the knowledge, skills, and attitudes that are attained as a result of an activity. An outcome measures is an instrument, device, or method that provides data on the outcome.
A finding that is included in the effectiveness rating. Excluded findings may include subgroups and subscales.
The sample on which the analysis was conducted.
The group to which the intervention group is compared, which may include a different intervention, business as usual, or no services.
The timing of the post-intervention outcome measure.
The number of students included in the analysis.
The mean score of students in the intervention group.
The mean score of students in the comparison group.
The WWC considers a finding to be statistically significant if the likelihood that the finding is due to chance alone, rather than a real difference, is less than five percent.
The WWC reviews studies for WWC products, Department of Education grant competitions, and IES performance measures.
The name and version of the document used to guide the review of the study.
The version of the WWC design standards used to guide the review of the study.
The result of the WWC assessment of the study. The rating is based on the strength of evidence of the effectiveness of the intervention. Studies are given a rating of Meets WWC Design Standards without Reservations, Meets WWC Design Standards with Reservations, or >Does Not Meet WWC Design Standards.
A related publication that was reviewed alongside the main study of interest.
Study findings for this report.
Based on the direction, magnitude, and statistical significance of the findings within a domain, the WWC characterizes the findings from a study as one of the following: statistically significant positive effects, substantively important positive effects, indeterminate effects, substantively important negative effects, and statistically significant negative effects. For more, please see the WWC Handbook.
The WWC may review studies for multiple purposes, including different reports and re-reviews using updated standards. Each WWC review of this study is listed in the dropdown. Details on any review may be accessed by making a selection from the drop down list.
Tier 1 Strong indicates strong evidence of effectiveness,
Tier 2 Moderate indicates moderate evidence of effectiveness, and
Tier 3 Promising indicates promising evidence of effectiveness,
as defined in the
non-regulatory guidance for ESSA
and the regulations for ED discretionary grants (EDGAR Part 77).