WWC review of this study

An evaluation of two contrasting approaches for improving reading achievement in a large urban district.

Skindrud, K., & Gersten, R. (2006). Elementary School Journal, 106(5), 389–407. Retrieved from: https://eric.ed.gov/?id=EJ750504

  • Quasi-Experimental Design
     examining 
    531
     Students
    , grades
    2-3
No statistically significant positive
findings
Meets WWC standards with reservations

Reviewed: March 2017

Reading achievement outcomes—Indeterminate effects found
Outcome
measure
Comparison Period Sample Intervention
mean
Comparison
mean
Significant? Improvement
index

SAT-9: Language

Success for All® vs. Open Court Reading©

1 Year

Grade: 3, lowest 25%, Cohort 2;
97 students

28.8

29.6

No

--
More Outcomes

SAT-9: Reading

Success for All® vs. Open Court Reading©

2 Years

Grade: 3, Cohort 1;
434 students

38.6

43.9

No

-12
 
 
Show Supplemental Findings

SAT-9: Language

Success for All® vs. Open Court Reading©

1 Year

Grade: 2, Cohort 1;
434 students

37.2

44.3

No

-16
 
 

SAT-9: Language

Success for All® vs. Open Court Reading©

1 Year

Grade: 2, lowest 25%, Cohort 1;
114 students

22.15

29.8

Yes

-19
 
 

SAT-9: Reading

Success for All® vs. Open Court Reading©

1 Year

Grade: 2/lowest 25%/Cohort 1;
108 students

25.8

33.6

No

-24
 
 

SAT-9: Language

Success for All® vs. Open Court Reading©

2 Years

Grade: 3, lowest 25%, Cohort 1;
114 students

29.5

38.3

No

-25
 
 

SAT-9: Reading

Success for All® vs. Open Court Reading©

2 Years

Grade: 3, lowest 25%, cohort 1;
108 students

25.4

34.6

No

-25
 
 

Characteristics of study sample as reported by study author.

  • Race
    Not specified
    100%

  • Urban
    • B
    • A
    • C
    • D
    • E
    • F
    • G
    • I
    • H
    • J
    • K
    • L
    • P
    • M
    • N
    • O
    • Q
    • R
    • S
    • V
    • U
    • T
    • W
    • X
    • Z
    • Y
    • a
    • h
    • i
    • b
    • d
    • e
    • f
    • c
    • g
    • j
    • k
    • l
    • m
    • n
    • o
    • p
    • q
    • r
    • s
    • t
    • u
    • x
    • w
    • y

    California

Setting

The study was conducted in 12 schools in the Sacramento City Unified School District (SCUSD), a large urban district in northern California.

Study sample

Under California’s interpretation of Reading First, all 59 elementary schools in SCUSD were required to implement one of two models of reading instruction, SFA® or Open Court Reading©. In the fall of 1997, four schools implemented SFA®. A matched sample of Open Court Reading© schools were created by rank-ordering SCUSD schools by poverty level (measured by the percentage of students eligible for free or reduced-price meals and percentage of students on Aid to Families with Dependent Children), and selecting two comparison schools for each SFA® school—those ranked just above and just below each SFA® school. The study included two cohorts of students: students in Cohort 1 began using the reading programs in grade 2, while students in Cohort 2 started in grade 3. A total of 936 students in Cohort 1 and Cohort 2 participated in the study. The WWC based its effectiveness rating on findings from 531 students from the two analytic samples that were found to be equivalent at baseline: Cohort 1: 142 students in the SFA® group and 292 students in the comparison group—these students were followed from second to third grade; and Cohort 2: 36 students in the SFA® group and 61 students in the comparison group—these students were followed through third grade. The analytical sample for Cohort 2 includes only low-achieving students (that is, lowest 25% on a standardized test of reading achievement). Results for the full sample of Cohort 2 students are not included in this report because, based on information obtained from the authors, that sample of students was not equivalent on key characteristics at baseline.

Intervention Group

Students in the intervention group received reading instruction through SFA®. Students were put into homogeneous groups, across classrooms and grades, based on reading skills. They received 90 minutes of reading instruction daily, outside of their homerooms. SFA® also prescribes additional writing instruction outside of these groups. The SFA® training consultants monitored implementation fidelity and observed additional writing instruction in all study schools during both study years. The authors noted that teachers in SFA® schools frequently included additional spelling and grammar, along with writing instruction, outside of the 90-minute reading block. SFA® prescribes a core reading curriculum only in grades K–1; in grades 2–6, the schools can choose their own reading curricula. The authors state that the materials and guidelines for instruction (Reading Roots for grade 1 and Reading Wings for grades 2–4), as well as the professional development, tutoring, and the SFA® school facilitator and regional consultant oversight procedures, all followed those outlined by the developers of the curriculum.

Comparison Group

Students in the comparison group received reading instruction using Open Court Reading©, a systematic approach to teaching alphabetics, print knowledge, and phonemic awareness. For this study, the district used the 1996 version of the curriculum, Open Court Collections for Young Scholars. Two hours of daily whole-class reading instruction was followed by 30 minutes of small-group instruction and/or independent work. All study students received a condensed selection of instructional content to “catch-up” students to Open Court Reading© content that they had not received in prior years (since they began using the curriculum in either second or third grade).

Support for implementation

At SFA® schools, training and technical assistance were provided by SFA® consultants from a regional SFA® office. The SFA® consultants assessed implementation fidelity and rated it as a typical level of implementation when compared with national implementation averages. At Open Court Reading© schools, teachers received 4 days of basic grade-level training in Year 1, followed by 4 days of advanced grade-level training in Year 2. Each Open Court Reading© school received a reading coach (either full-time or part-time, depending on school size). Curriculum experts met monthly with reading coaches and administrators to refine instruction and supervision and to solve problems. Reading coaches collected implementation information but were prohibited from sharing the information with the study authors; the district-level reading coordinator indicated that although some schools had implementation problems at the beginning of the study, these were resolved by the second study year.

Meets WWC standards with reservations

Reviewed: June 2016

Study sample characteristics were not reported.
No statistically significant positive
findings
Meets WWC standards with reservations

Reviewed: October 2014

Reading achievement outcomes—Substantively important positive effects found
Outcome
measure
Comparison Period Sample Intervention
mean
Comparison
mean
Significant? Improvement
index

SAT-9

Open Court Reading© vs. Success for All (SFA)

Spring 1998

Grade 3;
434 students

43.9

38.6

No

 
 
12

Characteristics of study sample as reported by study author.


  • 40% English language learners

  • Urban
    • B
    • A
    • C
    • D
    • E
    • F
    • G
    • I
    • H
    • J
    • K
    • L
    • P
    • M
    • N
    • O
    • Q
    • R
    • S
    • V
    • U
    • T
    • W
    • X
    • Z
    • Y
    • a
    • h
    • i
    • b
    • d
    • e
    • f
    • c
    • g
    • j
    • k
    • l
    • m
    • n
    • o
    • p
    • q
    • r
    • s
    • t
    • u
    • x
    • w
    • y

    California

Setting

The study was conducted in 12 schools in the Sacramento City Unified School District (SCUSD), a large urban district in northern California.

Study sample

Under California’s interpretation of Reading First, all 59 elementary schools in SCUSD were required to implement one of two models of reading instruction, Success for All (SFA)® or Open Court Reading©. In the fall of 1997, four schools implemented SFA®. A matched sample of Open Court Reading© schools were created by rank-ordering SCUSD schools by poverty level (measured by the percent of students eligible for free or reduced-price meals and percent of students on Aid to Families with Dependent Children), and selecting two comparison schools for each SFA® school—those ranked just above and just below each SFA® school. The study included two cohorts of students: students in Cohort 1 began using the reading programs in grade 2, while students in Cohort 2 began in grade 3. A total of 936 students in Cohort 1 and Cohort 2 participated in the study, including students continually enrolled at study schools from fall 1997 to spring 1999 who completed all study tests and did not repeat a grade. The WWC based its effectiveness rating on findings from 434 Cohort 1 students who participated in the study; 292 in the Open Court Reading© group and 142 in the comparison group—these students were followed from second to third grade. Results for the Cohort 2 students are not included in this report because, based on information obtained from the authors, that sample of students was not equivalent on key characteristics at baseline.

Intervention Group

Students in the intervention group received reading instruction using Open Court Reading©, a systematic approach to teaching alphabetics, print knowledge, and phonemic awareness. For this study, the district used the 1996 version of the curricula, Open Court Collections for Young Scholars. Two hours of daily whole-class reading instruction was followed by 30 minutes of small-group instruction and/or independent work. All study students received a condensed selection of instructional content to “catch-up” students to Open Court Reading© content that they had not received in prior years (since they began using the curriculum in either second or third grade).

Comparison Group

Students in the comparison group received reading instruction through SFA®. Students were put into homogeneous groups, across classrooms and grades, based on reading skills. They received 90 minutes of reading instruction daily, outside of their homerooms. SFA® also prescribes additional writing instruction outside of these groups. The SFA® training consultants monitored implementation fidelity and observed additional writing instruction in all study schools during both study years. The authors noted that teachers in SFA® schools frequently included additional spelling and grammar, along with writing instruction, outside of the 90-minute reading block. A core reading curriculum is only prescribed in grades K–1; in grades 2–6, the schools can choose their own reading curricula. The authors state that the materials and guidelines for instruction (Reading Roots for grade 1, and Reading Wings for grades 2–4), as well as the professional development, tutoring, and the SFA® school facilitator and regional consultant oversight procedures, all followed those outlined by the developers of the curriculum.

Outcome descriptions

The outcome measure was the Reading subtest from the SAT-9, administered in both spring 1998 and spring 1999. The authors converted all measures to normal curve equivalent scores. For a more detailed description of this outcome measure, see Appendix B. The Language subtest from the SAT-9 was reported by the authors; however, this outcome measure is not included in this report because it is not an eligible outcome under the Beginning Reading evidence review protocol. The intermediate findings (after 1 year of implementation) for second graders are reported in Appendix D.1.

Support for implementation

At Open Court Reading© schools, teachers received 4 days of basic grade-level training in year 1, followed by 4 days of advanced grade-level training in year 2. Each Open Court Reading© school received a reading coach (either full-time or part-time, depending on school size). Curriculum experts met monthly with reading coaches and administrators to refine instruction and supervision and to solve problems. Reading coaches collected implementation information but were prohibited from sharing the information with the study authors; the district-level reading coordinator indicated that although some schools had implementation problems at the beginning of the study, these were resolved by the second study year. At SFA® schools, training and technical assistance was provided by SFA® consultants from a regional SFA® office. The SFA® consultants assessed implementation fidelity and rated it as a typical level of implementation when compared with national implementation averages.

Reviewed: August 2012

Study sample characteristics were not reported.
 

Your export should download shortly as a zip archive.

This download will include data files for study and findings review data and a data dictionary.

Connect With the WWC

loading
back to top