WWC review of this study

Effectiveness of Selected Supplemental Reading Comprehension Interventions: Impacts on a First Cohort of Fifth-Grade Students. NCEE 2009-4032

James-Burdumy, Susanne; Mansfield, Wendy; Deke, John; Carey, Nancy; Lugo-Gil, Julieta; Hershey, Alan; Douglas, Aaron; Gersten, Russell; Newman-Gonchar, Rebecca; Dimino, Joseph; Faddis, Bonnie (2009). National Center for Education Evaluation and Regional Assistance. Retrieved from: https://eric.ed.gov/?id=ED505578

  • Randomized Controlled Trial
     examining 
    2,332
     Students
    , grade
    5

Reviewed: June 2010

No statistically significant positive
findings
Meets WWC standards without reservations
Comprehension outcomes—Indeterminate effect found for the domain
Outcome
measure
Comparison Period Sample Intervention
mean
Comparison
mean
Significant? Improvement
    index
Evidence
tier

ETS: Science Comprehension

Project CRISS® vs. Business as usual

spring

Grade 5;
1,153 students

501.44

500.76

No

--

ETS: Social Studies Comprehension

Project CRISS® vs. Business as usual

spring

Grade 5;
1,140 students

499.64

500.61

No

--

Group Reading Assessment and Diagnostic Evaluation (GRADE): Passage Comprehension subtest

Project CRISS® vs. Business as usual

spring

Grade 5;
2,332 students

100.48

101.06

No

--


Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.

Characteristics of study sample as reported by study author.


  • Rural, Suburban, Urban
    • B
    • A
    • C
    • D
    • E
    • F
    • G
    • I
    • H
    • J
    • K
    • L
    • P
    • M
    • N
    • O
    • Q
    • R
    • S
    • V
    • U
    • T
    • W
    • X
    • Z
    • Y
    • a
    • h
    • i
    • b
    • d
    • e
    • f
    • c
    • g
    • j
    • k
    • l
    • m
    • n
    • o
    • p
    • q
    • r
    • s
    • t
    • u
    • x
    • w
    • y

    Arizona, California, Florida, Georgia, Louisiana, Oregon, Texas, Wisconsin
  • Race
    Asian
    2%
    Black
    37%
    Native American
    1%
    White
    31%
  • Ethnicity
    Hispanic    
    30%
    Not Hispanic or Latino    
    70%

Setting

The study took place in ten geographically diverse school districts in eight states (Arizona, California, Florida, Georgia, Louisiana, Oregon, Texas, and Wisconsin). To be eligible for the study, school districts needed to have (1) at least 12 schools that received Title I funds, (2) at least 40% of students eligible for the federal free or reduced-price lunch program, and (3) at least 60 5th-grade students per school. The school districts in the study were significantly larger, more disadvantaged, and more urban than the average U.S. school district.

Study sample

The study, which explored the impact of Project CRISS® as well as three other reading comprehension curricula (ReadAbout, Read for Real, and Reading for Knowledge), included 6,350 5th-grade students from 89 schools in ten school districts. Districts that had at least 12 Title I schools and who were not implementing any of the four selected curricula were recruited into the study. Within each school district, schools were randomly assigned either to one of the four intervention conditions or to the control group. Eligible students attended study schools and were enrolled in grade 5 when baseline tests were administered or transferred in after baseline and before January 1, 2007. Multiage grade levels and non-mainstreamed special education students were excluded from the sample. The analysis that is included in this review focused on the effect of Project CRISS® and examined a sample of 1,155 students attending 17 Project CRISS® schools and 1,183 students attending 21 control schools.

Intervention Group

Intervention group students received Project CRISS® strategies as part of their regular instruction. Instructional components included: (1) use of student and teacher editions of Learning How to Learn, which provided detailed lesson plans, learning, and practice through use of Tough Terminators, a science trade book; (2) use of a variety of graphic organizers and note-taking, discussion, vocabulary, and writing strategies; and (3) application of strategies to regular science and social studies texts. Project CRISS® teachers, on average, were observed engaging in 78% of teaching practices important to intervention implementation. The study reported students’ reading comprehension outcomes after nine months of program implementation.

Comparison Group

Control group schools did not have access to any of the four curricula being tested. Control group teachers could, however, use other supplemental reading programs.

Outcome descriptions

For the pretest, students took the passage comprehension subtest of the Group Reading Assessment and Diagnostic Evaluation (GRADE) and the Test of Silent Contextual Reading Fluency (TOSCRF). For the posttest, all students took the passage comprehension subtest of the GRADE. Students were also randomly assigned to take one of two reading comprehension assessments developed by the Educational Testing Service (ETS) for this study; these tests focused on either science or social studies. For a more detailed description of these outcome measures, see Appendix A2.

Support for implementation

Project CRISS® teachers received 18 hours of initial training, including 12 hours on using the strategies in the teacher’s guide and six hours on using the student text and workbook. Teachers received a training manual, a teacher’s guide, a student text, and a wrap-around edition of the student workbook. In addition, teachers received six hours of follow-up training. Trainers also visited schools monthly to observe teachers and provide feedback. The developer also encouraged teachers to use bi-weekly study teams in which teachers review and discuss their use of CRISS strategies.

Reviewed: February 2010

No statistically significant positive
findings
Meets WWC standards without reservations
Academic achievement outcomes—Statistically significant negative effect found for the domain
Outcome
measure
Comparison Period Sample Intervention
mean
Comparison
mean
Significant? Improvement
    index
Evidence
tier

Social Studies ETS

Reading For Knowledge vs. Business as usual

2006-07

Reading For Knowledge;
2,219 students

499.40

501.70

No

--

Group Reading Assessment and Diagnostic Evaluation (GRADE): Passage Comprehension subtest

Reading For Knowledge vs. Business as usual

2006-07

Reading For Knowledge;
2,246 students

99.25

100.80

Yes

-4
 
 

Composite Test Score

Reading For Knowledge vs. Business as usual

2006-07

Reading For Knowledge;
2,211 students

-0.10

0.02

Yes

-6
 
 

Science ETS

Reading For Knowledge vs. Business as usual

2006-07

Reading For Knowledge;
2,219 students

495.70

501.50

Yes

-8
 
 


Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.

Characteristics of study sample as reported by study author.


  • Rural, Suburban, Urban
    • B
    • A
    • C
    • D
    • E
    • F
    • G
    • I
    • H
    • J
    • K
    • L
    • P
    • M
    • N
    • O
    • Q
    • R
    • S
    • V
    • U
    • T
    • W
    • X
    • Z
    • Y
    • a
    • h
    • i
    • b
    • d
    • e
    • f
    • c
    • g
    • j
    • k
    • l
    • m
    • n
    • o
    • p
    • q
    • r
    • s
    • t
    • u
    • x
    • w
    • y

    Arizona, California, Florida, Georgia, Louisiana, Oregon, Texas, Wisconsin
  • Race
    Asian
    2%
    Black
    37%
    Native American
    1%
    White
    31%
  • Ethnicity
    Hispanic    
    30%
    Not Hispanic or Latino    
    70%

Reviewed: February 2010

No statistically significant positive
findings
Meets WWC standards without reservations
Academic achievement outcomes—Indeterminate effect found for the domain
Outcome
measure
Comparison Period Sample Intervention
mean
Comparison
mean
Significant? Improvement
    index
Evidence
tier

Social Studies ETS

ReadAbout vs. Business as usual

2006-07

Read ABout;
2,257 students

501.20

501.70

No

--

Science ETS

ReadAbout vs. Business as usual

2006-07

Read ABout;
2,257 students

500.60

501.50

No

--

Composite Test Score

ReadAbout vs. Business as usual

2006-07

Read ABout;
2,250 students

-0.03

0.02

No

--

Group Reading Assessment and Diagnostic Evaluation (GRADE): Passage Comprehension subtest

ReadAbout vs. Business as usual

2006-07

Read ABout;
2,274 students

99.83

100.80

No

--


Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.

Characteristics of study sample as reported by study author.


  • Rural, Suburban, Urban
    • B
    • A
    • C
    • D
    • E
    • F
    • G
    • I
    • H
    • J
    • K
    • L
    • P
    • M
    • N
    • O
    • Q
    • R
    • S
    • V
    • U
    • T
    • W
    • X
    • Z
    • Y
    • a
    • h
    • i
    • b
    • d
    • e
    • f
    • c
    • g
    • j
    • k
    • l
    • m
    • n
    • o
    • p
    • q
    • r
    • s
    • t
    • u
    • x
    • w
    • y

    Arizona, California, Florida, Georgia, Louisiana, Oregon, Texas, Wisconsin
  • Race
    Asian
    2%
    Black
    37%
    Native American
    1%
    White
    31%
  • Ethnicity
    Hispanic    
    30%
    Not Hispanic or Latino    
    70%

Reviewed: February 2010

No statistically significant positive
findings
Meets WWC standards without reservations
Academic achievement outcomes—Indeterminate effect found for the domain
Outcome
measure
Comparison Period Sample Intervention
mean
Comparison
mean
Significant? Improvement
    index
Evidence
tier

Group Reading Assessment and Diagnostic Evaluation (GRADE): Passage Comprehension subtest

Read for Real vs. Business as usual

2006-07

Read for Real;
2,257 students

99.92

100.80

No

--

Social Studies ETS

Read for Real vs. Business as usual

2006-07

Read for Real;
2,233 students

499.80

501.70

No

--

Science ETS

Read for Real vs. Business as usual

2006-07

Read for Real;
2,233 students

500.10

501.50

No

--

Composite Test Scores

Read for Real vs. Business as usual

2006-07

Read for Real;
2,221 students

-0.05

0.02

No

--


Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.

Characteristics of study sample as reported by study author.


  • Rural, Suburban, Urban
    • B
    • A
    • C
    • D
    • E
    • F
    • G
    • I
    • H
    • J
    • K
    • L
    • P
    • M
    • N
    • O
    • Q
    • R
    • S
    • V
    • U
    • T
    • W
    • X
    • Z
    • Y
    • a
    • h
    • i
    • b
    • d
    • e
    • f
    • c
    • g
    • j
    • k
    • l
    • m
    • n
    • o
    • p
    • q
    • r
    • s
    • t
    • u
    • x
    • w
    • y

    Arizona, California, Florida, Georgia, Louisiana, Oregon, Texas, Wisconsin
  • Race
    Asian
    2%
    Black
    37%
    Native American
    1%
    White
    31%
  • Ethnicity
    Hispanic    
    30%
    Not Hispanic or Latino    
    70%

Reviewed: February 2010

No statistically significant positive
findings
Meets WWC standards without reservations
Academic achievement outcomes—Indeterminate effect found for the domain
Outcome
measure
Comparison Period Sample Intervention
mean
Comparison
mean
Significant? Improvement
    index
Evidence
tier

Science ETS

Project CRISS® vs. Business as usual

2006-07

Project CRISS;
2,307 students

502.20

501.50

No

--

Social Studies ETS

Project CRISS® vs. Business as usual

2006-07

Project CRISS;
2,307 students

500.80

501.70

No

--

Composite Test Score

Project CRISS® vs. Business as usual

2006-07

Project CRISS;
2,301 students

0.00

0.02

No

--

Group Reading Assessment and Diagnostic Evaluation (GRADE): Passage Comprehension subtest

Project CRISS® vs. Business as usual

2006-07

Project CRISS;
2,332 students

100.20

100.80

No

--


Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.

Characteristics of study sample as reported by study author.


  • Rural, Suburban, Urban
    • B
    • A
    • C
    • D
    • E
    • F
    • G
    • I
    • H
    • J
    • K
    • L
    • P
    • M
    • N
    • O
    • Q
    • R
    • S
    • V
    • U
    • T
    • W
    • X
    • Z
    • Y
    • a
    • h
    • i
    • b
    • d
    • e
    • f
    • c
    • g
    • j
    • k
    • l
    • m
    • n
    • o
    • p
    • q
    • r
    • s
    • t
    • u
    • x
    • w
    • y

    Arizona, California, Florida, Georgia, Louisiana, Oregon, Texas, Wisconsin
  • Race
    Asian
    2%
    Black
    37%
    Native American
    1%
    White
    31%
  • Ethnicity
    Hispanic    
    30%
    Not Hispanic or Latino    
    70%
 

Your export should download shortly as a zip archive.

This download will include data files for study and findings review data and a data dictionary.

Connect With the WWC

loading
back to top