WWC review of this study

Effectiveness of selected supplemental reading comprehension interventions: Impacts on a first cohort of fifth-grade students (NCEE 2009-4032).

James-Burdumy, S., Mansfield, W., Deke, J., Carey, N., Lugo-Gil, J., Hershey, A., Douglas, A., Gersten, R., Newman-Gonchar, R., Dimino, J., & Faddis, B. (2009). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. Retrieved from: https://eric.ed.gov/?id=ED505578

  • Randomized Controlled Trial
     examining 
    2,332
     Students
    , grade
    5
No statistically significant positive
findings
Meets WWC standards without reservations

Reviewed: June 2010

Comprehension outcomes—Indeterminate effects found
Outcome
measure
Comparison Period Sample Intervention
mean
Comparison
mean
Significant? Improvement
index

ETS: Science Comprehension

Project CRISS® vs. Business as usual

spring

Grade 5;
1,153 students

501.44

500.76

No

--
More Outcomes

ETS: Social Studies Comprehension

Project CRISS® vs. Business as usual

spring

Grade 5;
1,140 students

499.64

500.61

No

--

Group Reading Assessment and Diagnostic Evaluation (GRADE): Passage Comprehension subtest

Project CRISS® vs. Business as usual

spring

Grade 5;
2,332 students

100.48

101.06

No

--

Characteristics of study sample as reported by study author.


  • 77% Free or reduced price lunch
  • Race
    Asian
    2%
    Black
    37%
    Native American
    1%
    White
    31%
  • Ethnicity
    Hispanic
    30%
    Not Hispanic
    70%

  • Rural, Suburban, Urban
    • B
    • A
    • C
    • D
    • E
    • F
    • G
    • I
    • H
    • J
    • K
    • L
    • P
    • M
    • N
    • O
    • Q
    • R
    • S
    • V
    • U
    • T
    • W
    • X
    • Z
    • Y
    • a
    • h
    • i
    • b
    • d
    • e
    • f
    • c
    • g
    • j
    • k
    • l
    • m
    • n
    • o
    • p
    • q
    • r
    • s
    • t
    • u
    • x
    • w
    • y

    Arizona, California, Florida, Georgia, Louisiana, Oregon, Texas, Wisconsin

Setting

The study took place in ten geographically diverse school districts in eight states (Arizona, California, Florida, Georgia, Louisiana, Oregon, Texas, and Wisconsin). To be eligible for the study, school districts needed to have (1) at least 12 schools that received Title I funds, (2) at least 40% of students eligible for the federal free or reduced-price lunch program, and (3) at least 60 5th-grade students per school. The school districts in the study were significantly larger, more disadvantaged, and more urban than the average U.S. school district.

Study sample

The study, which explored the impact of Project CRISS® as well as three other reading comprehension curricula (ReadAbout, Read for Real, and Reading for Knowledge), included 6,350 5th-grade students from 89 schools in ten school districts. Districts that had at least 12 Title I schools and who were not implementing any of the four selected curricula were recruited into the study. Within each school district, schools were randomly assigned either to one of the four intervention conditions or to the control group. Eligible students attended study schools and were enrolled in grade 5 when baseline tests were administered or transferred in after baseline and before January 1, 2007. Multiage grade levels and non-mainstreamed special education students were excluded from the sample. The analysis that is included in this review focused on the effect of Project CRISS® and examined a sample of 1,155 students attending 17 Project CRISS® schools and 1,183 students attending 21 control schools.

Intervention Group

Intervention group students received Project CRISS® strategies as part of their regular instruction. Instructional components included: (1) use of student and teacher editions of Learning How to Learn, which provided detailed lesson plans, learning, and practice through use of Tough Terminators, a science trade book; (2) use of a variety of graphic organizers and note-taking, discussion, vocabulary, and writing strategies; and (3) application of strategies to regular science and social studies texts. Project CRISS® teachers, on average, were observed engaging in 78% of teaching practices important to intervention implementation. The study reported students’ reading comprehension outcomes after nine months of program implementation.

Comparison Group

Control group schools did not have access to any of the four curricula being tested. Control group teachers could, however, use other supplemental reading programs.

Outcome descriptions

For the pretest, students took the passage comprehension subtest of the Group Reading Assessment and Diagnostic Evaluation (GRADE) and the Test of Silent Contextual Reading Fluency (TOSCRF). For the posttest, all students took the passage comprehension subtest of the GRADE. Students were also randomly assigned to take one of two reading comprehension assessments developed by the Educational Testing Service (ETS) for this study; these tests focused on either science or social studies. For a more detailed description of these outcome measures, see Appendix A2.

Support for implementation

Project CRISS® teachers received 18 hours of initial training, including 12 hours on using the strategies in the teacher’s guide and six hours on using the student text and workbook. Teachers received a training manual, a teacher’s guide, a student text, and a wrap-around edition of the student workbook. In addition, teachers received six hours of follow-up training. Trainers also visited schools monthly to observe teachers and provide feedback. The developer also encouraged teachers to use bi-weekly study teams in which teachers review and discuss their use of CRISS strategies.

No statistically significant positive
findings
Meets WWC standards without reservations

Reviewed: February 2010

Academic achievement outcomes—Statistically significant negative effects found
Outcome
measure
Comparison Period Sample Intervention
mean
Comparison
mean
Significant? Improvement
index

Social Studies ETS

Reading For Knowledge vs. Business as usual

2006-07

Reading For Knowledge;
2,219 students

499.4

501.7

No

--
More Outcomes

Group Reading Assessment and Diagnostic Evaluation (GRADE): Passage Comprehension subtest

Reading For Knowledge vs. Business as usual

2006-07

Reading For Knowledge;
2,246 students

99.25

100.8

Yes

-4
 
 

Composite Test Score

Reading For Knowledge vs. Business as usual

2006-07

Reading For Knowledge;
2,211 students

-0.1

0.02

Yes

-6
 
 

Science ETS

Reading For Knowledge vs. Business as usual

2006-07

Reading For Knowledge;
2,219 students

495.7

501.5

Yes

-8
 
 

Characteristics of study sample as reported by study author.


  • 77% Free or reduced price lunch
  • Race
    Asian
    2%
    Black
    37%
    Native American
    1%
    White
    31%
  • Ethnicity
    Hispanic
    30%
    Not Hispanic
    70%

  • Rural, Suburban, Urban
    • B
    • A
    • C
    • D
    • E
    • F
    • G
    • I
    • H
    • J
    • K
    • L
    • P
    • M
    • N
    • O
    • Q
    • R
    • S
    • V
    • U
    • T
    • W
    • X
    • Z
    • Y
    • a
    • h
    • i
    • b
    • d
    • e
    • f
    • c
    • g
    • j
    • k
    • l
    • m
    • n
    • o
    • p
    • q
    • r
    • s
    • t
    • u
    • x
    • w
    • y

    Arizona, California, Florida, Georgia, Louisiana, Oregon, Texas, Wisconsin
No statistically significant positive
findings
Meets WWC standards without reservations

Reviewed: February 2010

Academic achievement outcomes—Indeterminate effects found
Outcome
measure
Comparison Period Sample Intervention
mean
Comparison
mean
Significant? Improvement
index

Social Studies ETS

ReadAbout vs. Business as usual

2006-07

Read ABout;
2,257 students

501.2

501.7

No

--
More Outcomes

Science ETS

ReadAbout vs. Business as usual

2006-07

Read ABout;
2,257 students

500.6

501.5

No

--

Composite Test Score

ReadAbout vs. Business as usual

2006-07

Read ABout;
2,250 students

-0.03

0.02

No

--

Group Reading Assessment and Diagnostic Evaluation (GRADE): Passage Comprehension subtest

ReadAbout vs. Business as usual

2006-07

Read ABout;
2,274 students

99.83

100.8

No

--

Characteristics of study sample as reported by study author.


  • 77% Free or reduced price lunch
  • Race
    Asian
    2%
    Black
    37%
    Native American
    1%
    White
    31%
  • Ethnicity
    Hispanic
    30%
    Not Hispanic
    70%

  • Rural, Suburban, Urban
    • B
    • A
    • C
    • D
    • E
    • F
    • G
    • I
    • H
    • J
    • K
    • L
    • P
    • M
    • N
    • O
    • Q
    • R
    • S
    • V
    • U
    • T
    • W
    • X
    • Z
    • Y
    • a
    • h
    • i
    • b
    • d
    • e
    • f
    • c
    • g
    • j
    • k
    • l
    • m
    • n
    • o
    • p
    • q
    • r
    • s
    • t
    • u
    • x
    • w
    • y

    Arizona, California, Florida, Georgia, Louisiana, Oregon, Texas, Wisconsin
No statistically significant positive
findings
Meets WWC standards without reservations

Reviewed: February 2010

Academic achievement outcomes—Indeterminate effects found
Outcome
measure
Comparison Period Sample Intervention
mean
Comparison
mean
Significant? Improvement
index

Group Reading Assessment and Diagnostic Evaluation (GRADE): Passage Comprehension subtest

Read for Real vs. Business as usual

2006-07

Read for Real;
2,257 students

99.92

100.8

No

--
More Outcomes

Social Studies ETS

Read for Real vs. Business as usual

2006-07

Read for Real;
2,233 students

499.8

501.7

No

--

Science ETS

Read for Real vs. Business as usual

2006-07

Read for Real;
2,233 students

500.1

501.5

No

--

Composite Test Scores

Read for Real vs. Business as usual

2006-07

Read for Real;
2,221 students

-0.05

0.02

No

--

Characteristics of study sample as reported by study author.


  • 77% Free or reduced price lunch
  • Race
    Asian
    2%
    Black
    37%
    Native American
    1%
    White
    31%
  • Ethnicity
    Hispanic
    30%
    Not Hispanic
    70%

  • Rural, Suburban, Urban
    • B
    • A
    • C
    • D
    • E
    • F
    • G
    • I
    • H
    • J
    • K
    • L
    • P
    • M
    • N
    • O
    • Q
    • R
    • S
    • V
    • U
    • T
    • W
    • X
    • Z
    • Y
    • a
    • h
    • i
    • b
    • d
    • e
    • f
    • c
    • g
    • j
    • k
    • l
    • m
    • n
    • o
    • p
    • q
    • r
    • s
    • t
    • u
    • x
    • w
    • y

    Arizona, California, Florida, Georgia, Louisiana, Oregon, Texas, Wisconsin
No statistically significant positive
findings
Meets WWC standards without reservations

Reviewed: February 2010

Academic achievement outcomes—Indeterminate effects found
Outcome
measure
Comparison Period Sample Intervention
mean
Comparison
mean
Significant? Improvement
index

Science ETS

Project CRISS® vs. Business as usual

2006-07

Project CRISS;
2,307 students

502.2

501.5

No

--
More Outcomes

Social Studies ETS

Project CRISS® vs. Business as usual

2006-07

Project CRISS;
2,307 students

500.8

501.7

No

--

Composite Test Score

Project CRISS® vs. Business as usual

2006-07

Project CRISS;
2,301 students

0

0.02

No

--

Group Reading Assessment and Diagnostic Evaluation (GRADE): Passage Comprehension subtest

Project CRISS® vs. Business as usual

2006-07

Project CRISS;
2,332 students

100.2

100.8

No

--

Characteristics of study sample as reported by study author.


  • 77% Free or reduced price lunch
  • Race
    Asian
    2%
    Black
    37%
    Native American
    1%
    White
    31%
  • Ethnicity
    Hispanic
    30%
    Not Hispanic
    70%

  • Rural, Suburban, Urban
    • B
    • A
    • C
    • D
    • E
    • F
    • G
    • I
    • H
    • J
    • K
    • L
    • P
    • M
    • N
    • O
    • Q
    • R
    • S
    • V
    • U
    • T
    • W
    • X
    • Z
    • Y
    • a
    • h
    • i
    • b
    • d
    • e
    • f
    • c
    • g
    • j
    • k
    • l
    • m
    • n
    • o
    • p
    • q
    • r
    • s
    • t
    • u
    • x
    • w
    • y

    Arizona, California, Florida, Georgia, Louisiana, Oregon, Texas, Wisconsin
 

Your export should download shortly as a zip archive.

This download will include data files for study and findings review data and a data dictionary.

Connect With the WWC

newsflash icon contact icon facebook icon twitter icon
loading
back to top