WWC review of this study

Effectiveness of Selected Supplemental Reading Comprehension Interventions: Impacts on a First Cohort of Fifth-Grade Students. NCEE 2009-4032

James-Burdumy, Susanne; Mansfield, Wendy; Deke, John; Carey, Nancy; Lugo-Gil, Julieta; Hershey, Alan; Douglas, Aaron; Gersten, Russell; Newman-Gonchar, Rebecca; Dimino, Joseph; Faddis, Bonnie (2009). National Center for Education Evaluation and Regional Assistance. Retrieved from: https://eric.ed.gov/?id=ED505578

examining

2,332

Students

, grade

5

Project CRISS® Intervention Report - Adolescent Literacy

Review Details

Reviewed: June 2010

No statistically significant positive
findings

Meets WWC standards without reservations

For:

Project CRISS® Intervention Report - Adolescent Literacy

Using:

Research Design:

Randomized Controlled Trial

Rating:

Meets WWC standards without reservations

This review may not reflect the full body of research evidence for this intervention.

Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.

Please see the WWC summary of evidence for Project CRISS®.

Findings

Comprehension outcomes—Indeterminate effect found for the domain
Outcome measure	Comparison	Period	Sample	Intervention mean	Comparison mean	Significant?	Improvement index
ETS: Science Comprehension	Project CRISS® vs. Business as usual	spring	Grade 5; 1,153 students	501.44	500.76	No	--
ETS: Social Studies Comprehension	Project CRISS® vs. Business as usual	spring	Grade 5; 1,140 students	499.64	500.61	No	--
Group Reading Assessment and Diagnostic Evaluation (GRADE): Passage Comprehension subtest	Project CRISS® vs. Business as usual	spring	Grade 5; 2,332 students	100.48	101.06	No	--

Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.

Sample Characteristics

Characteristics of study sample as reported by study author.

Rural, Suburban, Urban
- B
- A
- C
- D
- E
- F
- G
- I
- H
- J
- K
- L
- P
- M
- N
- O
- Q
- R
- S
- V
- U
- T
- W
- X
- Z
- Y
- a
- h
- i
- b
- d
- e
- f
- c
- g
- j
- k
- l
- m
- n
- o
- p
- q
- r
- s
- t
- u
- v
- x
- w
- y
Arizona, California, Florida, Georgia, Louisiana, Oregon, Texas, Wisconsin

Race
Asian		2%
Black		37%
Native American		1%
White		31%

Ethnicity
Hispanic		30%
Not Hispanic or Latino		70%

Study Details

Setting

The study took place in ten geographically diverse school districts in eight states (Arizona, California, Florida, Georgia, Louisiana, Oregon, Texas, and Wisconsin). To be eligible for the study, school districts needed to have (1) at least 12 schools that received Title I funds, (2) at least 40% of students eligible for the federal free or reduced-price lunch program, and (3) at least 60 5th-grade students per school. The school districts in the study were significantly larger, more disadvantaged, and more urban than the average U.S. school district.

Study sample

The study, which explored the impact of Project CRISS® as well as three other reading comprehension curricula (ReadAbout, Read for Real, and Reading for Knowledge), included 6,350 5th-grade students from 89 schools in ten school districts. Districts that had at least 12 Title I schools and who were not implementing any of the four selected curricula were recruited into the study. Within each school district, schools were randomly assigned either to one of the four intervention conditions or to the control group. Eligible students attended study schools and were enrolled in grade 5 when baseline tests were administered or transferred in after baseline and before January 1, 2007. Multiage grade levels and non-mainstreamed special education students were excluded from the sample. The analysis that is included in this review focused on the effect of Project CRISS® and examined a sample of 1,155 students attending 17 Project CRISS® schools and 1,183 students attending 21 control schools.

Intervention Group

Intervention group students received Project CRISS® strategies as part of their regular instruction. Instructional components included: (1) use of student and teacher editions of Learning How to Learn, which provided detailed lesson plans, learning, and practice through use of Tough Terminators, a science trade book; (2) use of a variety of graphic organizers and note-taking, discussion, vocabulary, and writing strategies; and (3) application of strategies to regular science and social studies texts. Project CRISS® teachers, on average, were observed engaging in 78% of teaching practices important to intervention implementation. The study reported students’ reading comprehension outcomes after nine months of program implementation.

Comparison Group

Control group schools did not have access to any of the four curricula being tested. Control group teachers could, however, use other supplemental reading programs.

Outcome descriptions

For the pretest, students took the passage comprehension subtest of the Group Reading Assessment and Diagnostic Evaluation (GRADE) and the Test of Silent Contextual Reading Fluency (TOSCRF). For the posttest, all students took the passage comprehension subtest of the GRADE. Students were also randomly assigned to take one of two reading comprehension assessments developed by the Educational Testing Service (ETS) for this study; these tests focused on either science or social studies. For a more detailed description of these outcome measures, see Appendix A2.

Support for implementation

Project CRISS® teachers received 18 hours of initial training, including 12 hours on using the strategies in the teacher’s guide and six hours on using the student text and workbook. Teachers received a training manual, a teacher’s guide, a student text, and a wrap-around edition of the student workbook. In addition, teachers received six hours of follow-up training. Trainers also visited schools monthly to observe teachers and provide feedback. The developer also encouraged teachers to use bi-weekly study teams in which teachers review and discuss their use of CRISS strategies.