A multisite cluster randomized field trial of Open Court Reading.

Findings

Comprehension outcomes—Statistically significant positive effect found for the domain

Outcome measure	Comparison	Period	Sample	Intervention mean	Comparison mean	Significant?	Improvement     index	Evidence tier
Comprehensive Test of Basic Skills, 5th Edition (CTBS/5): Reading Composite Score	Open Court Reading© vs. business as usual	October - May	Grades 1-3; 679 students	603.07	590.98	Yes	10

Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.

Study Details

Setting

The study initially included six schools—one each in Florida, Georgia, Idaho, Indiana, North Carolina, and Texas. Two schools were from rural areas, two from suburban areas, and two from urban areas. The Georgia school dropped out of the study.

Study sample

McGraw-Hill Education recruited a group of schools that had not previously used Open Court Reading© to participate in the study. The six schools that initially participated were given free Open Court Reading© materials, as well as a teacher training program and implementation support. At each school, classrooms were randomly assigned within each grade either to be enrolled in Open Court Reading© or to serve as the comparison group. The entire study sample consisted of 57 grade 1–5 classrooms containing a total of 1,099 students. The sample considered in this review, which aligns to the Beginning Reading review protocol, initially consisted of 44 grade 1–3 classrooms containing a total of 855 students. After attrition, the combined analysis sample consisted of 36 classrooms containing 679 students in grades 1–3; 379 students in the 20 Open Court Reading© classrooms and 300 students in 16 comparison classrooms. Of the participating students, more than 70% were minorities, and more than 75% were eligible for free or reduced-price lunches. Fewer than 15% were English as Second Language (ESL) students, and fewer than 10% were special education students.

Intervention Group

Open Court Reading© is a curriculum that includes textbooks, workbooks, decodable books, and anthologies. The curriculum consists of three main components: (a) Preparing to Read, (b) Reading and Responding, and (c) Language Arts. For this study, teachers were given a teacher’s edition of the curriculum that included scripted direct instruction lessons and diagnostic and assessment packages. The program is designed to be used for 2.5 hours per day with grades 1–2 and for 2 hours per day with grades 4–6. However, the authors report that external consultants observed that some teachers provided only 90 minutes of daily instruction. The intervention was implemented from fall to spring during the 2005–06 school year.

Comparison Group

The comparison classrooms used a “business-as-usual” curriculum and were instructed not to use Open Court Reading© or any of its materials. Principals mentioned that curricula currently in use by the comparison classrooms consisted of Reading Street by Scott Foresman, Literacy Place by Scholastic, McGraw-Hill Reading by MacMillan/McGraw-Hill, Collections by Harcourt, and Trophies by Harcourt. Consultants visited comparison classrooms and verified that they were not using Open Court Reading©.

Outcome descriptions

For both the pretest (October 2005) and the posttest (May 2006), students took the CTBS/5 Terra Nova Reading Comprehension and Vocabulary subtests. A Reading Composite score was also reported, which is the average of these two subtest measures. For a more detailed description of these outcome measures, see Appendix B. Findings for the combined student sample on the Reading Composite score can be found in Appendix C.2. Additional findings reflecting subtest outcomes separately for grades 1, 2, and 3 can be found in Appendix D.2.

Support for implementation

Teachers were provided training opportunities with external consultants, which consisted of 2- to 3-day summer workshops. In addition, the consultants, who had teaching experience and detailed knowledge of Open Court Reading© and were trained by McGraw-Hill Education, visited and observed classrooms, and provided feedback to teachers.

Findings

Comprehension outcomes—Indeterminate effect found for the domain

Outcome measure	Comparison	Period	Sample	Intervention mean	Comparison mean	Significant?	Improvement     index	Evidence tier
Comprehensive Test of Basic Skills, 5th Edition (CTBS/5): Terra Nova Reading Composite score	Open Court Reading© vs. Business as usual	Posttest	Grades 1-5; 917 students	612.77	604.82	No	--

Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.

Study Details

Setting

The study initially included six schools—one each in Florida, Georgia, Idaho, Indiana, North Carolina, and Texas. Two schools were from rural areas, two from suburban areas, and two from urban areas. The Georgia school dropped out of the study.

Study sample

SRA/McGraw-Hill recruited a group of schools that had not previously used Open Court Reading© to participate in the study. The six schools that initially participated were given free Open Court Reading© materials as well as a training program for teachers and implementation support. At each school, classrooms were randomly assigned within each grade either to be enrolled in Open Court Reading© or to serve as the comparison group. The initial sample consisted of 57 grade 1–5 classrooms containing a total of 1,099 students. The Georgia school dropped out of the study, which resulted in a loss of four classrooms in both the intervention and comparison groups. Some students were absent during the administration of the posttest. However, the resulting attrition rates of schools, classrooms, and students were low. The analysis sample consisted of students in grades 1–5 and included 507 students in the 27 Open Court Reading© classrooms and 410 students in the 22 comparison classrooms. Participating students were more than 70% minority, and more than 75% were eligible for free or reduced price lunches. Fewer than 15% were English as a Second Language (ESL) students, and fewer than 10% were special education students.

Intervention Group

Open Court Reading© is a curriculum that includes textbooks, workbooks, decodable books, and anthologies. The curriculum consists of three main components: (a) Preparing to Read, (b) Reading and Responding, and (c) Language Arts. Teachers were given a teacher’s edition of the curriculum that included scripted direct instruction lessons; and diagnostic and assessment packages. The program is designed to be used for 2.5 hours per day with grades 1–2 and for two hours per day with grades 4–6. However, the authors report that external consultants observed that some teachers provided only 90 minutes of daily instruction. The intervention was implemented from fall to spring during the 2005–06 school year.

Comparison Group

The comparison classrooms used a “business as usual” curriculum and were instructed not to use Open Court Reading© or any of its materials. Principals mentioned that curricula currently in use by the comparison classrooms consisted of Reading Street by Scott Foresman, Literacy Place by Scholastic, McGraw-Hill Reading by MacMillan/McGraw-Hill, Collections by Harcourt, and Trophies by Harcourt. Consultants visited comparison classrooms and verified that they were not using Open Court Reading©.

Outcome descriptions

For both the pretest (October 2005) and the posttest (May 2006), students took the Comprehensive Test of Basic Skills, 5th edition (CTBS/5) Terra Nova Reading Comprehension and Vocabulary subtests. A Reading Composite score also is reported, which is the average of these two subtest measures. For a more detailed description of these outcome measures, see Appendix B. Findings for the combined student sample on the Reading Composite score can be found in Appendix C. Additional findings reflecting subtest outcomes for the combined student sample and separately for grades 4 and 5 can be found in Appendix D.

Support for implementation

Teachers were provided training opportunities with external consultants, which consisted of two- to three-day summer workshops. In addition, the consultants, who had teaching experience and detailed knowledge of Open Court Reading© and were trained by SRA/McGraw-Hill, visited and observed classrooms and provided feedback to teachers.