WWC review of this study

Large scale, randomized cluster design study of the relative effectiveness of reform-based and traditional/verification curricula in supporting student science learning.

Granger, E. M., Bevis, T. H., Saka, Y., & Southerland, S. A. (2010, March). Paper presented at the annual meeting of the National Association for Research in Science Teaching, Philadelphia, PA.

  • Randomized controlled trial
    , grades

Reviewed: June 2012

At least one finding shows strong evidence of effectiveness
At least one statistically significant positive finding
Meets WWC standards without reservations
Science Achievement outcomes—Statistically significant positive effect found for the domain
Comparison Period Sample Intervention
Significant? Improvement

Space Science Contest test

Great Explorations in Math and Science (GEMS) Space Science Sequence vs. Business as usual


Grades 4-5;
2,594 students





Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.

Characteristics of study sample as reported by study author.

  • 2% English language learners

  • Female: 50%
    Male: 50%
    • B
    • A
    • C
    • D
    • E
    • F
    • G
    • I
    • H
    • J
    • K
    • L
    • P
    • M
    • N
    • O
    • Q
    • R
    • S
    • V
    • U
    • T
    • W
    • X
    • Z
    • Y
    • a
    • h
    • i
    • b
    • d
    • e
    • f
    • c
    • g
    • j
    • k
    • l
    • m
    • n
    • o
    • p
    • q
    • r
    • s
    • t
    • u
    • x
    • w
    • y

  • Race


The study was conducted in a county in central Florida during the 2007–08 and 2008–09 school years.

Study sample

The study used a randomized cluster experimental design. Volunteer teachers were matched on student demographics and grade level and then were randomly assigned to either the intervention group or the comparison group. Over a two-year period, 140 teachers were randomly assigned—70 to the intervention group and 70 to the comparison group. The total analysis sample across both years included 66 teachers in intervention classrooms and 59 teachers in comparison classrooms. The overall and differential attrition rates of teachers (11% and 10%, respectively) met WWC standards for low attrition. The student analysis sample included 1,418 students who received the GEMS® Space Science Sequence and 1,176 comparison group students who received the typical space science instruction available in the district. Attrition rates of students were unknown. About 40% of these students were in the fourth grade; the rest were fifth graders. The students were evenly split between boys and girls (50% male, 50% female). Almost one-third (31%) were eligible for free and reduced-price lunch. About 62% of the students were White. Three percent of the sample were English language learners. The study reported students’ outcomes immediately following completion of the space science sequence. These findings can be found in Appendix C. Additional findings reflecting students’ follow-up outcomes five months after the completion of the space science sequence can be found in Appendix D.

Intervention Group

Students in the intervention group received GEMS® Space Science Sequence (Lawrence Hall of Science, 2007) for grades 3–5, which was designed to address age-appropriate core concepts in space science and common misconceptions that students might have about them. Students investigated size and scale relative to distance, the Earth’s shape and gravity, how the Earth moves, and moon phases and eclipses in four units, over 24 sixty-minute class sessions. The curriculum had an explicit focus on the role of models and evidence in science. Throughout the unit, students evaluated alternative explanations and used evidence to support explanations and to critique the merits of an explanation.

Comparison Group

Comparison teachers used the standard district text for grades 4 and 5 to address the same space science content as the intervention group. The district curriculum was centered on a more didactic presentation of space science concepts, including direct instruction, reading of text, and students answering very focused questions.

Outcome descriptions

Student outcomes were assessed with the Space Science Content test (Sadler, Coyle, Cook-Smith, & Miller, 2007). The assessments were given to students prior to space science instruction, two weeks following completion of teaching the space science unit, and at the five-month follow-up. For a more detailed description of these outcome measures, see Appendix B. The study also used assessments that did not meet inclusion criteria as outcome measures for the Science topic area: the Homerton Science Attitudes survey, the Models and Evidence Questionnaire, and Views of Scientific Inquiry.

Support for implementation

Teachers in the intervention condition were given four days of preservice professional development to learn about the specific curriculum before the school year, a three-hour follow-up training before the curriculum was implemented, and access to a “science coach” midway through teaching the unit that was tested. In addition, all teachers were offered basic professional development related to the new textbook being used by the district in all space science classes. Teachers in both groups were instructed to refrain from adding any activities to those present in their assigned curriculum. The study does not provide information on the education or experience of teachers.

In the case of multiple manuscripts that report on one study, the WWC selects one manuscript as the primary citation and lists other manuscripts that describe the study as additional sources.

  • Granger, E. M., Bevis, T. H., Saka, Y., & Southerland, S. A. (2009, April). Comparing the efficacy of reform-based and traditional/verification curricula to support student learning about space science. Paper presented at the annual meeting of the National Association for Research in Science Teaching, Garden Grove, CA.


Your export should download shortly as a zip archive.

This download will include data files for study and findings review data and a data dictionary.

Connect With the WWC

back to top