Large scale, randomized cluster design study of the relative effectiveness of reform-based and traditional/verification curricula in supporting student science learning.
Granger, E. M., Bevis, T. H., Saka, Y., & Southerland, S. A. (2010, March). Paper presented at the annual meeting of the National Association for Research in Science Teaching, Philadelphia, PA.
-
examining2,594Students, grades4-5
Great Explorations in Math and Science (GEMS) Space Science Sequence Intervention Report - Science
Review Details
Reviewed: June 2012
- Randomized controlled trial
- Meets WWC standards without reservations
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Please see the WWC summary of evidence for Great Explorations in Math and Science (GEMS) Space Science Sequence.
Findings
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Space Science Contest test |
Great Explorations in Math and Science (GEMS) Space Science Sequence vs. Business as usual |
posttest |
Grades 4-5;
|
N/A |
N/A |
Yes |
|
|
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Sample Characteristics
Characteristics of study sample as reported by study author.
-
2% English language learners -
Female: 50%
Male: 50% -
- B
- A
- C
- D
- E
- F
- G
- I
- H
- J
- K
- L
- P
- M
- N
- O
- Q
- R
- S
- V
- U
- T
- W
- X
- Z
- Y
- a
- h
- i
- b
- d
- e
- f
- c
- g
- j
- k
- l
- m
- n
- o
- p
- q
- r
- s
- t
- u
- v
- x
- w
- y
Florida
-
Race White 62%
Study Details
Setting
The study was conducted in a county in central Florida during the 2007–08 and 2008–09 school years.
Study sample
The study used a randomized cluster experimental design. Volunteer teachers were matched on student demographics and grade level and then were randomly assigned to either the intervention group or the comparison group. Over a two-year period, 140 teachers were randomly assigned—70 to the intervention group and 70 to the comparison group. The total analysis sample across both years included 66 teachers in intervention classrooms and 59 teachers in comparison classrooms. The overall and differential attrition rates of teachers (11% and 10%, respectively) met WWC standards for low attrition. The student analysis sample included 1,418 students who received the GEMS® Space Science Sequence and 1,176 comparison group students who received the typical space science instruction available in the district. Attrition rates of students were unknown. About 40% of these students were in the fourth grade; the rest were fifth graders. The students were evenly split between boys and girls (50% male, 50% female). Almost one-third (31%) were eligible for free and reduced-price lunch. About 62% of the students were White. Three percent of the sample were English language learners. The study reported students’ outcomes immediately following completion of the space science sequence. These findings can be found in Appendix C. Additional findings reflecting students’ follow-up outcomes five months after the completion of the space science sequence can be found in Appendix D.
Intervention Group
Students in the intervention group received GEMS® Space Science Sequence (Lawrence Hall of Science, 2007) for grades 3–5, which was designed to address age-appropriate core concepts in space science and common misconceptions that students might have about them. Students investigated size and scale relative to distance, the Earth’s shape and gravity, how the Earth moves, and moon phases and eclipses in four units, over 24 sixty-minute class sessions. The curriculum had an explicit focus on the role of models and evidence in science. Throughout the unit, students evaluated alternative explanations and used evidence to support explanations and to critique the merits of an explanation.
Comparison Group
Comparison teachers used the standard district text for grades 4 and 5 to address the same space science content as the intervention group. The district curriculum was centered on a more didactic presentation of space science concepts, including direct instruction, reading of text, and students answering very focused questions.
Outcome descriptions
Student outcomes were assessed with the Space Science Content test (Sadler, Coyle, Cook-Smith, & Miller, 2007). The assessments were given to students prior to space science instruction, two weeks following completion of teaching the space science unit, and at the five-month follow-up. For a more detailed description of these outcome measures, see Appendix B. The study also used assessments that did not meet inclusion criteria as outcome measures for the Science topic area: the Homerton Science Attitudes survey, the Models and Evidence Questionnaire, and Views of Scientific Inquiry.
Support for implementation
Teachers in the intervention condition were given four days of preservice professional development to learn about the specific curriculum before the school year, a three-hour follow-up training before the curriculum was implemented, and access to a “science coach” midway through teaching the unit that was tested. In addition, all teachers were offered basic professional development related to the new textbook being used by the district in all space science classes. Teachers in both groups were instructed to refrain from adding any activities to those present in their assigned curriculum. The study does not provide information on the education or experience of teachers.
Additional Sources
In the case of multiple manuscripts that report on one study, the WWC selects one manuscript as the primary citation and lists other manuscripts that describe the study as additional sources.
-
Granger, E. M., Bevis, T. H., Saka, Y., & Southerland, S. A. (2009, April). Comparing the efficacy of reform-based and traditional/verification curricula to support student learning about space science. Paper presented at the annual meeting of the National Association for Research in Science Teaching, Garden Grove, CA.
An indicator of the effect of the intervention, the improvement index can be interpreted as the expected change in percentile rank for an average comparison group student if that student had received the intervention.
For more, please see the WWC Glossary entry for improvement index.
An outcome is the knowledge, skills, and attitudes that are attained as a result of an activity. An outcome measures is an instrument, device, or method that provides data on the outcome.
A finding that is included in the effectiveness rating. Excluded findings may include subgroups and subscales.
The sample on which the analysis was conducted.
The group to which the intervention group is compared, which may include a different intervention, business as usual, or no services.
The timing of the post-intervention outcome measure.
The number of students included in the analysis.
The mean score of students in the intervention group.
The mean score of students in the comparison group.
The WWC considers a finding to be statistically significant if the likelihood that the finding is due to chance alone, rather than a real difference, is less than five percent.
The WWC reviews studies for WWC products, Department of Education grant competitions, and IES performance measures.
The name and version of the document used to guide the review of the study.
The version of the WWC design standards used to guide the review of the study.
The result of the WWC assessment of the study. The rating is based on the strength of evidence of the effectiveness of the intervention. Studies are given a rating of Meets WWC Design Standards without Reservations, Meets WWC Design Standards with Reservations, or >Does Not Meet WWC Design Standards.
A related publication that was reviewed alongside the main study of interest.
Study findings for this report.
Based on the direction, magnitude, and statistical significance of the findings within a domain, the WWC characterizes the findings from a study as one of the following: statistically significant positive effects, substantively important positive effects, indeterminate effects, substantively important negative effects, and statistically significant negative effects. For more, please see the WWC Handbook.
The WWC may review studies for multiple purposes, including different reports and re-reviews using updated standards. Each WWC review of this study is listed in the dropdown. Details on any review may be accessed by making a selection from the drop down list.
Tier 1 Strong indicates strong evidence of effectiveness,
Tier 2 Moderate indicates moderate evidence of effectiveness, and
Tier 3 Promising indicates promising evidence of effectiveness,
as defined in the
non-regulatory guidance for ESSA
and the regulations for ED discretionary grants (EDGAR Part 77).