Setting
The study took place in 10 schools in Maryland’s Montgomery County School District. The
student population of this large suburban district was 43% White, 22% African American, 14%
Asian American, and 20% Hispanic. The study was part of a multiyear research project called
“Scaling up Curriculum for Achievement, Learning, and Equity Project” (SCALE-uP)
Study sample
In this randomized controlled trial, researchers created a sampling frame consisting of five
school profile categories, with approximately seven schools in each category. The sampling
frame was based on achievement and demographic factors. Each school category had a similar
profile determined by: the percentage of students eligible for free and reduced-price meals,
math and reading achievement scores, ethnicity, eligibility for English for Speakers of Other
Languages (ESOL) services, and eligibility for special education (SPED) services. Two schools
were randomly selected from each category to participate in the study. In each category, one
school of the matched pair was then randomly chosen to implement the intervention, and the
other served as the comparison school. The study school sample consisted of five schools
implementing GEMS® The Real Reasons for Seasons and five schools not implementing it.
The analysis is based on two cohorts of seventh-grade students that attended the study
schools during two consecutive school years. Cohorts 1 and 2 consisted of seventh-grade
students in the 2003–04 and 2004–05 school years, respectively. The Cohort 1 analysis
sample included 1,318 seventh-grade students who received GEMS® The Real Reasons for
Seasons and 1,051 seventh-grade students who received the regular science curriculum.
Cohort 2 included 1,287 seventh-grade students who received GEMS® The Real Reasons
for Seasons and 1,121 seventh-grade students who received the regular science curriculum.
Overall and differential attrition rates of students were low for Cohort 1 (9% and 3%, respectively)
and Cohort 2 (13% and 6%, respectively). The study reported student outcomes for the
two cohorts after the completion of the unit; these findings are included in the WWC effectiveness
rating and can be found in Appendix C. Additional findings for Cohort 1 subgroups by
gender, race/ethnicity, eligibility for ESOL services, and eligibility for SPED services are considered
supplemental findings by the WWC and can be found in Appendix D.
Intervention Group
The intervention teachers implemented the eight activities of the GEMS® The Real Reasons
for Seasons curriculum unit over a period of three weeks. Each activity required about 30–90
minutes of class time. The curriculum unit addressed common misconceptions about seasons
and was designed to either validate students’ accurate ideas about seasons or to address
common problems students experienced when learning about seasons. The curriculum came
with a teacher’s guide, student lab materials, and master copies for duplication or electronic
presentation. Montgomery County Public Schools purchased and distributed to teachers all
student lab materials needed for use with the unit. The GEMS® The Real Reasons for Seasons
curriculum was embedded in a larger astronomy unit using the district-approved curriculum.
Comparison Group
Comparison group teachers used regular curriculum materials normally available to Montgomery
County Public Schools’ teachers. The district materials addressed the same instructional
benchmarks as the GEMS® The Real Reasons for Seasons curriculum unit.
Outcome descriptions
Students took a concept assessment test for both the pretest and posttest. For Cohort 1,
the authors used the Reasons for the Seasons Assessment (RSA). For Cohort 2, the authors
used the Causes for the Seasons Assessment (CSA). Although named differently, essentially
the same concept test was used for data analysis for both cohorts of students. For a more
detailed description of this outcome measure, see Appendix B. Study authors also assessed
each student’s personal orientation toward learning using the Science Learning Orientation
and Engagement for Students Questionnaire. This outcome measure is outside the scope of
the Science review protocol and this review.
Support for implementation
The study did not describe any information about training provided to teachers or staff.