At least one finding shows promising evidence of effectiveness
At least one statistically significant positive finding
Meets WWC standards with reservations
For:
-
Department-funded evaluation (findings for Learning by Making)
Rating:
-
Meets WWC standards with reservations
because it uses a cluster quasi-experimental design that provides evidence of effects on individuals by satisfying the baseline equivalence requirement for the individuals in the analytic intervention and comparison groups.
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Characteristics of study sample as reported by study author.
-
13%
English language learners
-
Other or unknown: 100%
-
Rural
-
- B
- A
- C
- D
- E
- F
- G
- I
- H
- J
- K
- L
- P
- M
- N
- O
- Q
- R
- S
- V
- U
- T
- W
- X
- Z
- Y
- a
- h
- i
- b
- d
- e
- f
- c
- g
- j
- k
- l
- m
- n
- o
- p
- q
- r
- s
- t
- u
- v
- x
- w
- y
California
-
Race
Asian |
|
1% |
Black |
|
1% |
Native American |
|
14% |
Other or unknown |
|
42% |
White |
|
42% |
-
Ethnicity
Hispanic |
|
41% |
Not Hispanic or Latino |
|
59% |
-
Eligible for Free and Reduced Price Lunch
Free or reduced price lunch (FRPL) |
|
61% |
Other or unknown |
|
39% |
Setting
The study of the LbyM STEM 1 project took place in the 2016-2017 academic year. High school students were recruited from high-need rural high schools in Mendocino County, California. These six high-need rural high schools in Mendocino County primarily serve families with high rates of family poverty and low rates of college readiness.
Study sample
Across participating schools, 41% of students were Hispanic or Latino, 1% were African American, 42% were white, 1% were Asian or Pacific Islander, 14% were Native American, and 1% were two or more races. Sixty-one percent of students were eligible for free or reduced price lunch, and 13% were English learners.
Intervention Group
The Learning by Making (LbyM) curriculum integrates the NGSS Scientific and Engineering Design Practices and CCSS-M Standards for Mathematical Practice into a modified project-based learning approach whereby the use of mathematics is integrated with science content and technology as students do experiments related to real-world problems. During LbyM investigations, students write Logo code to read the sensors and obtain data, building on basic coding to perform individualized experiments, create simulations, and explore models. Students also analyze and interpret data while using mathematics and engaging in computational thinking without the difficulties usually encountered in learning complex programming languages. LbyM is implemented via professional development for teachers and the Learning by Making curriculum, which aims to improve teachers’ instructional competence, students’ engagement, students’ interest and confidence in STEM, and teachers’ instructional practices. The intended results of these changes include greater student high school math and science achievement, increased student college qualification and STEM majors chosen, increased college acceptance and completion, and increased STEM professionals from under-represented and high-need populations.
Comparison Group
Comparison students used business-as-usual mathematics and science curricula. No additional information is provided by the evaluators on the curriculum or professional development that constitutes business as usual, but students were likely exposed to instruction and support services as they had in the past.
Support for implementation
No implementation support was described separate from the intervention components, which provided professional development and the Learning by Making curriculum to teachers.