Effectiveness of "Enhanced Units": A Report of a Randomized Experiment in California and Virginia. Research Report
Jaciw, Andrew P.; Zacamy, Jenna; D'Apice, Hannah; Lin, Li; Kwong, Connie; Schellinger, Adam M. (2019). Empirical Education Inc. Retrieved from: https://eric.ed.gov/?id=ED603235
-
examining627Students, grades9-12
Grant Competition
Review Details
Reviewed: October 2022
- Grant Competition (findings for Enhanced Units (EU))
- Randomized Controlled Trial
- Meets WWC standards without reservations because it is a cluster randomized controlled trial with low cluster-level attrition and individual-level non-response.
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Findings
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Biology and U.S. History Combined |
Enhanced Units (EU) vs. Business as usual |
0 Days |
Full sample;
|
65.77 |
62.66 |
No |
-- |
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Sample Characteristics
Characteristics of study sample as reported by study author.
-
8% English language learners -
Other or unknown: 100% -
Rural, Suburban, Town, Urban
-
- B
- A
- C
- D
- E
- F
- G
- I
- H
- J
- K
- L
- P
- M
- N
- O
- Q
- R
- S
- V
- U
- T
- W
- X
- Z
- Y
- a
- h
- i
- b
- d
- e
- f
- c
- g
- j
- k
- l
- m
- n
- o
- p
- q
- r
- s
- t
- u
- v
- x
- w
- y
California, Virginia
-
Race Asian 17% Black 12% Native American 0% Other or unknown 13% Pacific Islander 1% Two or more races 5% White 52% -
Ethnicity Hispanic 12% Not Hispanic or Latino 88% -
Eligible for Free and Reduced Price Lunch Free or reduced price lunch (FRPL) 33% No FRPL 67%
Study Details
Setting
The study was conducted in 18 biology classes and 12 U.S. History classes in five high schools in three school districts in Virginia and California during the 2017-2018 school year. The schools were chosen from school districts that used data from the Strategic Instructional Network (SIM). Each of them had previously purchased SIM materials, had teachers trained in the use of some of the SIM routines and strategies; had a certified SIM trainer available; and identified a staff ‘champion’ for the study. One school was in a rural setting, one was in a town, two were in suburban settings, and one was in an urban setting. There were 13 teachers (7 biology and 6 U.S. History). To be eligible, teachers must have been teaching biology in grades 9-12 or U.S. history in grade 11. The school districts selected were chosen based on a convenience sample.
Study sample
School-level data for the five participating schools, drawn from publicly available 2017-2018 NCES data, are reported by the authors. About 8 percent of students were English language learners (based on district data). Approximately 12 percent of students had an IEP (based on district data). One-third (33%) of students were low socioeconomic status. Over half (52%) of students were White, with 17 percent Asian, 12 percent Hispanic, 12 percent Black, 5 percent multiracial or did not respond, 0.5 percent Pacific Islander, and 0.2 percent American Indian/Native Alaskan.
Intervention Group
Enhanced Units is an intervention that aims to integrate research-based content enhancement routines with technological enhancements to improve student content learning and higher order reasoning, especially for students with disabilities or other learning challenges. It combines multiple research-based routines and technology (specifically a Google application called ‘Co-organize your learning’ or CORGI). There are two key components to the intervention: 1) teachers receive sufficient support to implement the intervention, and 2) teachers increase their implementation of, adherence to, and quality of intervention instructional practices. These two components are intended to influence teacher classroom use of Strategic Instruction Model (SIM) instructional practices, which should then increase student collaboration and critical thinking in U.S. History and biology, which would in turn increase achievement on biology and U.S. History assessments, especially among special education students.
Comparison Group
The comparison condition was business as usual.
Support for implementation
Participating teachers were trained in implementing the intervention and received coaching throughout the school year. Professional development included three days of in-person professional development, which were divided into three sessions per school district. Each school district conducted the professional development independent of the other districts. At least 8 hours of coaching from SIM professional developers were providing throughout the year as ongoing coaching. Participating teachers implemented one practice intervention unit and two units that were included as study interventions. The EU professional development was observed in one school district by study researchers. Teachers completed daily implementation logs to record their use of the various EU routines in their daily instruction. Teachers also answered questions about whether routines had been used, intentionally or unintentionally, in the business as usual classrooms.
An indicator of the effect of the intervention, the improvement index can be interpreted as the expected change in percentile rank for an average comparison group student if that student had received the intervention.
For more, please see the WWC Glossary entry for improvement index.
An outcome is the knowledge, skills, and attitudes that are attained as a result of an activity. An outcome measures is an instrument, device, or method that provides data on the outcome.
A finding that is included in the effectiveness rating. Excluded findings may include subgroups and subscales.
The sample on which the analysis was conducted.
The group to which the intervention group is compared, which may include a different intervention, business as usual, or no services.
The timing of the post-intervention outcome measure.
The number of students included in the analysis.
The mean score of students in the intervention group.
The mean score of students in the comparison group.
The WWC considers a finding to be statistically significant if the likelihood that the finding is due to chance alone, rather than a real difference, is less than five percent.
The WWC reviews studies for WWC products, Department of Education grant competitions, and IES performance measures.
The name and version of the document used to guide the review of the study.
The version of the WWC design standards used to guide the review of the study.
The result of the WWC assessment of the study. The rating is based on the strength of evidence of the effectiveness of the intervention. Studies are given a rating of Meets WWC Design Standards without Reservations, Meets WWC Design Standards with Reservations, or >Does Not Meet WWC Design Standards.
A related publication that was reviewed alongside the main study of interest.
Study findings for this report.
Based on the direction, magnitude, and statistical significance of the findings within a domain, the WWC characterizes the findings from a study as one of the following: statistically significant positive effects, substantively important positive effects, indeterminate effects, substantively important negative effects, and statistically significant negative effects. For more, please see the WWC Handbook.
The WWC may review studies for multiple purposes, including different reports and re-reviews using updated standards. Each WWC review of this study is listed in the dropdown. Details on any review may be accessed by making a selection from the drop down list.
Tier 1 Strong indicates strong evidence of effectiveness,
Tier 2 Moderate indicates moderate evidence of effectiveness, and
Tier 3 Promising indicates promising evidence of effectiveness,
as defined in the
non-regulatory guidance for ESSA
and the regulations for ED discretionary grants (EDGAR Part 77).