Effectiveness of "Enhanced Units": A Report of a Randomized Experiment in California and Virginia. Research Report
Jaciw, Andrew P.; Zacamy, Jenna; D'Apice, Hannah; Lin, Li; Kwong, Connie; Schellinger, Adam M. (2019). Empirical Education Inc. Retrieved from: https://eric.ed.gov/?id=ED603235
-
examining627Students, grades9-12
Department-funded evaluation
Review Details
Reviewed: February 2022
- Department-funded evaluation (findings for Enhanced Units (EU))
- Randomized Controlled Trial
- Meets WWC standards without reservations because it is a cluster randomized controlled trial with low cluster-level attrition and individual-level non-response.
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Findings
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
End of unit Biology Assessment (researcher-developed) |
Enhanced Units (EU) vs. Business as usual |
0 Months |
Full sample (grades 9-12);
|
71.00 |
70.77 |
No |
-- |
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Biology and U.S. History Combined |
Enhanced Units (EU) vs. Business as usual |
0 Months |
Full sample (grades 9-12);
|
65.65 |
62.66 |
No |
-- |
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
End of unit US History Assessment (researcher-developed) |
Enhanced Units (EU) vs. Business as usual |
0 Months |
Full sample (grade 11);
|
56.51 |
49.39 |
Yes |
-- |
|
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Sample Characteristics
Characteristics of study sample as reported by study author.
-
8% English language learners -
Rural, Suburban, Urban
-
- B
- A
- C
- D
- E
- F
- G
- I
- H
- J
- K
- L
- P
- M
- N
- O
- Q
- R
- S
- V
- U
- T
- W
- X
- Z
- Y
- a
- h
- i
- b
- d
- e
- f
- c
- g
- j
- k
- l
- m
- n
- o
- p
- q
- r
- s
- t
- u
- v
- x
- w
- y
California, Virginia
-
Race Asian 17% Black 12% Native American 0% Other or unknown 17% Pacific Islander 1% White 53% -
Ethnicity Hispanic 12%
Study Details
Setting
The program included a pilot study and a field test in two consecutive years. The field test is the focus of this study. The study included five schools, three in Virginia and two in California. The schools according to National Center for Education Statistics (NCES) designations were: 20% rural, 20% town, 40% suburban, and 20% urban. The focus of the study was high school biology and U.S. history classes.
Study sample
The student sample was 52.4% White, 12.2% Black, 12.4% Hispanic, 17.2% Asian, 0.5% Pacific Islander, 0.2% American Indian/Native Alaskan, and 5% multi-racial/unknown. Eight percent were English learners, 11.7% students had Individualized Education Programs (IEPs), and 32.9% were from low socioeconomic status backgrounds.
Intervention Group
Enhanced Units (EU) is an intervention which aims to integrate research-based content enhancement routines with technological enhancements to improve student content learning and higher order reasoning, especially for students with disabilities or other learning challenges. The routines are based on the Strategic Instruction Model (SIM) and consist of unit organizers, question/exploration guides, cause and effect guides, and comparison (compare and contrast) tables. The technology developed during the grant to use with the routines is a Google application called CORGI, which stands for Co-organize your learning. Several student supports are built into the application including embedded videos about how to use the routines, models of expert examples, text to speech and speech to text, and support for vocabulary and translation. The combination of the routines and CORGI technology is therefore called EU. This EU project is the first to combine multiple research-based routines and technology as a means of teaching higher-order reasoning to secondary students.
Comparison Group
Students in the comparison condition received business as usual instruction from the same teachers that delivered the intervention. Teacher professional development emphasized the importance of not using EU materials in control classrooms.
Support for implementation
No implementation support was described separate from the intervention components, which included curricular materials, professional development, and coaching.
Grant Competition
Review Details
Reviewed: June 2021
- Grant Competition (findings for Enhanced Units (EU))
- Randomized Controlled Trial
- Meets WWC standards without reservations because it is a randomized controlled trial with low attrition.
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Findings
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Biology and U.S. History Combined |
Enhanced Units (EU) vs. Business as usual |
0 Days |
Full sample;
|
65.65 |
62.66 |
No |
-- | |
|
Biology and U.S. History Combined |
Enhanced Units (EU) vs. Business as usual |
0 Days |
Full sample;
|
65.65 |
62.66 |
No |
-- |
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Sample Characteristics
Characteristics of study sample as reported by study author.
-
8% English language learners -
Rural, Suburban, Urban
-
- B
- A
- C
- D
- E
- F
- G
- I
- H
- J
- K
- L
- P
- M
- N
- O
- Q
- R
- S
- V
- U
- T
- W
- X
- Z
- Y
- a
- h
- i
- b
- d
- e
- f
- c
- g
- j
- k
- l
- m
- n
- o
- p
- q
- r
- s
- t
- u
- v
- x
- w
- y
California, Virginia
-
Race Asian 17% Black 12% Native American 0% Other or unknown 18% Pacific Islander 1% White 52% -
Ethnicity Hispanic 12% Not Hispanic or Latino 88%
Study Details
Setting
The study was conducted across three districts in five schools located in urban, suburban, and rural areas. Three schools were in Virginia, and two schools were in California. Thirteen teachers who taught biology and U.S. history courses participated in the study.
Study sample
Twenty percent of schools were located in rural, town, and city/suburban areas, respectively, and 40% in suburban areas. White students comprised more than half the sample (52%) and Asians comprised 17% of the sample. The composition of Black and Hispanic students were similar (12%). Pacific Islander and American Indians/Native Alaskans comprised 0.5% and 0.2%, respectively. A small percentage of students in the sample reported being multi-racial or did not report their ethnicity (5%).
Intervention Group
Enhanced Units (EU) is a professional development intervention for teachers of secondary U.S. history and biology classes. The goal of EU is to integrate research-based content enhancement routines with technological enhancements to improve student content learning and higher order reasoning. Four routines used in this study were developed to provide ways to graphically highlight critical content, steps to follow in acquiring content, and ways to monitor progress and retention. These routines include (1) unit organizers, (2) question/exploration guides, (3) cause and effect guides, and (4) comparison (compare and contrast) tables. The technology teachers used with the routines is a Google application called Co-organize your learning (CORGI).
Comparison Group
The U.S. history and biology teachers in the control condition delivered instruction as usual.
Support for implementation
Teachers received approximately 3 days of professional development training on how to implement EU with CORGI in whole-class and small-group settings. Teachers also received coaching by request.
Grant Competition
Review Details
Reviewed: June 2019
- Grant Competition (findings for (Not applicable))
- The study is ineligible for review
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Findings
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Sample Characteristics
Study sample characteristics were not reported.An indicator of the effect of the intervention, the improvement index can be interpreted as the expected change in percentile rank for an average comparison group student if that student had received the intervention.
For more, please see the WWC Glossary entry for improvement index.
An outcome is the knowledge, skills, and attitudes that are attained as a result of an activity. An outcome measures is an instrument, device, or method that provides data on the outcome.
A finding that is included in the effectiveness rating. Excluded findings may include subgroups and subscales.
The sample on which the analysis was conducted.
The group to which the intervention group is compared, which may include a different intervention, business as usual, or no services.
The timing of the post-intervention outcome measure.
The number of students included in the analysis.
The mean score of students in the intervention group.
The mean score of students in the comparison group.
The WWC considers a finding to be statistically significant if the likelihood that the finding is due to chance alone, rather than a real difference, is less than five percent.
The WWC reviews studies for WWC products, Department of Education grant competitions, and IES performance measures.
The name and version of the document used to guide the review of the study.
The version of the WWC design standards used to guide the review of the study.
The result of the WWC assessment of the study. The rating is based on the strength of evidence of the effectiveness of the intervention. Studies are given a rating of Meets WWC Design Standards without Reservations, Meets WWC Design Standards with Reservations, or >Does Not Meet WWC Design Standards.
A related publication that was reviewed alongside the main study of interest.
Study findings for this report.
Based on the direction, magnitude, and statistical significance of the findings within a domain, the WWC characterizes the findings from a study as one of the following: statistically significant positive effects, substantively important positive effects, indeterminate effects, substantively important negative effects, and statistically significant negative effects. For more, please see the WWC Handbook.
The WWC may review studies for multiple purposes, including different reports and re-reviews using updated standards. Each WWC review of this study is listed in the dropdown. Details on any review may be accessed by making a selection from the drop down list.
Tier 1 Strong indicates strong evidence of effectiveness,
Tier 2 Moderate indicates moderate evidence of effectiveness, and
Tier 3 Promising indicates promising evidence of effectiveness,
as defined in the
non-regulatory guidance for ESSA
and the regulations for ED discretionary grants (EDGAR Part 77).