Reducing Achievement Gaps in Academic Writing for Latinos and English Learners in Grades 7-12
Olson, Carol Booth; Matuchniak, Tina; Chung, Huy Q.; Stumpf, Rachel; Farkas, George (2017). Journal of Educational Psychology, v109 n1 p1-21. Retrieved from: https://eric.ed.gov/?id=EJ1125530
-
examining233Students, grades7-12
Pathway to Academic Success (Pathway Project) Intervention Report - English Language Learners
Review Details
Reviewed: October 2021
- Randomized Controlled Trial
- Meets WWC standards with reservations because it is a compromised randomized controlled trial, but the analytic intervention and comparison groups satisfy the baseline equivalence requirement.
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Please see the WWC summary of evidence for Pathway to Academic Success (Pathway Project).
Findings
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|
|---|---|---|---|---|---|---|---|---|---|
|
Researcher-developed Academic Writing Assessment |
Pathway to Academic Success (Pathway Project) vs. Business as usual |
0 Days |
English Learners - Year 2;
|
5.55 |
4.40 |
Yes |
|
|
|
| Show Supplemental Findings | |||||||||
|
Researcher-developed Academic Writing Assessment |
Pathway to Academic Success (Pathway Project) vs. Business as usual |
0 Days |
All students in grade 7 - Year 2;
|
6.77 |
5.18 |
Yes |
|
||
|
Researcher-developed Academic Writing Assessment |
Pathway to Academic Success (Pathway Project) vs. Business as usual |
0 Days |
All students in grade 11 - Year 2;
|
7.84 |
6.15 |
Yes |
|
||
|
Researcher-developed Academic Writing Assessment |
Pathway to Academic Success (Pathway Project) vs. Business as usual |
0 Days |
All Hispanic or Latino students - Year 2;
|
6.72 |
5.30 |
Yes |
|
||
|
Researcher-developed Academic Writing Assessment |
Pathway to Academic Success (Pathway Project) vs. Business as usual |
0 Days |
All female students - Year 1;
|
7.22 |
6.22 |
Yes |
|
||
|
Researcher-developed Academic Writing Assessment |
Pathway to Academic Success (Pathway Project) vs. Business as usual |
0 Days |
Full sample - Year 1;
|
6.90 |
5.88 |
Yes |
|
||
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Sample Characteristics
Characteristics of study sample as reported by study author.
-
100% English language learners -
Female: 47%
Male: 53% -
Urban
-
- B
- A
- C
- D
- E
- F
- G
- I
- H
- J
- K
- L
- P
- M
- N
- O
- Q
- R
- S
- V
- U
- T
- W
- X
- Z
- Y
- a
- h
- i
- b
- d
- e
- f
- c
- g
- j
- k
- l
- m
- n
- o
- p
- q
- r
- s
- t
- u
- v
- x
- w
- y
California
-
Race Asian 6% Black 0% Native American 9% Other or unknown 80% White 4% -
Ethnicity Hispanic 80% Not Hispanic or Latino 20%
Study Details
Setting
The study took place in 16 secondary schools (eight middle and eight high schools) in the Anaheim Union High School District, an urban school district in California, during the 2012–13 and 2013–14 school years.
Study sample
Ninety-five teachers participated in the study, with 49 teachers in the Pathway to Academic Success Project group and 46 teachers in the comparison group. Because three Pathway to Academic Success Project teachers had two classrooms participating in the study, a total of 52 Pathway to Academic Success Project classes and 46 comparison group classes were in the study. The main sample consisted of 233 English learner students in grades 7 to 12. For this sample, all students were English learners, 80% of the students were Hispanic, 9% were Native American, 6% were Asian, 4% were White, and less than 1% were Black. Fifty-three percent of students were male, and 90% of students qualified for free or reduced-price lunch.
Intervention Group
The Pathway to Academic Success Project trains teachers to improve the reading and writing abilities of English learners who have an intermediate level of English proficiency by incorporating cognitive strategies into reading and writing instruction. The cognitive strategies include goal setting, tapping prior knowledge, asking questions, making predictions, articulating and revising understanding of text, and evaluating writing. The Pathway to Academic Success Project training lasted 2 years. During each school year, Pathway to Academic Success Project teachers participated in 46 total hours of training, including six full-day sessions (6 hours each) and five after-school sessions (2 hours each). Developers of the Pathway to Academic Success Project led the training with support from district literacy coaches who were experienced Pathway to Academic Success Project teachers. The first two professional development days focused on introducing teachers to the cognitive strategies toolkit and instructional strategies for teaching students to use the toolkit. Teachers received paper- and computer-based materials as models of curriculum and instruction for teaching students the cognitive strategies within the schools’ English language arts curricula. To reinforce the cognitive strategies toolkit, teachers received wall posters with visuals of the cognitive strategies and students received bookmarks with cognitive strategies sentence starters. In the third and fourth professional development days, teachers focused on analyzing students’ performance on the initial writing assessment to determine strengths and areas for growth and received further training on the implementation of cognitive strategies to enhance interpretive reading and analytical writing. In the fifth and sixth professional development days, teachers analyzed students’ post-test writing, reflected on their growth as writers, and made plans for Year 2. Teachers also engaged in professional learning communities within their school to discuss how to implement lessons from the training in their classrooms. Throughout year 1, teachers received coaching support from a retired teacher with previous experience with the Pathway to Academic Success Project. This teacher conducted three informal classroom observations and provided detailed written feedback to teachers. During year 2, a lead English language arts teacher within each school provided coaching support. In addition to classroom observations, coaches attended professional development sessions with teachers from their assigned school and assisted teachers in integrating Pathway to Academic Success Project strategies into their lessons. Pathway to Academic Success Project teachers also received the same 26 hours of professional development that teachers in the comparison condition received.
Comparison Group
Comparison group teachers participated in business-as-usual professional development and used the district English language arts textbook and novels for teaching. District professional development during years 1 and 2 included one full-day session led by district curriculum specialists on protocols for reviewing district benchmark assessments. In year 2, district curriculum specialists also led professional development on text complexity.
Support for implementation
Trained observers conducted classroom observations of Pathway to Academic Success Project implementation and comparison classrooms, and rated implementation using the Pathway to Academic Success Project Quality Checklist. Authors found that Pathway to Academic Success Project teachers implemented Pathway to Academic Success Project–specific strategies and activities at a higher rate than comparison group teachers at the final observation in the spring of year 2. Pathway to Academic Success Project teachers and comparison group teachers did not differ in their implementation of these strategies and activities in earlier observations. Similarly, authors found differences in the extent to which students demonstrated effective use of Pathway to Academic Success Project strategies in the final observation, but not in earlier observations.
Teaching Secondary Students to Write Effectively
Review Details
Reviewed: June 2017
-
Teaching Secondary Students to Write Effectively Practice Guide
(findings for Secondary Writing)
- Randomized Controlled Trial
- Meets WWC standards with reservations because it is a randomized controlled trial with low attrition, but the randomization was compromised.
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Findings
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Academic Writing Assessment |
Secondary Writing vs. Business as usual |
0 Days |
Full sample;
|
6.87 |
5.88 |
Yes |
|
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Sample Characteristics
Characteristics of study sample as reported by study author.
-
18% English language learners -
Female: 52%
Male: 48% -
Urban
-
- B
- A
- C
- D
- E
- F
- G
- I
- H
- J
- K
- L
- P
- M
- N
- O
- Q
- R
- S
- V
- U
- T
- W
- X
- Z
- Y
- a
- h
- i
- b
- d
- e
- f
- c
- g
- j
- k
- l
- m
- n
- o
- p
- q
- r
- s
- t
- u
- v
- x
- w
- y
California
-
Race Asian 18% Black 3% Native American 2% White 10% -
Ethnicity Hispanic 67% Not Hispanic or Latino 33%
Study Details
Setting
The study takes place in 16 secondary schools in the Anaheim Union School District in Anaheim, California. One or two classrooms from each of the 95 teachers (both intervention and comparison group teachers) was selected to use for the study.
Study sample
The 95 teachers participating in the study had 14.82 years of total teaching experience and 77% percent had earned a master’s degree. Students in the study are in grades 7-12. For both comparison and intervention groups, about half of students are males (51% and 50%). 68% of students in control groups are Hispanic; 13% are White, and 17% are Asian; the corresponding numbers for pathway group are 47%, 30% and 38%. English Learners are a slightly higher percentage of the control group (21% versus 18%), and Initially Fluent English Proficient students are a slightly higher percentage of the Pathway group (9% versus 6%). About 71% students in both control and pathway groups qualified for free or reduced price lunch.
Intervention Group
The intervention includes three core components: "(1) training in the use of cognitive strategies tool kit and curriculum materials, (2) intervention activities focused on the revision of the pre-test on-demand writing assessment into a multiple draft essay, and (3) coaching from a more experienced, veteran teacher previously trained in the Pathway Project on how to integrate a cognitive strategies approach into the existing English language arts curriculum." Teachers in the intervention group participated in 46 hours of training each school year (via six 6-hour released days interspersed throughout the school year and five 2-hour afterschool sessions). The training focused on methods for helping Latinos and mainstreamed ELs to develop the academic literacy necessary to meet the CCSS-ELA, with special emphasis on interpretive reading and analytical writing. These included: literary response and analysis, comprehension and analysis of informational nonfiction texts, and development of clear, coherent, focused essays. Training was led by the developers of the Pathway Project and supported by literacy coaches" with experience with the Pathway Project.
Comparison Group
Comparison teachers conducted business as usual, using the district English language arts textbook and core novels for teaching.
Support for implementation
Throughout the study, teachers in the intervention group received ongoing coaching support. In year 1, this support was provided by retired veteran teacher who had previous exposure to the intervention during an earlier study. In Year 2, "a lead English language arts teacher from each school who had been groomed during Year 1 of the training assumed the coaching role. Pathway coaches attended professional development trainings along with the school team whom they were assigned, conducted three informal non-evaluative classroom observations and assisted teachers in integrating interpretive reading and analytical writing instruction using the cognitive strategies approach into the lessons in their textbook."
An indicator of the effect of the intervention, the improvement index can be interpreted as the expected change in percentile rank for an average comparison group student if that student had received the intervention.
For more, please see the WWC Glossary entry for improvement index.
An outcome is the knowledge, skills, and attitudes that are attained as a result of an activity. An outcome measures is an instrument, device, or method that provides data on the outcome.
A finding that is included in the effectiveness rating. Excluded findings may include subgroups and subscales.
The sample on which the analysis was conducted.
The group to which the intervention group is compared, which may include a different intervention, business as usual, or no services.
The timing of the post-intervention outcome measure.
The number of students included in the analysis.
The mean score of students in the intervention group.
The mean score of students in the comparison group.
The WWC considers a finding to be statistically significant if the likelihood that the finding is due to chance alone, rather than a real difference, is less than five percent.
The WWC reviews studies for WWC products, Department of Education grant competitions, and IES performance measures.
The name and version of the document used to guide the review of the study.
The version of the WWC design standards used to guide the review of the study.
The result of the WWC assessment of the study. The rating is based on the strength of evidence of the effectiveness of the intervention. Studies are given a rating of Meets WWC Design Standards without Reservations, Meets WWC Design Standards with Reservations, or >Does Not Meet WWC Design Standards.
A related publication that was reviewed alongside the main study of interest.
Study findings for this report.
Based on the direction, magnitude, and statistical significance of the findings within a domain, the WWC characterizes the findings from a study as one of the following: statistically significant positive effects, substantively important positive effects, indeterminate effects, substantively important negative effects, and statistically significant negative effects. For more, please see the WWC Handbook.
The WWC may review studies for multiple purposes, including different reports and re-reviews using updated standards. Each WWC review of this study is listed in the dropdown. Details on any review may be accessed by making a selection from the drop down list.
Tier 1 Strong indicates strong evidence of effectiveness,
Tier 2 Moderate indicates moderate evidence of effectiveness, and
Tier 3 Promising indicates promising evidence of effectiveness,
as defined in the
non-regulatory guidance for ESSA
and the regulations for ED discretionary grants (EDGAR Part 77).