The Pathway to Academic Success: Scaling Up a Text-Based Analytical Writing Intervention for Latinos and English Learners in Secondary School
Olson, Carol Booth; Woodworth, Katrina; Arshan, Nicole; Black, Rebecca; Chung, Huy Q.; D'Aoust, Catherine; Dewar, Tim; Friedrich, Linda; Godfrey, Lauren; Land, Robert; Matuchniak, Tina; Scarcella, Robin; Stowell, Laurie (2020). Journal of Educational Psychology, v112 n4 p701-717. Retrieved from: https://eric.ed.gov/?id=EJ1249837
-
examining211Students, grades7-12
Pathway to Academic Success (Pathway Project) Intervention Report - English Language Learners
Review Details
Reviewed: October 2021
- Randomized Controlled Trial
- Meets WWC standards with reservations because it is a cluster randomized controlled trial with high cluster-level attrition, but the analytic intervention and comparison groups satisfy the baseline equivalence requirement.
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Please see the WWC summary of evidence for Pathway to Academic Success (Pathway Project).
Findings
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|
|---|---|---|---|---|---|---|---|---|---|
|
Analytic Writing Continuum for Literary Analysis (AWC-LA) |
Pathway to Academic Success (Pathway Project) vs. Business as usual |
0 Days |
English learners (year 2 continuing students);
|
3.31 |
3.21 |
No |
-- | ||
| Show Supplemental Findings | |||||||||
|
Analytic Writing Continuum for Literary Analysis (AWC-LA) |
Pathway to Academic Success (Pathway Project) vs. Business as usual |
0 Days |
English learners (year 1);
|
3.20 |
2.90 |
Yes |
|
||
|
Analytic Writing Continuum for Literary Analysis (AWC-LA) |
Pathway to Academic Success (Pathway Project) vs. Business as usual |
0 Days |
Non-English learners (year 1);
|
3.40 |
3.20 |
Yes |
|
||
|
Analytic Writing Continuum for Literary Analysis (AWC-LA) |
Pathway to Academic Success (Pathway Project) vs. Business as usual |
0 Days |
Non-English learners (year 2 first timers);
|
3.70 |
3.39 |
Yes |
|
||
|
Analytic Writing Continuum for Literary Analysis (AWC-LA) |
Pathway to Academic Success (Pathway Project) vs. Business as usual |
0 Days |
English learners (year 2 first timers);
|
3.35 |
3.03 |
No |
-- | ||
|
Analytic Writing Continuum for Literary Analysis (AWC-LA) |
Pathway to Academic Success (Pathway Project) vs. Business as usual |
0 Days |
Non-English learners (year 2 continuing students);
|
3.47 |
3.46 |
No |
-- | ||
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Sample Characteristics
Characteristics of study sample as reported by study author.
-
55% English language learners -
Female: 48%
Male: 52% -
Suburban, Urban
-
- B
- A
- C
- D
- E
- F
- G
- I
- H
- J
- K
- L
- P
- M
- N
- O
- Q
- R
- S
- V
- U
- T
- W
- X
- Z
- Y
- a
- h
- i
- b
- d
- e
- f
- c
- g
- j
- k
- l
- m
- n
- o
- p
- q
- r
- s
- t
- u
- v
- x
- w
- y
California
-
Race Asian 6% Other or unknown 93% White 1% -
Ethnicity Hispanic 89% Not Hispanic or Latino 11%
Study Details
Setting
The study took place in 40 schools serving grades 7 to 12 within four public school districts in urban and suburban areas of southern California during the 2014–15 and 2015–16 school years. Participating districts were associated with one of four National Writing Project sites.
Study sample
The study consisted of 230 secondary school English language arts or English language development teachers (113 Pathway to Academic Success Project teachers and 117 comparison group teachers). The main findings are based on 211 students in grades 7 to 12, which the authors describe as including English learners and redesignated English proficient students. Eighty-nine percent of the students were Hispanic, 6% were Asian, 1% were White, and 4% were another race or ethnicity or missing this information. Fifty-two percent of English learner students were male. Just over half of the 211 students (55%) met the definition of English learners for this review, which included current English learners and students who were recently (no earlier than 2 years before the start of the study) reclassified as English proficient.
Intervention Group
The Pathway to Academic Success Project trains teachers to improve the reading and writing abilities of English learners who have an intermediate level of English proficiency by incorporating cognitive strategies into reading and writing instruction. The cognitive strategies include goal setting, tapping prior knowledge, asking questions, making predictions, articulating and revising understanding of text, and evaluating writing. The Pathway to Academic Success Project training lasted 2 years. During each school year, Pathway to Academic Success Project teachers participated in 46 total hours of training, including five full-day sessions (6 hours each) and five after-school sessions (2 hours each). Developers of the Pathway to Academic Success Project led the training with support from district literacy coaches who were experienced Pathway to Academic Success Project teachers. The first two professional development days focused on introducing teachers to the cognitive strategies toolkit and instructional strategies for teaching students to use the toolkit. Teachers received paper- and computer-based materials as models of curriculum and instruction for teaching students the cognitive strategies within the schools’ English language arts curricula. To reinforce the cognitive strategies toolkit, teachers received wall posters with visuals of the cognitive strategies and students received bookmarks with cognitive strategies sentence starters. In the third and fourth professional development days, teachers focused on analyzing students’ performance on the initial writing assessment to determine strengths and areas for growth and received further training on the implementation of cognitive strategies to enhance interpretive reading and analytical writing. In the fifth and sixth professional development days, teachers analyzed students’ post-test writing, reflected on their growth as writers, and made plans for Year 2. National Writing Project (NWP) site directors led the professional development with support from study co-directors, doctoral students, or NWP teachers and consultants. Each school identified a teacher to serve as a coordinator and liaison between the NWP Site Director and the school. Pathway to Academic Success Project teachers also received business-as-usual professional development provided by the school district.
Comparison Group
Comparison group teachers participated in business-as-usual professional development and used the district English language arts textbook and novels for teaching. All comparison group teachers attended a half-day professional development training on Houghton Mifflin Harcourt's Collections textbook series. Several districts also conducted professional development on district benchmark assessments and the new state Smarter Balanced Assessment Consortium test.
Support for implementation
The intervention developer provided support for implementation. Professional development sessions were staggered so that site directors could watch the intervention developer deliver the session to one site before leading that session for their sites. The site directors agreed to implement certain elements of the intervention with fidelity, but had flexibility to adapt other elements to their site. Implementation fidelity was assessed based on teacher participation in professional development, the extent to which the content of the professional development was consistent with the program model, and annual teacher surveys about professional development and instructional practices. Authors found that teacher participation in professional development fell short of implementation targets but the content of the professional development met expectations. Three of the four study sites met the implementation target of at least 90% of teachers attending four of the five full-day professional development sessions, and two sites met the target of 90% of teachers attending at least three of the five after-school sessions. Pathway to Academic Success Project teachers reported receiving more English language arts–focused professional development than comparison group teachers and that professional development had a greater emphasis on cognitive strategies. However, intervention and comparison group teachers reported spending similar amounts of instructional time on analytical essay writing and reading strategies.
Additional Sources
In the case of multiple manuscripts that report on one study, the WWC selects one manuscript as the primary citation and lists other manuscripts that describe the study as additional sources.
-
Woodworth, K., Arshan, N., & Gallagher, H.A. (2017). UC Irvine Writing Project’s Pathway to Academic Success program: An Investing in Innovation (i3) validation grant evaluation. Technical report. Menlo Park, CA: SRI International. https://www.sri.com/wp-content/uploads/pdf/pathway_i3_sri_technical_report_21dec17_final_in_jan.pdf
An indicator of the effect of the intervention, the improvement index can be interpreted as the expected change in percentile rank for an average comparison group student if that student had received the intervention.
For more, please see the WWC Glossary entry for improvement index.
An outcome is the knowledge, skills, and attitudes that are attained as a result of an activity. An outcome measures is an instrument, device, or method that provides data on the outcome.
A finding that is included in the effectiveness rating. Excluded findings may include subgroups and subscales.
The sample on which the analysis was conducted.
The group to which the intervention group is compared, which may include a different intervention, business as usual, or no services.
The timing of the post-intervention outcome measure.
The number of students included in the analysis.
The mean score of students in the intervention group.
The mean score of students in the comparison group.
The WWC considers a finding to be statistically significant if the likelihood that the finding is due to chance alone, rather than a real difference, is less than five percent.
The WWC reviews studies for WWC products, Department of Education grant competitions, and IES performance measures.
The name and version of the document used to guide the review of the study.
The version of the WWC design standards used to guide the review of the study.
The result of the WWC assessment of the study. The rating is based on the strength of evidence of the effectiveness of the intervention. Studies are given a rating of Meets WWC Design Standards without Reservations, Meets WWC Design Standards with Reservations, or >Does Not Meet WWC Design Standards.
A related publication that was reviewed alongside the main study of interest.
Study findings for this report.
Based on the direction, magnitude, and statistical significance of the findings within a domain, the WWC characterizes the findings from a study as one of the following: statistically significant positive effects, substantively important positive effects, indeterminate effects, substantively important negative effects, and statistically significant negative effects. For more, please see the WWC Handbook.
The WWC may review studies for multiple purposes, including different reports and re-reviews using updated standards. Each WWC review of this study is listed in the dropdown. Details on any review may be accessed by making a selection from the drop down list.
Tier 1 Strong indicates strong evidence of effectiveness,
Tier 2 Moderate indicates moderate evidence of effectiveness, and
Tier 3 Promising indicates promising evidence of effectiveness,
as defined in the
non-regulatory guidance for ESSA
and the regulations for ED discretionary grants (EDGAR Part 77).