WWC review of this study

Reducing achievement gaps in academic writing for Latinos and English learners in grades 7–12.

Olson, C. B., Matuchniak, T., Chung, H. Q., Stumpf, R., & Farkas, G. (2016). Journal of Educational Psychology, v109 n1 p1-21. Retrieved from: https://eric.ed.gov/?id=EJ1125530

  • Randomized Controlled Trial
     examining 
    233
     Students
    , grades
    7-12

Reviewed: October 2021

At least one finding shows moderate evidence of effectiveness
At least one statistically significant positive finding
Meets WWC standards with reservations
Writing quality outcomes—Statistically significant positive effects found
Outcome
measure
Comparison Period Sample Intervention
mean
Comparison
mean
Significant? Improvement
    index
Evidence
tier

Researcher-developed Academic Writing Assessment

Pathway to Academic Success (Pathway Project) vs. Business as usual

0 Days

English Learners - Year 2;
233 students

5.55

4.40

Yes

 
 
28
 
More Outcomes
Show Supplemental Findings

Researcher-developed Academic Writing Assessment

Pathway to Academic Success (Pathway Project) vs. Business as usual

0 Days

All students in grade 7 - Year 2;
323 students

6.77

5.18

Yes

 
 
30

Researcher-developed Academic Writing Assessment

Pathway to Academic Success (Pathway Project) vs. Business as usual

0 Days

All students in grade 11 - Year 2;
202 students

7.84

6.15

Yes

 
 
28

Researcher-developed Academic Writing Assessment

Pathway to Academic Success (Pathway Project) vs. Business as usual

0 Days

All Hispanic or Latino students - Year 2;
786 students

6.72

5.30

Yes

 
 
26

Researcher-developed Academic Writing Assessment

Pathway to Academic Success (Pathway Project) vs. Business as usual

0 Days

All female students - Year 1;
946 students

7.22

6.22

Yes

 
 
19

Researcher-developed Academic Writing Assessment

Pathway to Academic Success (Pathway Project) vs. Business as usual

0 Days

Full sample - Year 1;
1,817 students

6.90

5.88

Yes

 
 
17

Characteristics of study sample as reported by study author.


  • 100% English language learners

  • Female: 47%
    Male: 53%

  • Urban
    • B
    • A
    • C
    • D
    • E
    • F
    • G
    • I
    • H
    • J
    • K
    • L
    • P
    • M
    • N
    • O
    • Q
    • R
    • S
    • V
    • U
    • T
    • W
    • X
    • Z
    • Y
    • a
    • h
    • i
    • b
    • d
    • e
    • f
    • c
    • g
    • j
    • k
    • l
    • m
    • n
    • o
    • p
    • q
    • r
    • s
    • t
    • u
    • x
    • w
    • y

    California
  • Race
    Asian
    6%
    Black
    0%
    Native American
    9%
    Other or unknown
    80%
    White
    4%
  • Ethnicity
    Hispanic    
    80%
    Not Hispanic or Latino    
    20%

Setting

The study took place in 16 secondary schools (eight middle and eight high schools) in the Anaheim Union High School District, an urban school district in California, during the 2012–13 and 2013–14 school years.

Study sample

Ninety-five teachers participated in the study, with 49 teachers in the Pathway to Academic Success Project group and 46 teachers in the comparison group. Because three Pathway to Academic Success Project teachers had two classrooms participating in the study, a total of 52 Pathway to Academic Success Project classes and 46 comparison group classes were in the study. The main sample consisted of 233 English learner students in grades 7 to 12. For this sample, all students were English learners, 80% of the students were Hispanic, 9% were Native American, 6% were Asian, 4% were White, and less than 1% were Black. Fifty-three percent of students were male, and 90% of students qualified for free or reduced-price lunch.

Intervention Group

The Pathway to Academic Success Project trains teachers to improve the reading and writing abilities of English learners who have an intermediate level of English proficiency by incorporating cognitive strategies into reading and writing instruction. The cognitive strategies include goal setting, tapping prior knowledge, asking questions, making predictions, articulating and revising understanding of text, and evaluating writing. The Pathway to Academic Success Project training lasted 2 years. During each school year, Pathway to Academic Success Project teachers participated in 46 total hours of training, including six full-day sessions (6 hours each) and five after-school sessions (2 hours each). Developers of the Pathway to Academic Success Project led the training with support from district literacy coaches who were experienced Pathway to Academic Success Project teachers. The first two professional development days focused on introducing teachers to the cognitive strategies toolkit and instructional strategies for teaching students to use the toolkit. Teachers received paper- and computer-based materials as models of curriculum and instruction for teaching students the cognitive strategies within the schools’ English language arts curricula. To reinforce the cognitive strategies toolkit, teachers received wall posters with visuals of the cognitive strategies and students received bookmarks with cognitive strategies sentence starters. In the third and fourth professional development days, teachers focused on analyzing students’ performance on the initial writing assessment to determine strengths and areas for growth and received further training on the implementation of cognitive strategies to enhance interpretive reading and analytical writing. In the fifth and sixth professional development days, teachers analyzed students’ post-test writing, reflected on their growth as writers, and made plans for Year 2. Teachers also engaged in professional learning communities within their school to discuss how to implement lessons from the training in their classrooms. Throughout year 1, teachers received coaching support from a retired teacher with previous experience with the Pathway to Academic Success Project. This teacher conducted three informal classroom observations and provided detailed written feedback to teachers. During year 2, a lead English language arts teacher within each school provided coaching support. In addition to classroom observations, coaches attended professional development sessions with teachers from their assigned school and assisted teachers in integrating Pathway to Academic Success Project strategies into their lessons. Pathway to Academic Success Project teachers also received the same 26 hours of professional development that teachers in the comparison condition received.

Comparison Group

Comparison group teachers participated in business-as-usual professional development and used the district English language arts textbook and novels for teaching. District professional development during years 1 and 2 included one full-day session led by district curriculum specialists on protocols for reviewing district benchmark assessments. In year 2, district curriculum specialists also led professional development on text complexity.

Support for implementation

Trained observers conducted classroom observations of Pathway to Academic Success Project implementation and comparison classrooms, and rated implementation using the Pathway to Academic Success Project Quality Checklist. Authors found that Pathway to Academic Success Project teachers implemented Pathway to Academic Success Project–specific strategies and activities at a higher rate than comparison group teachers at the final observation in the spring of year 2. Pathway to Academic Success Project teachers and comparison group teachers did not differ in their implementation of these strategies and activities in earlier observations. Similarly, authors found differences in the extent to which students demonstrated effective use of Pathway to Academic Success Project strategies in the final observation, but not in earlier observations.

Reviewed: June 2017

At least one statistically significant positive finding
Meets WWC standards with reservations
Overall writing quality outcomes—Statistically significant positive effects found
Outcome
measure
Comparison Period Sample Intervention
mean
Comparison
mean
Significant? Improvement
    index
Evidence
tier

Academic Writing Assessment

Secondary Writing vs. Business as usual

0 Days

Full sample;
1,817 students

6.87

5.88

Yes

 
 
18

Characteristics of study sample as reported by study author.


  • 18% English language learners

  • Female: 52%
    Male: 48%

  • Urban
    • B
    • A
    • C
    • D
    • E
    • F
    • G
    • I
    • H
    • J
    • K
    • L
    • P
    • M
    • N
    • O
    • Q
    • R
    • S
    • V
    • U
    • T
    • W
    • X
    • Z
    • Y
    • a
    • h
    • i
    • b
    • d
    • e
    • f
    • c
    • g
    • j
    • k
    • l
    • m
    • n
    • o
    • p
    • q
    • r
    • s
    • t
    • u
    • x
    • w
    • y

    California
  • Race
    Asian
    18%
    Black
    3%
    Native American
    2%
    White
    10%
  • Ethnicity
    Hispanic    
    67%
    Not Hispanic or Latino    
    33%

Setting

The study takes place in 16 secondary schools in the Anaheim Union School District in Anaheim, California. One or two classrooms from each of the 95 teachers (both intervention and comparison group teachers) was selected to use for the study.

Study sample

The 95 teachers participating in the study had 14.82 years of total teaching experience and 77% percent had earned a master’s degree. Students in the study are in grades 7-12. For both comparison and intervention groups, about half of students are males (51% and 50%). 68% of students in control groups are Hispanic; 13% are White, and 17% are Asian; the corresponding numbers for pathway group are 47%, 30% and 38%. English Learners are a slightly higher percentage of the control group (21% versus 18%), and Initially Fluent English Proficient students are a slightly higher percentage of the Pathway group (9% versus 6%). About 71% students in both control and pathway groups qualified for free or reduced price lunch.

Intervention Group

The intervention includes three core components: "(1) training in the use of cognitive strategies tool kit and curriculum materials, (2) intervention activities focused on the revision of the pre-test on-demand writing assessment into a multiple draft essay, and (3) coaching from a more experienced, veteran teacher previously trained in the Pathway Project on how to integrate a cognitive strategies approach into the existing English language arts curriculum." Teachers in the intervention group participated in 46 hours of training each school year (via six 6-hour released days interspersed throughout the school year and five 2-hour afterschool sessions). The training focused on methods for helping Latinos and mainstreamed ELs to develop the academic literacy necessary to meet the CCSS-ELA, with special emphasis on interpretive reading and analytical writing. These included: literary response and analysis, comprehension and analysis of informational nonfiction texts, and development of clear, coherent, focused essays. Training was led by the developers of the Pathway Project and supported by literacy coaches" with experience with the Pathway Project.

Comparison Group

Comparison teachers conducted business as usual, using the district English language arts textbook and core novels for teaching.

Support for implementation

Throughout the study, teachers in the intervention group received ongoing coaching support. In year 1, this support was provided by retired veteran teacher who had previous exposure to the intervention during an earlier study. In Year 2, "a lead English language arts teacher from each school who had been groomed during Year 1 of the training assumed the coaching role. Pathway coaches attended professional development trainings along with the school team whom they were assigned, conducted three informal non-evaluative classroom observations and assisted teachers in integrating interpretive reading and analytical writing instruction using the cognitive strategies approach into the lessons in their textbook."

 

Your export should download shortly as a zip archive.

This download will include data files for study and findings review data and a data dictionary.

Connect With the WWC

loading
back to top