WWC review of this study

The struggle to pass algebra: Online vs. face-to-face credit recovery for at-risk urban students.

Heppen, J. B., Sorensen, N., Allensworth, E., Walters, K., Rickles, J., Taylor, S. S., & Michelman, V. (2017). Journal of Research on Educational Effectiveness, 10(2), 272-296. Retrieved from: https://eric.ed.gov/?id=EJ1135796

  • Randomized Controlled Trial
     examining 
    12,224
     Students
    , grades
    9-10

Reviewed: September 2017

No statistically significant positive
findings
Meets WWC standards without reservations
Algebra outcomes—Statistically significant negative effects found
Outcome
measure
Comparison Period Sample Intervention
mean
Comparison
mean
Significant? Improvement
    index
ESSA
rating

End-of-Semester Algebra Exam

Online Algebra I credit recovery courses vs. Face-to-Face Algebra credit recovery

0 Days

Full sample;
838 students

272.97

279.62

Yes

-8
 
 
General academic achievement (high school) outcomes—Indeterminate effects found
Outcome
measure
Comparison Period Sample Intervention
mean
Comparison
mean
Significant? Improvement
    index
ESSA
rating

Score on the PLAN Algebra subtest

Online Algebra I credit recovery courses vs. Face-to-Face Algebra credit recovery

0 Days

Full sample;
878 students

5.42

5.27

No

--
General Mathematics Achievement outcomes—Indeterminate effects found
Outcome
measure
Comparison Period Sample Intervention
mean
Comparison
mean
Significant? Improvement
    index
ESSA
rating

PLAN Math

Online Algebra I credit recovery courses vs. Face-to-Face Algebra credit recovery

0 Days

Full sample;
878 students

14.16

13.94

No

--
Progressing in school outcomes—Statistically significant negative effects found
Outcome
measure
Comparison Period Sample Intervention
mean
Comparison
mean
Significant? Improvement
    index
ESSA
rating

Algebra Credit Recovery

Online Algebra I credit recovery courses vs. Face-to-Face Algebra credit recovery

0 Days

Full sample;
1,224 students

66.00

78.00

Yes

-14
 
 
More Outcomes
Show Supplemental Findings

On-Track Indicator

Online Algebra I credit recovery courses vs. Face-to-Face Algebra credit recovery

2 Semesters

Full sample;
1,015 students

28.00

25.00

No

--

Credit in Geometry or higher

Online Algebra I credit recovery courses vs. Face-to-Face Algebra credit recovery

1 Semester

Full sample;
1,120 students

53.00

54.00

No

--

Credit in Geometry or higher

Online Algebra I credit recovery courses vs. Face-to-Face Algebra credit recovery

2 Semesters

Full sample;
1,056 students

47.00

48.00

No

--

Cumulative math credits earned

Online Algebra I credit recovery courses vs. Face-to-Face Algebra credit recovery

2 Semesters

Full sample;
1,015 students

2.39

2.51

No

--

Characteristics of study sample as reported by study author.


  • 47% English language learners

  • 86% Free or reduced price lunch

  • Female: 38%
    Male: 62%
  • Race
    Black
    33%
    Not specified
    2%
    White
    8%
  • Ethnicity
    Hispanic
    57%
    Not Hispanic
    43%

  • Urban
    • B
    • A
    • C
    • D
    • E
    • F
    • G
    • I
    • H
    • J
    • K
    • L
    • P
    • M
    • N
    • O
    • Q
    • R
    • S
    • V
    • U
    • T
    • W
    • X
    • Z
    • Y
    • a
    • h
    • i
    • b
    • d
    • e
    • f
    • c
    • g
    • j
    • k
    • l
    • m
    • n
    • o
    • p
    • q
    • r
    • s
    • t
    • u
    • x
    • w
    • y

    Illinois

Setting

The study was conducted in high schools in the Chicago Public Schools district. Schools that had large numbers of students failing Algebra IB (second-semester Algebra) and were open for summer school were recruited to participate in the study. Seventeen high schools participated in the study. The study took place during 2011 and 2012, with 4 schools participating only in 2011, 2 schools only in 2012, and the remainder of the schools participating in both years.

Study sample

Students participating in the study had failed Algebra IB and generally exhibited poor academic performance in other areas. On average, participating students had failed 4.5 semester courses and had prior math scores that were 0.29 standard deviations below the district average. In the previous year 40% of participating students had been suspended, 5% had moved schools, and the average student had missed 30 school days. Of the total 1,224 student who participated in the study across the two school years, 38 percent were female, 57 percent were Hispanic, 33 percent were African American, 8 percent were White, 2 percent were other races or ethnicities, 86 percent were eligible for free or reduced-price lunch, 12 percent were eligible for special education services, and 47 percent spoke Spanish as their home or native language. In the Algebra I condition 38% of students were female, 12% were special education students, 56% were Latino, and 46% were native Spanish speakers. In the face-to-face condition 37% of students were female, 12% were special education students, 58% were Latino, and 48% were native Spanish speakers.

Intervention Group

The intervention condition is The Aventa Learning (now called FuelEducation) online Algebra I summer course. The online credit recovery course was offered during a three- to four-week summer session and was scheduled for 60 hours. The course took place on site via computers at participating schools and was facilitated by an in-class mentor and taught by an online teacher. The in-class mentors were certified teachers, but were not necessarily licensed to teach mathematics. Aventa hired online teachers who were certified to teach mathematics. The course was organized into five units related to systems of equations, polynomials, quadratics and radicals, rational expressions, and exponentials. Students were encouraged to progress through the units in order, but had flexibility in how they chose to proceed. Students on average completed 2.5 out of the 5 units. Students communicated with the online teachers through back and forth online chats, whiteboard demonstrations, and the learning management system, which often did not require the teacher and student to be online at the same time. Students also had periodic opportunities for real-time communication with teachers and other students using an online platform called Elluminate Live! Students received immediate feedback on practice problems and assessment through the online system.

Comparison Group

The second credit recovery intervention is a face-to-face Algebra I summer course. The face-to-face summer course was a traditional Algebra I credit recovery course. The course was offered during a three- to four-week summer session and was scheduled for 60 hours. Certified math instructors taught each course at the participating schools. The teachers determined the content, sequence, and pacing of the material covered in the face-to-face classes. On average across the face-to-face classes, the study team observed that approximately half of the content focused on second semester Algebra, and half focused on first semester Algebra and pre-Algebra topics. The study team also observed that topics in 28 percent of the face-to-face classes were presented in an order that did not follow a sequence typically seen in a textbook or district-pacing guide. In-person communication occurred between face-to-face instructors and students during class time. Students did not receive immediate feedback on their performance.

Support for implementation

In-class mentors in the online class attended a training session before the summer course began. They received training on the use of the online course, how to monitor student progress, and how to communicate with the teachers of the online classes. The online teachers were given concurrent professional development and were supervised by Aventa.

Reviewed: February 2017

No statistically significant positive
findings
Meets WWC standards without reservations
College readiness outcomes—Statistically significant negative effects found
Outcome
measure
Comparison Period Sample Intervention
mean
Comparison
mean
Significant? Improvement
    index
ESSA
rating

On track for high school graduation (%)

Online Algebra I credit recovery courses vs. Business as usual

2 Years

Full sample;
1,015 students

28.00

25.00

No

--
More Outcomes

Recovered Algebra I credit (%)

Online Algebra I credit recovery courses vs. Business as usual

1 Semester

Full sample;
1,224 students

66.00

78.00

Yes

-14
 
 
General academic achievement (high school) outcomes—Indeterminate effects found
Outcome
measure
Comparison Period Sample Intervention
mean
Comparison
mean
Significant? Improvement
    index
ESSA
rating

Score on the PLAN math test

Online Algebra I credit recovery courses vs. Business as usual

1 Year

Full sample;
878 students

14.16

13.94

No

--
More Outcomes
Show Supplemental Findings

Score on the PLAN Algebra subtest

Online Algebra I credit recovery courses vs. Business as usual

2 Years

Full sample;
878 students

5.42

5.27

No

--

Score on the end-of-course posttest

Online Algebra I credit recovery courses vs. Business as usual

1 Semester

Full sample;
1,224 students

272.97

279.62

No

--
Progressing in school outcomes—Indeterminate effects found
Outcome
measure
Comparison Period Sample Intervention
mean
Comparison
mean
Significant? Improvement
    index
ESSA
rating

Cumulative math credits earned

Online Algebra I credit recovery courses vs. Business as usual

2 Years

Full sample;
1,015 students

2.39

2.51

No

--
More Outcomes
Show Supplemental Findings

Earned course credit in Geometry or higher (%)

Online Algebra I credit recovery courses vs. Business as usual

1 Year

Full sample;
1,120 students

53.00

54.00

No

--

Earned course credit in Geometry or higher (%)

Online Algebra I credit recovery courses vs. Business as usual

2 Years

Full sample;
1,056 students

47.00

48.00

No

--

Characteristics of study sample as reported by study author.


  • 86% Free or reduced price lunch

  • Female: 38%
    Male: 62%
  • Race
    Black
    33%
    Not specified
    59%
    White
    8%
  • Ethnicity
    Hispanic
    57%
    Not Hispanic
    43%

  • Urban
    • B
    • A
    • C
    • D
    • E
    • F
    • G
    • I
    • H
    • J
    • K
    • L
    • P
    • M
    • N
    • O
    • Q
    • R
    • S
    • V
    • U
    • T
    • W
    • X
    • Z
    • Y
    • a
    • h
    • i
    • b
    • d
    • e
    • f
    • c
    • g
    • j
    • k
    • l
    • m
    • n
    • o
    • p
    • q
    • r
    • s
    • t
    • u
    • x
    • w
    • y

    Midwest

Setting

The study took place in 17 Chicago Public Schools that participated in the study during the summers of 2011 and 2012. Schools were recruited to participate in the research because they had summer school programs and large numbers of students who failed Algebra I in the second semester of their freshman year.

Study sample

Over the two summers, 1,224 students participated. There were 613 students assigned to the intervention (online) condition, while 611 students were assigned to the comparison (face-to-face) condition. Eleven schools participated in both 2011 and 2012, four participated in 2011 only, and two schools participated in 2012 only. Each school had at least two credit recovery courses: one online, and the other face-to-face. There were 63 school staff who participated as either teachers and/or mentors. Over the two cohorts, there were 34 face-to-face algebra teachers and 30 in-class mentors. Aventa Learning, the online course provider, selected six online teachers for the study, all of whom were certified to teach mathematics. Students in the intervention condition were 38% female and 56% Latino, while the comparison condition was 37% female and 58% Latino. The full sample (i.e., not disaggregated by condition) was 38% female, 57% Hispanic, 33% African American, 8% White, and 2% other races/ethnicities. In the full sample, 86% of the students were eligible for free or reduced-price lunch, 12% were eligible for special education services, and 47% were native Spanish speakers. The proportion of students who passed Algebra IA (first semester of Algebra) was similar across groups (40% in the intervention group and 41% in the comparison group). Only 5% of students in each condition came from census blocks with concentrated poverty.

Intervention Group

Summer Algebra credit recovery courses designed by Aventa Learning were offered to high school students who had failed Algebra I in the second semester of their freshman year. The intervention condition was an online course that focused on typical second-semester Algebra I (designed as a 60-hour course). This included systems of equations, polynomials, quadratics and radicals, rational expressions, and exponentials. The instructional content of the online version was standardized with clear ordering of topics, but flexible in terms of student pacing (in face-to-face classes, teachers have flexibility of content and sequencing, but pacing is generally uniform for the whole class). The online courses were delivered via computer in computer labs at the high schools. Each course was taught by two instructors: one online teacher and one in-class mentor. The course took place over one or both of two 3- to 4-week summer sessions at each participating school.

Comparison Group

The comparison condition received traditional, face-to-face Algebra I instruction by a certified mathematics instructor. The content tended to include both second semester Algebra I topics, as well as pre-Algebra and first semester Algebra I topics; about 50% of the content was from the second semester Algebra I course, while the remaining 50% was derived from first semester Algebra and pre-Algebra courses. The course took place over one or both of two 3- to 4-week summer sessions at each participating school. Face-to-face courses had one instructor and were delivered in traditional classrooms. The face-to-face courses used teacher-created and published materials, including textbooks. Teachers had flexibility of content and sequencing, but pacing was generally uniform for the whole class. In-class mentors provided feedback and communication on students’ progress.

Support for implementation

The in-class mentors and the face-to-face teachers were paid their regular teaching rates, and the online teachers were paid by the number of students enrolled in the course, which ended up being slightly higher than face-to-face teachers. In-class mentors received training on how to use the online course system, how to monitor student progress, and how to communicate with online teachers. The online teachers received ongoing professional development and support from Aventa. The face-to-face teachers received traditional supports from their schools and district for teaching summer Algebra classes.

In the case of multiple manuscripts that report on one study, the WWC selects one manuscript as the primary citation and lists other manuscripts that describe the study as additional sources.

  • Heppen, J., Allensworth, E., Sorensen, N., Rickles, J., Walters, K., Taylor, S., ... Clements, P. (2016). Getting back on track: Comparing the effects of online and face-to-face credit recovery in Algebra I (Research Brief 1). Retrieved from http://www.air.org/

  • Rickles, J., Heppen, H., Taylor, S., Allensworth, E., Michelman, V., Sorensen, N.,...Clements, P. (2016). Getting back on track: Who need to recover algebra credit after ninth grade? (Research Brief 3). Retrieved from http://www.air.org/

  • Taylor, S., Clements, P., Heppen, J., Rickles, J., Sorensen, N., Walters, K., ... Michelman, V. (2016). Getting back on track: The role of in-person instructional support for students taking online credit recovery (Research Brief 2). Retrieved from http://www.air.org/

 

Your export should download shortly as a zip archive.

This download will include data files for study and findings review data and a data dictionary.

Connect With the WWC

loading
back to top