WWC review of this study

The Open Learning Initiative: Measuring the effectiveness of the OLI statistics course in accelerating student learning.

Lovett, M., Meyer, O., & Thille, C. (2008). Journal of Interactive Media in Education, 14, 1–16. doi:10.5334/2008-14. Retrieved from: https://eric.ed.gov/?id=EJ840810

  • Randomized Controlled Trial
     examining 
    61
     Students
    , grade
    PS

Reviewed: August 2019

At least one finding shows promising evidence of effectiveness
At least one statistically significant positive finding
Meets WWC standards without reservations
Academic achievement outcomes—Statistically significant positive effects found
Outcome
measure
Comparison Period Sample Intervention
mean
Comparison
mean
Significant? Improvement
    index
Evidence
tier

Comprehensive Assessment of Outcomes in a First Statistics course (CAOS)

Open Learning Initiative (OLI) vs. Business as usual

0 Days

Full sample (2007 RCT);
61 students

68.00

53.00

Yes

 
 
21
 


Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.

Characteristics of study sample as reported by study author.


  • Urban
    • B
    • A
    • C
    • D
    • E
    • F
    • G
    • I
    • H
    • J
    • K
    • L
    • P
    • M
    • N
    • O
    • Q
    • R
    • S
    • V
    • U
    • T
    • W
    • X
    • Z
    • Y
    • a
    • h
    • i
    • b
    • d
    • e
    • f
    • c
    • g
    • j
    • k
    • l
    • m
    • n
    • o
    • p
    • q
    • r
    • s
    • t
    • u
    • x
    • w
    • y

    Pennsylvania

Setting

The 2007 RCT study which meets standards without reservations takes place in an introduction to statistics course at Carnegie Mellon University.

Study sample

The study does not provide demographic or other characteristics on the students in the study other than that they are undergraduates at Carnegie Mellon University.

Intervention Group

The intervention condition for the 2007 study involves (a) working with an online learning environment to acquire most of the course content, (b) meeting with an instructor approximately two times a week for 50-minute sessions to ask questions and review more challenging material, and (c) doing all of this at a pace designed to complete the semester’s material in approximately half the time (8 weeks instead of 15). The statistics course is a typical college-level, non-calculus-based introduction to statistics.

Comparison Group

The comparison condition was the traditional teaching of the Introductory Statistics course. The content was the same as that delivered in the intervention condition.

Support for implementation

The technology in the study is the OLI course itself, an interactive online statistics course.

Reviewed: March 2016

At least one finding shows promising evidence of effectiveness
At least one statistically significant positive finding
Meets WWC standards without reservations
Academic achievement outcomes—Statistically significant positive effects found
Outcome
measure
Comparison Period Sample Intervention
mean
Comparison
mean
Significant? Improvement
    index
Evidence
tier

Percentage of correct answers on the Comprehensive Assessment of Outcomes in a first Statistics course (CAOS)

The Open Learning Initiative (OLI) vs. Business as usual

8 Weeks

2007;
61 students

68.00

53.00

Yes

 
 
21
 


Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.

Characteristics of study sample as reported by study author.


  • Urban
    • B
    • A
    • C
    • D
    • E
    • F
    • G
    • I
    • H
    • J
    • K
    • L
    • P
    • M
    • N
    • O
    • Q
    • R
    • S
    • V
    • U
    • T
    • W
    • X
    • Z
    • Y
    • a
    • h
    • i
    • b
    • d
    • e
    • f
    • c
    • g
    • j
    • k
    • l
    • m
    • n
    • o
    • p
    • q
    • r
    • s
    • t
    • u
    • x
    • w
    • y

    Pennsylvania

Setting

The study was conducted in 2007 at Carnegie Mellon University, Pittsburgh, PA.

Study sample

The students were undergraduates at Carnegie Mellon University; no other information about demographics or grade level was provided.

Intervention Group

The intervention group was comprised of students who volunteered to participate in the OLI course and were randomly selected to receive the intervention. The OLI-Statistics course provided instruction through an online learning environment, two weekly group sessions with an instructor to address the most challenging material, and was paced to complete the introductory statistics curriculum in 8 weeks instead of 15. The group sessions were tailored by the instructor using reports of student performance generated by the online system.

Comparison Group

The comparison group volunteered to participate in the intervention but were not selected. Instead, they received treatment as usual in the form of the regular instructor-led Introductory Statistics course at Carnegie Mellon. This course involved three 50-minute lectures per week plus one 50-minute lab session.

Support for implementation

No information was provided about training and support for implementation of the intervention, which is delivered primarily in an online format. The authors did not mention special training or support for the faculty who conducted the in-person meetings.

Reviewed: February 2016

No statistically significant positive
findings
Meets WWC standards without reservations
Academic achievement outcomes—Substantively important positive effects found
Outcome
measure
Comparison Period Sample Intervention
mean
Comparison
mean
Significant? Improvement
    index
Evidence
tier

Comprehensive Assessment of Outcomes in a First Statistics course (CAOS)

The Open Learning Initiative (OLI) vs. Business as usual

2007

2007 study;
61 students

0.73

0.53

No

--


Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.

Characteristics of study sample as reported by study author.


  • Urban
    • B
    • A
    • C
    • D
    • E
    • F
    • G
    • I
    • H
    • J
    • K
    • L
    • P
    • M
    • N
    • O
    • Q
    • R
    • S
    • V
    • U
    • T
    • W
    • X
    • Z
    • Y
    • a
    • h
    • i
    • b
    • d
    • e
    • f
    • c
    • g
    • j
    • k
    • l
    • m
    • n
    • o
    • p
    • q
    • r
    • s
    • t
    • u
    • x
    • w
    • y

    Pennsylvania
 

Your export should download shortly as a zip archive.

This download will include data files for study and findings review data and a data dictionary.

Connect With the WWC

loading
back to top