WWC review of this study

Project COMPASS: Final evaluation report.

Edmunds, J. A., Gicheva, D., Thrift, B., & Hull, M. (2019). U.S. Department of Education, Fund for the Improvement of Postsecondary Education. Retrieved from: https://eric.ed.gov/?id=ED611756

  • Randomized Controlled Trial
     examining 
    2,295
     Students
    , grade
    PS

Reviewed: December 2021

At least one statistically significant positive finding
Meets WWC standards without reservations
Postsecondary Academic Achievement outcomes—Indeterminate effects found
Outcome
measure
Comparison Period Sample Intervention
mean
Comparison
mean
Significant? Improvement
    index
Evidence
tier

Percent completed course with a C or higher

Project COMPASS vs. Business as usual

0 Days

Full sample;
2,295 students

53.42

52.57

No

--
More Outcomes
Show Supplemental Findings

Percent completed course with a D or higher

Project COMPASS vs. Business as usual

0 Days

Incoming academic performance below median;
1,120 students

52.96

46.95

Yes

 
 
6

Percent completed course with a D or higher

Project COMPASS vs. Business as usual

0 Days

Full sample;
2,295 students

59.32

56.64

No

--

Percent completed course with a C or higher

Project COMPASS vs. Business as usual

0 Days

Incoming academic performance below median;
1,120 students

45.39

42.03

No

--
Progressing in college outcomes—Indeterminate effects found
Outcome
measure
Comparison Period Sample Intervention
mean
Comparison
mean
Significant? Improvement
    index
Evidence
tier

College persistence - enrolled in higher education for another term

Project COMPASS vs. Business as usual

0 Days

PSY-150 and BUS-110 students;
1,943 students

72.26

70.60

No

--
More Outcomes
Show Supplemental Findings

Withdraw rate

Project COMPASS vs. Business as usual

0 Days

Incoming academic performance below median;
1,120 students

35.54

46.27

Yes

 
 
11

Percent Drop or Withdraw

Project COMPASS vs. Business as usual

0 Days

Full sample;
2,295 students

31.92

38.54

Yes

 
 
7

College persistence - enrolled in higher education for another year

Project COMPASS vs. Business as usual

1 Year

PSY-150 and BUS-110 students;
1,943 students

80.33

77.10

No

--


Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.

Characteristics of study sample as reported by study author.


  • Female: 60%
    Male: 40%

  • Urban
    • B
    • A
    • C
    • D
    • E
    • F
    • G
    • I
    • H
    • J
    • K
    • L
    • P
    • M
    • N
    • O
    • Q
    • R
    • S
    • V
    • U
    • T
    • W
    • X
    • Z
    • Y
    • a
    • h
    • i
    • b
    • d
    • e
    • f
    • c
    • g
    • j
    • k
    • l
    • m
    • n
    • o
    • p
    • q
    • r
    • s
    • t
    • u
    • x
    • w
    • y

    North Carolina
  • Race
    Black
    31%
    Other or unknown
    69%
  • Ethnicity
    Hispanic    
    10%
    Not Hispanic or Latino    
    90%

Setting

The study was conducted in three courses, Psychology 150 (PSY-150), Business 110 (BUS-110), and Computer and Information Science 110 (CIS-110) at Wake Technical Community College, a large community college located in North Carolina. Within each course, students were assigned to either an intervention or comparison course section (PSY-150: 16 intervention sections and 20 comparison sections; BUS-110: eight intervention sections and nine comparison sections; and CIS-110: six intervention sections and eight comparison sections). The PSC-150 and BUS-110 sections included in the study were from the fall and spring 2017-2018 academic year and the CIS-110 sections were from the fall and spring of the 2018-2019 academic year.

Study sample

The study sample demographics are as follows: 60% of students were female and 31% were Black. One-tenth (10%) of students were Hispanic. About half (48%) were eligible for Pell grants. The average age of students was approximately 26 years old.

Intervention Group

Students in the intervention condition were enrolled in a Project COMPASS section of the course. Project COMPASS uses a series of technology-enhanced strategies including live-streamed student gatherings and live text chats to support increased student-teacher and student-student interactions and improve social, cognitive, and teaching presence in the online setting. The technology tools included use of web conferencing, web messaging with automated features, video presentations, video chat, and desktop sharing. As part of the project, instructors were trained in the use of these technologies and strategies, as well as strategies designed specifically to support minority students. Each course lasted one semester, which was approximately 16 weeks.

Comparison Group

Students in the comparison condition were enrolled the in the same courses, but taught by teachers who had not been trained in the Project COMPASS protocol.

Support for implementation

Project COMPASS instructors were required to complete a 30-hour training program focused on best practices for online teaching, including online course design, instruction, accessibility, and communication. Other support included biweekly lunches to discuss challenges with implementation; topics discussed included texting tools, tips for Blackboard use, and video editing. Also, implementation support was provided by an instructional designer, instructional technologist, and media production assistant. Additional resources included training, video captioning, course formatting, as well as access to a repository of resources such as instructional guides, activities, and assessments.

 

Your export should download shortly as a zip archive.

This download will include data files for study and findings review data and a data dictionary.

Connect With the WWC

loading
back to top