WWC review of this study

Mobilizing developmental education: The causal effect of mobile app courseware on the college outcomes of developmental education students.

Giani, M. S., & Martin, A. (2021). Educational Evaluation and Policy Analysis, 43(4), 668–687. Retrieved from: https://eric.ed.gov/?id=EJ1317418

  • Randomized Controlled Trial
     examining 
    2,122
     Students
    , grade
    PS

Reviewed: December 2021

No statistically significant positive
findings
Meets WWC standards without reservations
College Degree Attainment outcomes—Indeterminate effects found
Outcome
measure
Comparison Period Sample Intervention
mean
Comparison
mean
Significant? Improvement
    index
Evidence
tier

Received an Associate's degree

Mobile app courseware vs. Business as usual

0 Days

Students who only enrolled in one of the three targeted course sections;
1,720 students

8.30

7.80

No

--
Industry-recognized credential, certificate, or license outcomes—Indeterminate effects found
Outcome
measure
Comparison Period Sample Intervention
mean
Comparison
mean
Significant? Improvement
    index
Evidence
tier

Received a certificate

Mobile app courseware vs. Business as usual

0 Days

Students who only enrolled in one of the three targeted course sections;
1,720 students

13.60

13.60

No

--
Postsecondary Academic Achievement outcomes—Indeterminate effects found
Outcome
measure
Comparison Period Sample Intervention
mean
Comparison
mean
Significant? Improvement
    index
Evidence
tier

Next-Semester GPA

Mobile app courseware vs. Business as usual

1 Semester

Full sample;
1,180 students

1.97

1.87

No

--
Progressing in college outcomes—Indeterminate effects found
Outcome
measure
Comparison Period Sample Intervention
mean
Comparison
mean
Significant? Improvement
    index
Evidence
tier

College persistence - enrolled in higher education for another term

Mobile app courseware vs. Business as usual

1 Semester

Students who only enrolled in one of the three targeted course sections ;
1,720 students

70.30

67.00

No

--
More Outcomes

Credit earned next semester

Mobile app courseware vs. Business as usual

1 Semester

Students who only enrolled in one of the three targeted course sections;
1,720 students

5.62

5.15

No

--
Progressing in Developmental College Courses outcomes—Indeterminate effects found
Outcome
measure
Comparison Period Sample Intervention
mean
Comparison
mean
Significant? Improvement
    index
Evidence
tier

Developmental education passing

Mobile app courseware vs. Business as usual

0 Days

Full sample;
2,122 students

56.60

51.30

No

--
More Outcomes

Developmental education grade

Mobile app courseware vs. Business as usual

0 Days

Full sample;
2,010 students

1.81

1.67

No

--


Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.

Characteristics of study sample as reported by study author.


  • Female: 64%
    Male: 36%

  • Urban
    • B
    • A
    • C
    • D
    • E
    • F
    • G
    • I
    • H
    • J
    • K
    • L
    • P
    • M
    • N
    • O
    • Q
    • R
    • S
    • V
    • U
    • T
    • W
    • X
    • Z
    • Y
    • a
    • h
    • i
    • b
    • d
    • e
    • f
    • c
    • g
    • j
    • k
    • l
    • m
    • n
    • o
    • p
    • q
    • r
    • s
    • t
    • u
    • x
    • w
    • y

    Louisiana
  • Race
    Black
    40%
    Other or unknown
    14%
    White
    46%
  • Ethnicity
    Hispanic    
    6%
    Not Hispanic or Latino    
    94%

Setting

The study was conducted at the Bossier Parish Community College (BPCC) and Northwestern State University (NSU) in the mid-sized community of Shreveport, a metropolitan area in northwest Louisiana.

Study sample

Almost two-thirds (64%) percent of the study sample were female and 65% received Pell grants. The following percentages describe the population students at BPCC: 46% White non-Hispanic, 40% Black, and 8% for all other racial/ethnic groups; 6% of the students were Hispanic/Latinx. However, the authors noted that more than 50% of students reported “other” for their race/ethnicity.

Intervention Group

The Open Campus™ apps consist of free and fully online modules of developmental education content to assist undergraduates in brushing up on core academic topics before taking developmental education placement exams. Undergraduates enrolled in developmental education courses used the Open Campus™ app from spring 2017 through spring 2019 as a supplemental instructional resource in many classrooms. After downloading the app and creating an account, students were presented the option to enroll in English 99 (Developmental Writing), Math 98 (Beginning Algebra I), or Math 99 (Beginning Algebra II). After enrolling, students completed a 50-item quiz within the app that diagnoses topic areas for which they needed the greatest assistance and were shown the percentage of correct answers followed by instructions on which app modules to focus on. Each module consisted of three to five-minute video lectures where instructors explained the concepts and a 10-item quiz with additional questions related to the specific topic area. For the module quizzes, students were shown each question, whether their answer was correct or incorrect, and for each incorrect answer, they were pointed to the specific module to review to learn more about that topic. Students could attempt the quizzes as many times as needed, but only their highest grade was kept in their profile. The mobile apps contained supplemental features to increase engagement, such as digital badges based on student progress within the modules, student head-to-head quiz challenges with a leader board, and email nudges to inactive students in the app for a prolonged period. The email notifications are an opt-in feature, and students can unsubscribe from the notification.

Comparison Group

Students in the comparison condition were enrolled in similar developmental education courses as students in the intervention condition but did not have access to the Open Campus™ mobile application. However, these students did have access to all other campus resources available to students in the developmental education track.

Support for implementation

Developmental education faculty members were offered a professional development workshop immediately prior to the spring 2017 semester for cohort 1 and fall 2017 semester for cohort 2. The workshop provided instructors with hands-on time using the apps, with additional examples of how they could incorporate the apps into their instruction, with information on which course sections would be assigned to the intervention and comparison groups to allow for additional preparation time if their section was selected for implementation.

 

Your export should download shortly as a zip archive.

This download will include data files for study and findings review data and a data dictionary.

Connect With the WWC

loading
back to top