WWC review of this study

Three Studies on Student Outcomes in Higher Education

Cannon, R. (2016). The University of Wisconsin-Madison.

  • Randomized Controlled Trial
     examining 
    485
     Students
    , grade
    PS

Reviewed: July 2021

No statistically significant positive
findings
Meets WWC standards without reservations
Academic achievement outcomes—Indeterminate effects found
Outcome
measure
Comparison Period Sample Intervention
mean
Comparison
mean
Significant? Improvement
    index
Evidence
tier

College GPA

Text message service vs. Business as usual

0 Semesters

Full sample;
428 students

3.21

3.20

No

--
Progressing in college outcomes—Indeterminate effects found
Outcome
measure
Comparison Period Sample Intervention
mean
Comparison
mean
Significant? Improvement
    index
Evidence
tier

college enrollment

Text message service vs. Business as usual

0 Semesters

Full sample;
482 students

84.80

86.60

No

--


Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.

Characteristics of study sample as reported by study author.


  • Female: 52%
    Male: 48%

  • Suburban
    • B
    • A
    • C
    • D
    • E
    • F
    • G
    • I
    • H
    • J
    • K
    • L
    • P
    • M
    • N
    • O
    • Q
    • R
    • S
    • V
    • U
    • T
    • W
    • X
    • Z
    • Y
    • a
    • h
    • i
    • b
    • d
    • e
    • f
    • c
    • g
    • j
    • k
    • l
    • m
    • n
    • o
    • p
    • q
    • r
    • s
    • t
    • u
    • x
    • w
    • y

    West
  • Race
    Other or unknown
    69%
    White
    31%

Setting

The study took place at a medium-sized public master's university campus located outside of a major city center on the west coast of the United States.

Study sample

The randomized sample consisted of 485 students with 300 randomly assigned to two treatment groups (150 in each group) and 185 randomly assigned to the control group. The treatment group had the following demographic characteristics: 31.7 percent White, 28.3 percent under-represented minority, 50 percent female, 51.7 percent first-generation, and 42.7 percent Pell-eligible. The comparison group had the following demographic characteristics: 30.8 percent White, 34.1 percent under-represented minority, 54.4 percent female, 46.2 percent first-generation, and 47.3 percent Pell-eligible.

Intervention Group

The text messaging intervention was supplemental to the college’s regular outreach to all students (whether in the intervention or comparison group) regarding deadlines and resources distributed through emails, campus flyers, and social media as well as a limited amount of targeted individual outreach by advisors and orientation leaders. A New England company was the primary developer and deliverer of the intervention, as a third party, in the winter and spring quarters of the 2014-15 academic year. The company started delivering the outreach on February 20, 2015, which was approximately one week in advance of the campus FAFSA priority date. There were two intervention groups. One group received messages with additional social framing and the other group received similar messages without social framing. The final messages received by students in both groups were delivered in spring 2015. The University of Missouri campus support services staff were typically unaware of which messages were sent on which days. In addition, many of the staff who were in direct contact with students were minimally aware or unaware that the intervention was being implemented. The daily text message prompts were delivered to first-year students with a focus on the use of campus supports and key deadlines. Students in the intervention group received an introductory email before the start of the intervention informing them of the service and providing them with an opportunity to opt out before receiving the first text message. Students were given an additional explicit opportunity to opt out as part of the first text message by replying “STOP”, and opting out remained possible throughout the duration of the intervention delivery. Students who did not opt out received brief targeted text messages delivered once per day, Sunday through Friday. The messages varied daily in content, and the exact wording was developed by the third party provider, except for a series of text messages focused on FAFSA renewal that were explicitly prepared by the office of institutional research and the office of financial aid. The messages were worded to remind students of key deadlines, tell them about campus resources, guide them in creating goals to take advantage of those resources, encourage them to make academic progress, solicit updates on their frame of mind, solicit “advice” for other students, provide advice from other students, provide opportunities for other students, and provide opportunities for questions and support. The majority of text messages were meant to be bidirectional, allowing the student to respond to text messages mostly phrased as questions with “yes/no” responses.

Comparison Group

Students in the comparison condition did not receive the text messaging as a supplemental outreach but instead received the college’s regular outreach to all students regarding deadlines and resources distributed through emails, campus flyers, and social media as well as a limited amount of targeted individual outreach by advisors and orientation leaders.

 

Your export should download shortly as a zip archive.

This download will include data files for study and findings review data and a data dictionary.

Connect With the WWC

loading
back to top