WWC review of this study

Digital messaging to improve college enrollment and success.

Avery, C., Castleman, B. L., Hurwitz, M., Long, B. T., & Page, L. C. (2021). Economics of Education Review, 84. https://doi.org/10.1016/j.econedurev.2021.102170.

  • Randomized Controlled Trial
     examining 
    70,285
     Students
    , grades
    12-PS

Reviewed: October 2021

No statistically significant positive
findings
Meets WWC standards without reservations
Academic achievement outcomes—Indeterminate effects found
Outcome
measure
Comparison Period Sample Intervention
mean
Comparison
mean
Significant? Improvement
    index
Evidence
tier

SAT score

Nudging intervention vs. Business as usual

-6 Months

National sample;
47,061 students

905.93

906.38

No

--
College enrollment outcomes—Statistically significant negative effects found
Outcome
measure
Comparison Period Sample Intervention
mean
Comparison
mean
Significant? Improvement
    index
Evidence
tier

Enrolled in any college

Nudging intervention vs. Business as usual

0 Days

National sample;
70,285 students

59.40

60.70

Yes

-1
 
 
More Outcomes
Show Supplemental Findings

Delayed enrollment in any college

Nudging intervention vs. Business as usual

1 Year

National sample;
70,285 students

54.40

55.50

Yes

-1
 
 
College readiness outcomes—Indeterminate effects found
Outcome
measure
Comparison Period Sample Intervention
mean
Comparison
mean
Significant? Improvement
    index
Evidence
tier

FAFSA submission

Nudging intervention vs. Business as usual

0 Days

National sample;
70,285 students

63.10

63.60

No

--
More Outcomes

Took SAT

Nudging intervention vs. Business as usual

-6 Months

National sample;
70,285 students

66.00

67.50

No

--
Show Supplemental Findings

FAFSA submission

Nudging intervention vs. Business as usual

-6 Months

National sample;
70,285 students

13.20

12.90

No

--

FAFSA submission

Nudging intervention vs. Business as usual

-1 Months

National sample;
70,285 students

59.90

60.30

No

--

FAFSA submission

Nudging intervention vs. Business as usual

-2 Months

National sample;
70,285 students

56.10

56.10

No

--

FAFSA submission

Nudging intervention vs. Business as usual

-3 Months

National sample;
70,285 students

51.40

51.40

No

--

FAFSA submission

Nudging intervention vs. Business as usual

-4 Months

National sample;
70,285 students

46.00

45.90

No

--

FAFSA submission

Nudging intervention vs. Business as usual

-5 Months

National sample;
70,285 students

32.60

32.40

No

--

FAFSA Completion

Nudging intervention vs. Business as usual

-2 Months

National sample;
70,285 students

50.10

50.40

No

--

FAFSA Completion

Nudging intervention vs. Business as usual

-3 Months

National sample;
70,285 students

45.30

45.70

No

--

FAFSA Completion

Nudging intervention vs. Business as usual

-4 Months

National sample;
70,285 students

40.30

40.30

No

--

FAFSA Completion

Nudging intervention vs. Business as usual

-5 Months

National sample;
70,285 students

27.50

27.60

No

--

FAFSA Completion

Nudging intervention vs. Business as usual

-6 Months

National sample;
70,285 students

10.70

10.60

No

--

FAFSA Completion

Nudging intervention vs. Business as usual

0 Months

National sample;
70,285 students

57.30

57.90

No

--

FAFSA Completion

Nudging intervention vs. Business as usual

-1 Months

National sample;
70,285 students

53.90

54.50

No

--


Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.

Characteristics of study sample as reported by study author.


  • Female: 55%
    Male: 45%
  • Race
    Asian
    7%
    Black
    20%
    Native American
    1%
    Other or unknown
    48%
    White
    24%
  • Ethnicity
    Hispanic    
    41%
    Not Hispanic or Latino    
    59%

Setting

There were two studies conducted in this report: a national study and a Texas schools study. The Texas schools study had insufficient data to meet WWC group design standards; therefore, this WWC review only provides a description of the national study. The national study was conducted in 745 schools across 15 states that primarily served low-income students. The intervention was conducted with students in the high school graduating class of 2016 beginning in the spring of students’ junior year of high school through the summer after high school graduation. Students were followed for up to one year after high school graduation.

Study sample

The total sample size in the national study was 70,285 students. However, since the intervention was delivered via text message, it is important to note that only 60,742 students had a valid cell phone number. A total of 36,521 students in 386 schools were randomly assigned to the intervention group and 33,764 students in 359 schools were assigned to the comparison group. In the national study, approximately 24% of students were White, 20% were Black, 7% were Asian, and less than 1% were Native American. About forty percent (41%) of students were Hispanic and 55% were female.

Intervention Group

In the national study, treatment group students received scheduled monthly text messages. The scheduled text messages were designed to cover specific topics including college affordability, search, application, selection, and transition. The text messages were scripted to engage students to respond. The national study employed seven full-time advisors to send personalized replies to students' responses to the scripted text messages. Researchers partnered with uAspire and Signal Vine to deliver the text messages.

Comparison Group

In the national study, students in the comparison group received their usual high school counseling services, as well as scripted text messages once every two months. Comparison group students who responded to the text messages and sought more information received a scripted sequence of automated text responses, pre-programmed on the Signal Vine text messaging platform, rather than personalized replies from an advisor.

Support for implementation

In the national study, the uAspire company recruited and trained seven full-time advisors to work with students.

 

Your export should download shortly as a zip archive.

This download will include data files for study and findings review data and a data dictionary.

Connect With the WWC

loading
back to top