WWC review of this study

Can Light-Touch College-Going Interventions Make a Difference? Evidence from a Statewide Experiment in Michigan

Hyman, Joshua (2020). Journal of Policy Analysis and Management, 39(1), 159-190. Retrieved from: https://eric.ed.gov/?id=EJ1238270

  • Randomized Controlled Trial
     examining 
    49,156
     Students
    , grades
    11-PS

Reviewed: August 2021

No statistically significant positive
findings
Meets WWC standards without reservations
College enrollment outcomes—Indeterminate effects found
Outcome
measure
Comparison Period Sample Intervention
mean
Comparison
mean
Significant? Improvement
    index
Evidence
tier

Enroll in any college

College enrollment intervention vs. Business as usual

2 Years

Full sample;
49,156 students

84.40

84.30

No

--
More Outcomes
Show Supplemental Findings

Enroll in any college

College enrollment intervention vs. Business as usual

2 Years

Not specified "Non-White";
7,325 students

85.10

83.80

No

--

Enroll in any college

College enrollment intervention vs. Business as usual

2 Years

Male;
23,792 students

81.80

81.10

No

--

Enrolled in a selective 4-year college

College enrollment intervention vs. Business as usual

2 Years

Full sample;
49,156 students

8.40

8.10

No

--

College enrollment - 4 year college

College enrollment intervention vs. Business as usual

2 Years

Full sample;
49,156 students

67.80

67.50

No

--

College enrollment - 2 year college

College enrollment intervention vs. Business as usual

2 Years

Full sample;
49,156 students

16.60

16.80

No

--

Immediately enroll in college

College enrollment intervention vs. Business as usual

1 Year

Full sample;
49,156 students

82.00

82.00

No

--

Enroll in any college

College enrollment intervention vs. Business as usual

2 Years

Female;
25,364 students

86.80

87.30

No

--
Progressing in college outcomes—Indeterminate effects found
Outcome
measure
Comparison Period Sample Intervention
mean
Comparison
mean
Significant? Improvement
    index
Evidence
tier

College persistence into second year (any college)

College enrollment intervention vs. Business as usual

2 Years

Full sample;
49,156 students

73.90

74.20

No

--
More Outcomes
Show Supplemental Findings

College persistence into second year (any college)

College enrollment intervention vs. Business as usual

2 Years

Free or reduced price lunch;
13,198 students

61.20

60.60

No

--

College persistence into second year (4-year colleges only)

College enrollment intervention vs. Business as usual

2 Years

Full sample;
49,156 students

54.60

54.70

No

--

College persistence into third year (any college)

College enrollment intervention vs. Business as usual

3 Years

Full sample;
49,156 students

65.30

65.40

No

--


Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.

Characteristics of study sample as reported by study author.


  • Female: 52%
    Male: 48%

  • Rural, Suburban, Urban
    • B
    • A
    • C
    • D
    • E
    • F
    • G
    • I
    • H
    • J
    • K
    • L
    • P
    • M
    • N
    • O
    • Q
    • R
    • S
    • V
    • U
    • T
    • W
    • X
    • Z
    • Y
    • a
    • h
    • i
    • b
    • d
    • e
    • f
    • c
    • g
    • j
    • k
    • l
    • m
    • n
    • o
    • p
    • q
    • r
    • s
    • t
    • u
    • x
    • w
    • y

    Michigan
  • Race
    Asian
    5%
    Black
    6%
    Other or unknown
    4%
    White
    85%
  • Ethnicity
    Hispanic    
    3%
    Not Hispanic or Latino    
    97%

Setting

The study was conducted with a sample of 12th grade students from public high schools in Michigan in urban (10 percent), suburban (54 percent), and rural (36 percent) settings.

Study sample

The sample included 49,156 high-achieving high school seniors who were randomly assigned to either receive a letter encouraging them to consider college or no additional information about college. High-achieving is defined by the author as students who scored at or above the statewide median on the American College Testing (ACT) exam in the 11th grade, which was mandatory in Michigan. A little over half of the students in the total sample were female (52 percent). Eighty-five percent of students were White, six percent were Black, five percent were Asian, and three percent were Hispanic. About a quarter of the students were economically disadvantaged based on free/reduced price lunch status (27 percent) and 2 percent were designated as special education students.

Intervention Group

During fall 2014, when the students were in 12th grade, students in the intervention group received a letter encouraging them to apply to college. The letter provided a QR code directing them to an informational website about the college and financial aid application process. Four versions of the letters were sent, each with a bolded phrase emphasizing a different aspect of the college application process: learn how to apply to college; learn which college is right for them; learn how to make college affordable; and a version with all three topics. Approximately 10 percent of students who received the letter navigated to the website. The author then followed up using administrative data to determine college enrollment and persistence outcomes.

Comparison Group

The students in the comparison group received no additional information as part of the study aside from the ACT score report that all students received in the previous spring.

Support for implementation

The letters and the website, once created, did not require further support for implementation.

 

Your export should download shortly as a zip archive.

This download will include data files for study and findings review data and a data dictionary.

Connect With the WWC

loading
back to top