Is Information Enough? The Effect of Information about Education Tax Benefits on Student Outcomes
Bergman, Peter; Denning, Jeffrey T.; Manoli, Dayanand (2019). Journal of Policy Analysis and Management, v38 n3 p706-731. Retrieved from: https://eric.ed.gov/?id=EJ1218020
-
examining1,042,303Students, gradePS
Single Study Review
Review Details
Reviewed: December 2021
- Single Study Review (findings for Tax credit information campaign)
- Randomized Controlled Trial
- Meets WWC standards without reservations because it is a randomized controlled trial with low attrition.
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Findings
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Graduation from 4 year college |
Tax credit information campaign vs. Business as usual |
1 Year |
ReEnroll Sample;
|
18.80 |
18.60 |
No |
-- | |
|
Graduation from 2 year college |
Tax credit information campaign vs. Business as usual |
1 Year |
ReEnroll Sample;
|
14.50 |
14.50 |
No |
-- |
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|
|---|---|---|---|---|---|---|---|---|---|
|
College enrollment in any college |
Tax credit information campaign vs. Business as usual |
2 Months |
ReApply Sample;
|
44.50 |
44.40 |
No |
-- | ||
| Show Supplemental Findings | |||||||||
|
Enrollment in a four-year public university in Texas |
Tax credit information campaign vs. Business as usual |
2 Months |
ReApply Sample;
|
13.10 |
13.00 |
No |
-- | ||
|
Enrollment in a community college in Texas |
Tax credit information campaign vs. Business as usual |
-6 Months |
ReApply Sample;
|
26.40 |
26.40 |
No |
-- | ||
|
Enrollment in a four-year public university in Texas |
Tax credit information campaign vs. Business as usual |
-6 Months |
ReApply Sample;
|
12.90 |
12.90 |
No |
-- | ||
|
Any enrollment in public 2 year college, Texas |
Tax credit information campaign vs. Business as usual |
2 Months |
ReApply Sample;
|
17.20 |
17.30 |
No |
-- | ||
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Filed FAFSA |
Tax credit information campaign vs. Business as usual |
5 Months |
ReEnroll Sample;
|
70.50 |
70.70 |
No |
-- | |
|
Filed FAFSA |
Tax credit information campaign vs. Business as usual |
4 Months |
ReApply Sample;
|
26.80 |
26.90 |
No |
-- |
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|
|---|---|---|---|---|---|---|---|---|---|
|
College re-enrollment in any postsecondary institution |
Tax credit information campaign vs. Business as usual |
5 Months |
ReEnroll Sample;
|
62.70 |
62.70 |
No |
-- | ||
| Show Supplemental Findings | |||||||||
|
College re-enrollment in a community college in Texas |
Tax credit information campaign vs. Business as usual |
5 Months |
ReEnroll Sample;
|
24.60 |
24.80 |
No |
-- | ||
|
College re-enrollment in a 4-year Texas public university |
Tax credit information campaign vs. Business as usual |
5 Months |
ReEnroll Sample;
|
36.40 |
36.20 |
No |
-- | ||
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Sample Characteristics
Characteristics of study sample as reported by study author.
-
Female: 57%
Male: 43% -
Rural, Suburban, Urban
-
- B
- A
- C
- D
- E
- F
- G
- I
- H
- J
- K
- L
- P
- M
- N
- O
- Q
- R
- S
- V
- U
- T
- W
- X
- Z
- Y
- a
- h
- i
- b
- d
- e
- f
- c
- g
- j
- k
- l
- m
- n
- o
- p
- q
- r
- s
- t
- u
- v
- x
- w
- y
Texas
-
Race Asian 2% Black 14% Other or unknown 84% -
Ethnicity Hispanic 39% Not Hispanic or Latino 61%
Study Details
Setting
The intervention was delivered to three types of college applicants in Texas who used the ApplyTexas.org portal: (1) rising high school seniors who applied to college; (2) students enrolled in a Texas college; and (3) students who had previously applied to Texas universities or colleges but who did not enroll.
Study sample
The Enroll sample has the following characteristics: 45% of students were male, 14% were Black, 6% were Asian, and 5% were of another race. About one-fourth of students (39%) were Hispanic. The ReEnroll sample has the following characteristics: 43% of students were male, 13% were Black, 4% were Asian, and 4% were of another race. About one-fourth of students (37%) were Hispanic. The ReApply sample has the following characteristics: 43% of students were male, 15% were Black, 2% were Asian, and 5% were of another race; 41% of students were Hispanic.
Intervention Group
Students in the intervention group received information via e-mail or mailed letters about tax benefits associated with college enrollment. Letters and e-mails came from ApplyTexas, a well-known official application portal. Variants of the treatment changed the content along the following dimensions costs/benefits/neutral; simple/complex/more tax credits; a separate e-mail regarding the Free Application for Federal Student Aid (FAFSA); a FAFSA reminder in the tax e-mail; and a “peer” treatment for the Enroll sample. The Enroll sample received a first tax e-mail on April 1, 2014; a second tax e-mail on July 16, 2014; a letter on June 1, 2014; a separate FAFSA e-mail on February 18, 2014; and the outcome (college enrollment) was measured in fall 2014. The ReEnroll sample received a first tax e-mail on January 17, 2014; a second tax e-mail on March 25, 2014; and the outcome (college enrollment) was measured in fall 2014. The ReApply sample received a first tax e-mail on November 6, 2013; a second tax e-mail on July 16, 2014; and the outcome (college enrollment) was measured in spring 2014 and fall 2014.
Comparison Group
Students in the comparison group did not receive any communication regarding tax benefits for college.
Support for implementation
The researchers did not receive any support for implementation. The researchers partnered with ApplyTexas such that communication came from a well-known official applicant portal.
An indicator of the effect of the intervention, the improvement index can be interpreted as the expected change in percentile rank for an average comparison group student if that student had received the intervention.
For more, please see the WWC Glossary entry for improvement index.
An outcome is the knowledge, skills, and attitudes that are attained as a result of an activity. An outcome measures is an instrument, device, or method that provides data on the outcome.
A finding that is included in the effectiveness rating. Excluded findings may include subgroups and subscales.
The sample on which the analysis was conducted.
The group to which the intervention group is compared, which may include a different intervention, business as usual, or no services.
The timing of the post-intervention outcome measure.
The number of students included in the analysis.
The mean score of students in the intervention group.
The mean score of students in the comparison group.
The WWC considers a finding to be statistically significant if the likelihood that the finding is due to chance alone, rather than a real difference, is less than five percent.
The WWC reviews studies for WWC products, Department of Education grant competitions, and IES performance measures.
The name and version of the document used to guide the review of the study.
The version of the WWC design standards used to guide the review of the study.
The result of the WWC assessment of the study. The rating is based on the strength of evidence of the effectiveness of the intervention. Studies are given a rating of Meets WWC Design Standards without Reservations, Meets WWC Design Standards with Reservations, or >Does Not Meet WWC Design Standards.
A related publication that was reviewed alongside the main study of interest.
Study findings for this report.
Based on the direction, magnitude, and statistical significance of the findings within a domain, the WWC characterizes the findings from a study as one of the following: statistically significant positive effects, substantively important positive effects, indeterminate effects, substantively important negative effects, and statistically significant negative effects. For more, please see the WWC Handbook.
The WWC may review studies for multiple purposes, including different reports and re-reviews using updated standards. Each WWC review of this study is listed in the dropdown. Details on any review may be accessed by making a selection from the drop down list.
Tier 1 Strong indicates strong evidence of effectiveness,
Tier 2 Moderate indicates moderate evidence of effectiveness, and
Tier 3 Promising indicates promising evidence of effectiveness,
as defined in the
non-regulatory guidance for ESSA
and the regulations for ED discretionary grants (EDGAR Part 77).