The Effects of Expanding Pell Grant Eligibility for Short Occupational Training Programs: Results from the Experimental Sites Initiative. Evaluation Report. NCEE 2021-001
Thomas, Jaime; Gonzalez, Naihobe; Paxton, Nora; Wiegand, Andrew; Hebbar, Leela (2020). National Center for Education Evaluation and Regional Assistance. Retrieved from: https://eric.ed.gov/?id=ED609406
-
examining2,684Students, gradePS
Single Study Review
Review Details
Reviewed: June 2021
- Single Study Review (findings for Expanded Pell Grant eligibility)
- Randomized Controlled Trial
- Meets WWC standards without reservations because it is a randomized controlled trial with low attrition.
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Findings
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|
|---|---|---|---|---|---|---|---|---|---|
|
Enrolled in any college |
Expanded Pell Grant eligibility vs. Business as usual |
1 Year |
Experiment 1: Short-Term Training;
|
82.70 |
63.10 |
Yes |
|
|
|
|
Enrolled in any college |
Expanded Pell Grant eligibility vs. Business as usual |
8 Months |
Experiment 2: Very Short-Term Training;
|
78.20 |
64.20 |
Yes |
|
|
|
| Show Supplemental Findings | |||||||||
|
Enrolled in Study School |
Expanded Pell Grant eligibility vs. Business as usual |
1 Year |
Experiment 1: Short-Term Training;
|
77.90 |
51.90 |
Yes |
|
||
|
Enrolled in a Certificate Program at Any School |
Expanded Pell Grant eligibility vs. Business as usual |
1 Year |
Experiment 1: Short-Term Training;
|
48.10 |
31.30 |
Yes |
|
||
|
Enrolled in Study School |
Expanded Pell Grant eligibility vs. Business as usual |
8 Months |
Experiment 2: Very Short-Term Training;
|
66.40 |
51.80 |
Yes |
|
||
|
Enrolled in a Certificate Program at Any School |
Expanded Pell Grant eligibility vs. Business as usual |
8 Months |
Experiment 2: Very Short-Term Training;
|
56.50 |
48.30 |
Yes |
|
||
|
Enrolled in An Associate's Degree Program at Any School |
Expanded Pell Grant eligibility vs. Business as usual |
8 Months |
Experiment 2: Very Short-Term Training;
|
3.30 |
2.70 |
No |
-- | ||
|
Enrolled in An Associate's Degree Program at Any School |
Expanded Pell Grant eligibility vs. Business as usual |
1 Year |
Experiment 1: Short-Term Training;
|
4.80 |
5.60 |
No |
-- | ||
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|
|---|---|---|---|---|---|---|---|---|---|
|
Completed a Program at Any School |
Expanded Pell Grant eligibility vs. Business as usual |
30 Months |
Experiment 1: Short-Term Training;
|
54.20 |
39.40 |
Yes |
|
|
|
|
Completed a Program at Any School |
Expanded Pell Grant eligibility vs. Business as usual |
10 Months |
Experiment 2: Very Short-Term Training;
|
64.30 |
54.90 |
Yes |
|
|
|
|
Completed a Program at Any School |
Expanded Pell Grant eligibility vs. Business as usual |
10 Months |
Experiment 2: Very Short-Term Training;
|
64.30 |
54.90 |
Yes |
-- | ||
| Show Supplemental Findings | |||||||||
|
Completed a Certificate program at Any School |
Expanded Pell Grant eligibility vs. Business as usual |
30 Months |
Experiment 1: Short-Term Training;
|
39.10 |
20.00 |
Yes |
|
||
|
Completed a Program at Study School |
Expanded Pell Grant eligibility vs. Business as usual |
30 Months |
Experiment 1: Short-Term Training;
|
52.40 |
35.60 |
Yes |
|
||
|
Completed a Program at Study School |
Expanded Pell Grant eligibility vs. Business as usual |
10 Months |
Experiment 2: Very Short-Term Training;
|
47.00 |
37.70 |
Yes |
|
||
|
Completed a Certificate program at Any School |
Expanded Pell Grant eligibility vs. Business as usual |
10 Months |
Experiment 2: Very Short-Term Training;
|
44.10 |
38.00 |
Yes |
|
||
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Sample Characteristics
Characteristics of study sample as reported by study author.
-
Female: 41%
Male: 60% -
Rural, Suburban, Urban
-
- B
- A
- C
- D
- E
- F
- G
- I
- H
- J
- K
- L
- P
- M
- N
- O
- Q
- R
- S
- V
- U
- T
- W
- X
- Z
- Y
- a
- h
- i
- b
- d
- e
- f
- c
- g
- j
- k
- l
- m
- n
- o
- p
- q
- r
- s
- t
- u
- v
- x
- w
- y
Midwest, Northeast, South, West
Study Details
Setting
The U.S. Department of Education (ED) piloted two experimental expansions to Pell Grant eligibility. Experiment 1 consisted of a Pell Grant offer for short-term occupational training programs (up to 1 year for full-time students and 2 years for part-time students), and Experiment 2 consisted of a Pell Grant offer for very short-term occupational training programs (8 to 15 weeks). A small number of students (78) were interested in and eligible for both experiments. The study took place in 46 postsecondary schools and programs at mostly public, two-year colleges. About half of the postsecondary schools were concentrated in the Southeast region of the country. Thirty-five schools participated in Experiment 1 and 28 schools participated in Experiment 2. Seventeen schools participated in both experiments (which is not included in the review).
Study sample
The study schools identified and randomly assigned eligible students within schools in each of the two experiments between November 2012 and March 2017. For the 35 schools that participated in Experiment 1, there were 414 students in the analysis. All the students had a bachelor’s degree, 64 percent were female, and 93 percent were independent of their parents. On average, they were 36 years of age and had a gross income of $20,670. Almost a quarter (24 percent) were already enrolled in a study school though not in the program for which they hoped to receive an experimental Pell Grant, as required by the study. For the 28 schools that participated in Experiment 2, there were 2,270 students in these analyses. About half (53 percent) had some college education, 36 percent were female, and 85 percent were independent. On average, they were 32 years of age and had a gross income of $22,451. Fourteen percent were already enrolled in a study school.
Intervention Group
Under the authority to waive federal financial aid regulations under the Experimental Sites Initiative of the Higher Education Act, the two experimental pilots waived specific eligibility rules for a limited number of postsecondary schools that volunteered to participate and randomly assigned students to the expanded Pell Grant eligible condition. Experiment 1 allowed income-eligible students with a bachelor’s degree to obtain Pell Grants for short-term occupational training programs that lasted up to one year for students enrolled full-time and up to two years for students enrolled part-time. Experiment 2 allowed income-eligible students to obtain Pell Grants for very short-term programs from eight to fifteen weeks. Only students with annual family incomes of up to about $50,000, who did not already have a bachelor’s degree, and enrolled in programs that last at least a typical semester (8 to 15 weeks) were eligible to receive the grant. Consistent with current federal aid rules, both experiments called for schools to require students to use the experimental Pell Grant for credit-earning programs leading to an educational certificate. Grant amounts were based on the program’s length and the number of credits awarded.
Comparison Group
Students in the comparison group could receive any other financial aid for which they were eligible, as determined by study schools; however, they did qualify for the experimental Pell Grant under expanded eligibility for short or very short-term occupational training.
Support for implementation
No supports for implementation were described.
An indicator of the effect of the intervention, the improvement index can be interpreted as the expected change in percentile rank for an average comparison group student if that student had received the intervention.
For more, please see the WWC Glossary entry for improvement index.
An outcome is the knowledge, skills, and attitudes that are attained as a result of an activity. An outcome measures is an instrument, device, or method that provides data on the outcome.
A finding that is included in the effectiveness rating. Excluded findings may include subgroups and subscales.
The sample on which the analysis was conducted.
The group to which the intervention group is compared, which may include a different intervention, business as usual, or no services.
The timing of the post-intervention outcome measure.
The number of students included in the analysis.
The mean score of students in the intervention group.
The mean score of students in the comparison group.
The WWC considers a finding to be statistically significant if the likelihood that the finding is due to chance alone, rather than a real difference, is less than five percent.
The WWC reviews studies for WWC products, Department of Education grant competitions, and IES performance measures.
The name and version of the document used to guide the review of the study.
The version of the WWC design standards used to guide the review of the study.
The result of the WWC assessment of the study. The rating is based on the strength of evidence of the effectiveness of the intervention. Studies are given a rating of Meets WWC Design Standards without Reservations, Meets WWC Design Standards with Reservations, or >Does Not Meet WWC Design Standards.
A related publication that was reviewed alongside the main study of interest.
Study findings for this report.
Based on the direction, magnitude, and statistical significance of the findings within a domain, the WWC characterizes the findings from a study as one of the following: statistically significant positive effects, substantively important positive effects, indeterminate effects, substantively important negative effects, and statistically significant negative effects. For more, please see the WWC Handbook.
The WWC may review studies for multiple purposes, including different reports and re-reviews using updated standards. Each WWC review of this study is listed in the dropdown. Details on any review may be accessed by making a selection from the drop down list.
Tier 1 Strong indicates strong evidence of effectiveness,
Tier 2 Moderate indicates moderate evidence of effectiveness, and
Tier 3 Promising indicates promising evidence of effectiveness,
as defined in the
non-regulatory guidance for ESSA
and the regulations for ED discretionary grants (EDGAR Part 77).