Castleman, Benjamin L.; Page, Lindsay C.; Schooley, Korynn (2014). Journal of Policy Analysis and Management, v33 n2 p320-344. Retrieved from: https://eric.ed.gov/?id=EJ1027721
Reviewed: March 2018
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Please see the WWC summary of evidence for Summer Counseling.
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|
|---|---|---|---|---|---|---|---|---|---|
|
Enrollment in any college in fall semester after graduating |
Summer Counseling vs. Business as usual |
1 Month |
Full sample;
|
86.00 |
82.70 |
Yes |
|
|
|
| Show Supplemental Findings | |||||||||
|
Enrollment in any college in fall semester after graduating |
Summer Counseling vs. Business as usual |
1 Month |
Pell grant eligibility: EFC=0, Boston;
|
88.60 |
76.30 |
Yes |
|
||
|
Enrollment in any college in fall semester after graduating |
Summer Counseling vs. Business as usual |
1 Month |
Free or reduced price lunch: Fulton;
|
71.90 |
63.40 |
No |
-- | ||
|
Enrollment in any college in fall semester after graduating |
Summer Counseling vs. Business as usual |
1 Month |
Boston;
|
83.00 |
78.40 |
No |
-- | ||
|
Enrollment in any college in fall semester after graduating |
Summer Counseling vs. Business as usual |
1 Month |
Fulton;
|
87.60 |
85.40 |
No |
-- | ||
|
Enrollment in any college in fall semester after graduating |
Summer Counseling vs. Business as usual |
1 Month |
Pell grant eligibility, EFC>0, Boston;
|
85.70 |
83.30 |
No |
-- | ||
|
Enrollment in any college in fall semester after graduating |
Summer Counseling vs. Business as usual |
1 Month |
Free or reduced price lunch: No, Fulton;
|
92.60 |
92.80 |
No |
-- | ||
|
Enrollment in any college in fall semester after graduating |
Summer Counseling vs. Business as usual |
1 Month |
Pell grant eligibility: No, Boston;
|
83.50 |
94.30 |
No |
-- | ||
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|
|---|---|---|---|---|---|---|---|---|---|
|
Continuous first-year enrollment |
Summer Counseling vs. Business as usual |
9 Months |
Full sample;
|
82.40 |
78.50 |
Yes |
|
|
|
|
Continuous enrollment into sophomore year |
Summer Counseling vs. Business as usual |
16 Months |
Full sample;
|
71.30 |
66.30 |
Yes |
|
|
|
| Show Supplemental Findings | |||||||||
|
Continuous first-year enrollment |
Summer Counseling vs. Business as usual |
9 Months |
Pell grant eligibility: EFC=0, Boston;
|
86.50 |
72.60 |
Yes |
|
||
|
Continuous enrollment into sophomore year |
Summer Counseling vs. Business as usual |
16 Months |
Pell grant eligibility: EFC>0, Boston;
|
81.90 |
66.20 |
Yes |
|
||
|
Continuous enrollment into sophomore year |
Summer Counseling vs. Business as usual |
16 Months |
Pell grant eligibility, EFC=0, Boston;
|
77.60 |
64.40 |
Yes |
|
||
|
Continuous first-year enrollment |
Summer Counseling vs. Business as usual |
9 Months |
Pell grant eligibility: EFC > 0, Boston;
|
88.70 |
85.10 |
No |
-- | ||
|
Continuous enrollment into sophomore year |
Summer Counseling vs. Business as usual |
16 Months |
Non free or reduced price lunch: Fulton;
|
83.40 |
80.90 |
No |
-- | ||
|
Continuous first-year enrollment |
Summer Counseling vs. Business as usual |
9 Months |
Free or reduced price lunch: Fulton;
|
62.30 |
59.30 |
No |
-- | ||
|
Continuous first-year enrollment |
Summer Counseling vs. Business as usual |
9 Months |
Not free or reduced price lunch: Fulton;
|
90.20 |
89.40 |
No |
-- | ||
|
Continuous enrollment into sophomore year |
Summer Counseling vs. Business as usual |
16 Months |
Free or reduced price lunch: Fulton;
|
41.40 |
39.20 |
No |
-- | ||
|
Continuous enrollment into sophomore year |
Summer Counseling vs. Business as usual |
16 Months |
Pell grant eligibility: No, Boston;
|
64.90 |
78.90 |
No |
-- | ||
|
Continuous first-year enrollment |
Summer Counseling vs. Business as usual |
9 Months |
Pell grant eligibility: No, Boston;
|
79.70 |
95.70 |
Yes |
|
||
Characteristics of study sample as reported by study author.
| Asian | 10% | |
| Black | 43% |
| Hispanic | 13% | |
| Not Hispanic or Latino | 87% |
This intervention took place in Boston from June 27, 2011 to August 10, 2011. In Fulton, the intervention took place between June 6, 2011 and July 11, 2011. The counseling sessions in Boston took place primarily at the provider's (uAspire) Center for College Affordability in Boston. In Fulton, the intervention mostly took place over the phone.
Ethnic minority students comprised more than 90 percent of the Boston sample (32% Black, 24% Latino, and 20% Asian). Nearly 85% of the Boston sample completed the Free Application for Federal Student Aid (FAFSA). Of those who did complete the FAFSA, 62% had an EFC of zero and another 23% had an EFC that was nonzero, but still within the range of Pell-eligibility. Sixty five percent of the students were female. In Fulton, ethnic minority students comprised 61% of the sample (49% Black, 6% Hispanic, 4% Asian) and 54% of the students were female. The students in the sample were relatively high performing relative to similar high school students in the Atlanta area. Thirty-seven percent of the students qualified for free- or reduced-price lunch.
Boston: Counselors made multiple attempts via phone, email, text, and Facebook to contact each intervention group student to offer support. Upon reaching students, advisors offered each a $25 gift card incentive to attend an in-person meeting. Researchers provided a protocol to the advisors for the outreach and support they were to provide. During the first in-person meeting, counselors completed a college assessment protocol that included the following elements: (1) Counselors reviewed the student’s financial aid award letter and provided guidance based on the student’s level of unmet financial need; (2) counselors briefed the student on the calendar of key summer deadlines at the college the student planned to attend, and helped the student understand and complete paperwork the student had already received from that college; and, (3) Counselors assessed whether the student faced social or emotional barriers to college enrollment in the fall. After the assessment, counselors helped students create a list of tasks they needed to complete in order to start college that fall. Counselors followed up with students individually to check on their progress in completing these tasks. Subsequent to the initial assessment meeting, much of the communication between counselors and students happened via phone, e-mail, and text, though counselors also conducted in-person follow-up meetings with students when they felt it important to do so. Fulton: Counselors made multiple attempts via phone, email, text, and Facebook to contact each intervention group student to offer support. Upon reaching students, advisors offered counseling to the students. The researchers did not supply a protocol to the counselors in the Georgia site but encouraged counselors to follow their existing professional protocols for working with students. Counselors were encouraged to use an intake form that listed numerous tasks required for college enrollment during their initial contact with students. Counselors who met with students in person primarily used the school from which they were working, but the counselors depended on phone conversations to provide most of their support. Counselors logged whether and when they interacted with students (both intervention and comparison). Counselors indicated that many of their interactions with students focused on issues of financial aid. Counselors also reported addressing a variety of informational questions, such as how to access a college’s web portal, how to complete required paperwork, and what the matriculation process entailed. About 35% of the students had any communication with a counselor; approximately 25% of the non-FRL students had contact with a counselor, while nearly 54% of the FRL students had contact with a counselor. Counselors logged whether and when they interacted with students (both intervention and comparison). Counselors indicated that many of their interactions with students focused on issues of financial aid. Counselors also reported addressing a variety of informational questions, such as how to access a college’s web portal, how to complete required paperwork, and what the matriculation process entailed. In Boston, more than 75% of students in the intervention group communicated with an advisor and 52% of students in the intervention group had at least one face-to-face meeting with an advisor.
The comparison group students did not receive outreach though they were assigned to a counselor. In both sites, counselors were instructed not to deny support to any comparison group student who actively sought help.
The study authors provided the uAspire counselors in Boston with a protocol for their outreach activities and supplied the assessment protocol that guided the counselors’ advising. In Fulton, the study authors provided supplemental training for the counselors that focused on the federal and state financial aid application process.
Reviewed: February 2016
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Reviewed: March 2015
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|
|---|---|---|---|---|---|---|---|---|---|
|
Immediate Enrollment |
College Counseling After High School vs. Business as usual |
Posttest |
Atlanta sample;
|
N/A |
N/A |
Yes |
|
|
|
| Show Supplemental Findings | |||||||||
|
Immediate Enrollment |
College Counseling After High School vs. Business as usual |
Posttest |
Atlanta: FRPL;
|
N/A |
N/A |
Yes |
|
||
|
Immediate Enrollment |
College Counseling After High School vs. Business as usual |
Posttest |
Atlanta: Non-FRPL;
|
N/A |
N/A |
No |
-- | ||
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|
|---|---|---|---|---|---|---|---|---|---|
|
Persistence into the sophomore year |
College Counseling After High School vs. Business as usual |
Posttest |
Atlanta sample;
|
N/A |
N/A |
Yes |
|
|
|
| Show Supplemental Findings | |||||||||
|
Persistence into sophomore semester |
College Counseling After High School vs. Business as usual |
Posttest |
Atlanta: FRPL;
|
N/A |
N/A |
No |
-- | ||
|
Persistence into sophomore semester |
College Counseling After High School vs. Business as usual |
Posttest |
Atlanta: Non-FRPL;
|
N/A |
N/A |
No |
-- | ||
|
Persistence into second semester |
College Counseling After High School vs. Business as usual |
Posttest |
Atlanta sample;
|
N/A |
N/A |
No |
-- | ||
|
Persistence into second semester |
College Counseling After High School vs. Business as usual |
Posttest |
Atlanta: FRPL;
|
N/A |
N/A |
No |
-- | ||
|
Persistence into second semester |
College Counseling After High School vs. Business as usual |
Posttest |
Atlanta: Non-FRPL;
|
N/A |
N/A |
No |
-- | ||
Characteristics of study sample as reported by study author.
| Asian | 4% | |
| Black | 49% | |
| Other or unknown | 2% | |
| White | 39% |
| Hispanic | 6% | |
| Not Hispanic or Latino | 94% |
An indicator of the effect of the intervention, the improvement index can be interpreted as the expected change in percentile rank for an average comparison group student if that student had received the intervention.
For more, please see the WWC Glossary entry for improvement index.
An outcome is the knowledge, skills, and attitudes that are attained as a result of an activity. An outcome measures is an instrument, device, or method that provides data on the outcome.
A finding that is included in the effectiveness rating. Excluded findings may include subgroups and subscales.
The sample on which the analysis was conducted.
The group to which the intervention group is compared, which may include a different intervention, business as usual, or no services.
The timing of the post-intervention outcome measure.
The number of students included in the analysis.
The mean score of students in the intervention group.
The mean score of students in the comparison group.
The WWC considers a finding to be statistically significant if the likelihood that the finding is due to chance alone, rather than a real difference, is less than five percent.
The WWC reviews studies for WWC products, Department of Education grant competitions, and IES performance measures.
The name and version of the document used to guide the review of the study.
The version of the WWC design standards used to guide the review of the study.
The result of the WWC assessment of the study. The rating is based on the strength of evidence of the effectiveness of the intervention. Studies are given a rating of Meets WWC Design Standards without Reservations, Meets WWC Design Standards with Reservations, or >Does Not Meet WWC Design Standards.
A related publication that was reviewed alongside the main study of interest.
Study findings for this report.
Based on the direction, magnitude, and statistical significance of the findings within a domain, the WWC characterizes the findings from a study as one of the following: statistically significant positive effects, substantively important positive effects, indeterminate effects, substantively important negative effects, and statistically significant negative effects. For more, please see the WWC Handbook.
The WWC may review studies for multiple purposes, including different reports and re-reviews using updated standards. Each WWC review of this study is listed in the dropdown. Details on any review may be accessed by making a selection from the drop down list.
Tier 1 Strong indicates strong evidence of effectiveness,
Tier 2 Moderate indicates moderate evidence of effectiveness, and
Tier 3 Promising indicates promising evidence of effectiveness,
as defined in the
non-regulatory guidance for ESSA
and the regulations for ED discretionary grants (EDGAR Part 77).
Resources
Newsflash
Contact Us
Facebook
Twitter
YouTube