The bottom line on college counseling
Barr, A., & Castleman, B. (2017). Boston, MA: Bottom Line. Retrieved from https://www.bottomline.org/sites/default/files/The%20Bottom%20Line%20on%20College%20Counseling%20RCTPaper_10_2017.pdf.
-
examining2,422Students, grades11-PS
Bottom Line Intervention Report - Transition to College
Review Details
Reviewed: March 2021
- Randomized Controlled Trial
- Meets WWC standards without reservations because it is a randomized controlled trial with low attrition.
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Please see the WWC summary of evidence for Bottom Line.
Findings
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|
|---|---|---|---|---|---|---|---|---|---|
|
Enrolled in a 4-year college |
Bottom Line vs. Business as usual |
0 Months |
Full sample;
|
80.60 |
70.30 |
Yes |
|
|
|
| Show Supplemental Findings | |||||||||
|
Enrolled in any college |
Bottom Line vs. Business as usual |
0 Months |
Full sample;
|
89.70 |
82.70 |
Yes |
|
||
|
Enrolled in a 2-year college |
Bottom Line vs. Business as usual |
0 Months |
Full sample;
|
9.30 |
12.70 |
Yes |
|
||
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|
|---|---|---|---|---|---|---|---|---|---|
|
Continuously enrolled for the three semesters following high school |
Bottom Line vs. Business as usual |
0 Months |
Cohort 1;
|
80.40 |
70.50 |
Yes |
|
|
|
| Show Supplemental Findings | |||||||||
|
Enrolled in a 4-year college for a second year |
Bottom Line vs. Business as usual |
0 Months |
Cohort 1;
|
78.00 |
63.00 |
Yes |
|
||
|
Enrolled in any college for a second year |
Bottom Line vs. Business as usual |
0 Months |
Cohort 1;
|
87.00 |
79.00 |
Yes |
|
||
|
Enrolled in a 2-year college for a second year |
Bottom Line vs. Business as usual |
0 Months |
Cohort 1;
|
9.70 |
15.90 |
Yes |
|
||
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Sample Characteristics
Characteristics of study sample as reported by study author.
-
Female: 70%
Male: 30% -
Urban
-
- B
- A
- C
- D
- E
- F
- G
- I
- H
- J
- K
- L
- P
- M
- N
- O
- Q
- R
- S
- V
- U
- T
- W
- X
- Z
- Y
- a
- h
- i
- b
- d
- e
- f
- c
- g
- j
- k
- l
- m
- n
- o
- p
- q
- r
- s
- t
- u
- v
- x
- w
- y
Massachusetts, New York
-
Race Asian 24% Black 32% Other or unknown 44% -
Ethnicity Hispanic 32% Not Hispanic or Latino 68%
Study Details
Setting
The study was conducted with two cohorts of high school students, representing the graduating classes of 2015 and 2016 in Boston, New York City, and Worcester, Massachusetts. Students who received the intervention were encouraged to attend a Bottom Line target college where they could continue the program; approximately 50 percent of students in the intervention group did so. There are about 30 of these target colleges, and they were described in the study as ones that offered an optimal combination of quality and affordability. These institutions are located in the same geographic region as the study high schools; examples include Boston University, the State University of New York at Albany, and target campuses in the City University of New York and University of Massachusetts systems.
Study sample
The high school students in the sample were from families that earn less than 200 percent of the federal poverty guidelines. Program eligibility required that students have earned at least a 2.5 high school GPA. About 81 percent of students in the sample were first-generation college students; approximately 70 percent were female and 32 percent were Hispanic. Thirty-two percent of students in the sample were Black, 24 percent were Asian, and race was not specified for 44 percent of students.
Intervention Group
The Bottom Line college advising model provides two programs: Bottom Line Access for high school juniors and seniors and Bottom Line Success for students who attend a target college. The study followed students who were randomly assigned to the Bottom Line Access program as they transitioned to the Bottom Line Success program. Advisors interacted with students during high school, on average, 13 times during a 15-month period in which the intervention was delivered, starting with May of the student's junior year of high school through August of the year of graduation. Most meetings involved working on college applications or financial aid. Overall, this entailed 10-15 hours of contact time between advisors and students. After high school graduation, students who chose to attend a Bottom Line target college were matched to a new advisor at their college to continue to receive advising through Bottom Line Success.
Comparison Group
Students in the comparison group had access to existing college advising support services. The study surveyed Cohort 1 students and found that almost all respondents in both groups applied for college (100 percent of intervention students and 99 percent of comparison students) and financial aid (99 percent of intervention students and 97 percent of comparison students). The authors did not administer the survey to students in Cohort 2.
Support for implementation
Bottom Line is a 501(c)(3) privately funded organization that serves low-income and first-generation students. The organization currently serves students in Boston, New York City, Chicago, and Worcester, Massachusetts. https://www.bottomline.org/what-we-do. Financial support for the study was provided by the Michael & Susan Dell Foundation, the Coalition for Evidence-Based Policy, and the Laura and John Arnold Foundation.
Additional Sources
In the case of multiple manuscripts that report on one study, the WWC selects one manuscript as the primary citation and lists other manuscripts that describe the study as additional sources.
-
Castleman, Benjamin L.; Goodman, Joshua. (2015). Intensive College Counseling and the College Enrollment Choices of Low Income Students. Society for Research on Educational Effectiveness.
-
Barr, A., & Castleman, B. (2016). Advising students to and through college: Experimental evidence from the Bottom Line advising program. Boston, MA: Bottom Line. Retrieved from https://www.bottomline.org/sites/default/files/Advising%20Students%20To%20and%20Through%20College_web.pdf
Single Study Review
Review Details
Reviewed: September 2019
- Single Study Review (findings for Bottom Line)
- Randomized Controlled Trial
- Meets WWC standards without reservations because it is a randomized controlled trial with low attrition.
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Findings
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|
|---|---|---|---|---|---|---|---|---|---|
|
Enrolled in four-year institution |
Bottom Line vs. Business as usual |
0 Months |
Full sample;
|
80.60 |
70.30 |
Yes |
|
|
|
| Show Supplemental Findings | |||||||||
|
College enrollment |
Bottom Line vs. Business as usual |
0 Months |
Full sample;
|
89.70 |
82.70 |
Yes |
|
||
|
Enrollment in any 2-year college (%) |
Bottom Line vs. Business as usual |
0 Months |
Full sample;
|
9.30 |
12.70 |
Yes |
|
||
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Sample Characteristics
Characteristics of study sample as reported by study author.
-
Female: 70%
Male: 30% -
Urban
-
- B
- A
- C
- D
- E
- F
- G
- I
- H
- J
- K
- L
- P
- M
- N
- O
- Q
- R
- S
- V
- U
- T
- W
- X
- Z
- Y
- a
- h
- i
- b
- d
- e
- f
- c
- g
- j
- k
- l
- m
- n
- o
- p
- q
- r
- s
- t
- u
- v
- x
- w
- y
Massachusetts, New York
-
Race Asian 23% Black 33% -
Ethnicity Hispanic 32% Not Hispanic or Latino 68%
Study Details
Setting
The study was conducted with two cohorts of high school students (graduating classes of 2015 and 2016) in Boston and Worcester, MA, and New York City. Students who received the intervention were encouraged to attend a Bottom Line Counseling target college where they could continue the program; approximately 50 percent of students in the intervention group did so. There are about 30 of these target colleges and they are described in the study as ones that offer an optimal combination of quality and affordability. These institutions operate within the geographic region; examples include Boston University, the State University of New York at Albany, and target campuses in the City University of New York and University of Massachusetts systems.
Study sample
The high school students in the sample are from families that make less than 200 percent of the federal poverty guidelines. Program eligibility required that students have earned at least a 2.5 GPA. In terms of demographic information, about 80% of the sample were first generation college students; approximately 70% were female, 33% Black, 32% Hispanic, and 23% were Asian.
Intervention Group
The Bottom Line (BL) college advising model provides two programs: BL Access for high school juniors and seniors, and BL Success, which provides advising for students who attend a target college. The study followed students randomly assigned to the BL Access program as they transitioned to the BL Success program. Advising is offered through counselors, who interacted with students during high school an average of 13 times during the 15-month period in which the intervention was delivered. The BL Access services started in May of high school junior year and proceed through August after graduation. Most meetings involved working on college applications or financial aid. After high school graduation, students who chose to attend a BL-target institution were matched to a new advisor at their college. They continued to receive advising through BL Success. Overall, BL advising entailed 10-15 hours of contact time between counselors and students.
Comparison Group
Students in the comparison group had access to existing college advising support services. The study surveyed Cohort 1 comparison students and found that most respondents applied for college and financial aid.
Support for implementation
Bottom Line is a 501(c)(3) privately funded organization that serves low income and first-generation students. Advisors spend their first year learning the organization's curriculum, building relationships with students and staff, serving their caseload, and mastering advisor competencies. Returning advisors take on additional responsibilities. The organization currently serves students in Boston, New York City, Worcester, MA and Chicago, IL. For more information, see Bottom Line's website: https://www.bottomline.org/what-we-do
An indicator of the effect of the intervention, the improvement index can be interpreted as the expected change in percentile rank for an average comparison group student if that student had received the intervention.
For more, please see the WWC Glossary entry for improvement index.
An outcome is the knowledge, skills, and attitudes that are attained as a result of an activity. An outcome measures is an instrument, device, or method that provides data on the outcome.
A finding that is included in the effectiveness rating. Excluded findings may include subgroups and subscales.
The sample on which the analysis was conducted.
The group to which the intervention group is compared, which may include a different intervention, business as usual, or no services.
The timing of the post-intervention outcome measure.
The number of students included in the analysis.
The mean score of students in the intervention group.
The mean score of students in the comparison group.
The WWC considers a finding to be statistically significant if the likelihood that the finding is due to chance alone, rather than a real difference, is less than five percent.
The WWC reviews studies for WWC products, Department of Education grant competitions, and IES performance measures.
The name and version of the document used to guide the review of the study.
The version of the WWC design standards used to guide the review of the study.
The result of the WWC assessment of the study. The rating is based on the strength of evidence of the effectiveness of the intervention. Studies are given a rating of Meets WWC Design Standards without Reservations, Meets WWC Design Standards with Reservations, or >Does Not Meet WWC Design Standards.
A related publication that was reviewed alongside the main study of interest.
Study findings for this report.
Based on the direction, magnitude, and statistical significance of the findings within a domain, the WWC characterizes the findings from a study as one of the following: statistically significant positive effects, substantively important positive effects, indeterminate effects, substantively important negative effects, and statistically significant negative effects. For more, please see the WWC Handbook.
The WWC may review studies for multiple purposes, including different reports and re-reviews using updated standards. Each WWC review of this study is listed in the dropdown. Details on any review may be accessed by making a selection from the drop down list.
Tier 1 Strong indicates strong evidence of effectiveness,
Tier 2 Moderate indicates moderate evidence of effectiveness, and
Tier 3 Promising indicates promising evidence of effectiveness,
as defined in the
non-regulatory guidance for ESSA
and the regulations for ED discretionary grants (EDGAR Part 77).