Stay late or start early? Experimental evidence on the benefits of college matriculation support from high schools versus colleges
Castleman, B.L., Owen, L., & Page, L.C. (2015). Economics of Education Review, 47, 168-179. https://doi.org/10.1016/j.econedurev.2015.05.010.
-
examining1,602Students, grades12-PS
Practice Guide
Review Details
Reviewed: October 2023
- Practice Guide (findings for Summer counseling for high school graduates – Castleman et al. (2015))
- Randomized Controlled Trial
- Meets WWC standards without reservations because it is a randomized controlled trial with low attrition.
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Findings
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|
|---|---|---|---|---|---|---|---|---|---|
|
Enrolled in any college |
Summer counseling for high school graduates – Castleman et al. (2015) vs. Business as usual |
0 Days |
Full sample;
|
91.70 |
91.80 |
No |
-- | ||
| Show Supplemental Findings | |||||||||
|
Enrolled in any college |
Summer counseling for high school graduates – Castleman et al. (2015) vs. Business as usual |
0 Days |
College-based postsecondary counseling (University of New Mexico);
|
92.40 |
91.80 |
No |
-- | ||
|
Enrolled in any college |
Summer counseling for high school graduates – Castleman et al. (2015) vs. Business as usual |
0 Days |
High school-based postsecondary counseling (Albuquerque Public Schools);
|
90.90 |
91.80 |
No |
-- | ||
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Sample Characteristics
Characteristics of study sample as reported by study author.
-
1% English language learners -
Female: 59%
Male: 41% -
Urban
-
- B
- A
- C
- D
- E
- F
- G
- I
- H
- J
- K
- L
- P
- M
- N
- O
- Q
- R
- S
- V
- U
- T
- W
- X
- Z
- Y
- a
- h
- i
- b
- d
- e
- f
- c
- g
- j
- k
- l
- m
- n
- o
- p
- q
- r
- s
- t
- u
- v
- x
- w
- y
New Mexico
-
Race Asian 5% Black 85% Other or unknown 8% White 2% -
Ethnicity Hispanic 50% Not Hispanic or Latino 50% -
Eligible for Free and Reduced Price Lunch Free or reduced price lunch (FRPL) 29% No FRPL 71%
Study Details
Setting
The study took place with graduates of an unknown number of high schools in Albuquerque, New Mexico.
Study sample
The researchers randomly assigned 1,074 students to the intervention group and 528 students to the comparison group. All 1,602 students who were randomly assigned to conditions were included in the study. All students in the study were graduates of Albuquerque Public School high schools who had been accepted to the University of New Mexico. Approximately 41% of students were male, 29% were eligible for free or reduced-priced lunch, and 1% were English learners. Eighty-five percent of students were Black, 5% were Asian, 2% were White, and 8% did not report race. Approximately 50% of students were Hispanic or Latino.
Intervention Group
The intervention was designed to support recent high school graduates during the summer between high school and college follow-through on their college plans. Twenty-one school counselors reached out via phone, email, and text message to help students complete summer tasks related to enrolling in college, including finalizing financial aid, evaluating loan options, registering for orientation, and registering for academic placement tests. Counselors supported students on an individual basis. Each counselor was responsible for contacting between 60 and 100 students for 10-20 hours per week over 5-6 weeks during the summer. Eight counselors were based at University of New Mexico, and 13 were based at Albuquerque Public Schools. Among the students assigned to receive the intervention, 52% of students met with a counselor during the summer months.
Comparison Group
Students in the comparison group were not contacted by an intervention-trained counselor. Students may have received other business-as-usual counseling services over the summer.
Support for implementation
Prior to the start of the intervention, the study research team provided counselors with training, which lasted an unknown amount of time, on how to review financial aid award letters and tuition bills and how to access and complete required college paperwork. Counselors were also provided with researcher-developed checklists of what to cover when they met with students. College-based counselors also participated in a day-long, on-campus orientation covering university-specific details on required summer tasks for students and an introduction to campus staff and resources available to assist students. The university also provided a staff liaison to answer questions from the college-based counselors.
Summer Counseling Intervention Report - Transition to College
Review Details
Reviewed: March 2018
- Randomized Controlled Trial
- Meets WWC standards without reservations because it is a randomized controlled trial with low attrition.
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Please see the WWC summary of evidence for Summer Counseling.
Findings
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|
|---|---|---|---|---|---|---|---|---|---|
|
Enrollment in any college in fall semester after graduating |
Summer Counseling vs. Business as usual |
1 Month |
Full sample;
|
91.70 |
91.80 |
No |
-- | ||
| Show Supplemental Findings | |||||||||
|
Enrollment in any college in fall semester after graduating |
Summer Counseling vs. Business as usual |
1 Month |
Male; Hispanic or Latino;
|
93.50 |
84.00 |
Yes |
|
||
|
Enrollment in any college in fall semester after graduating |
Summer Counseling vs. Business as usual |
1 Month |
College-based counseling (UNM);
|
92.40 |
91.80 |
No |
-- | ||
|
Enrollment in any college in fall semester after graduating |
Summer Counseling vs. Business as usual |
1 Month |
High school based counseling (APS);
|
90.90 |
91.80 |
No |
-- | ||
|
Enrollment in any college in fall semester after graduating |
Summer Counseling vs. Business as usual |
1 Month |
Male; Not Hispanic or Latino;
|
89.60 |
91.00 |
No |
-- | ||
|
Enrollment in any college in fall semester after graduating |
Summer Counseling vs. Business as usual |
1 Month |
Female; Hispanic or Latino;
|
91.90 |
93.00 |
No |
-- | ||
|
Enrollment in any college in fall semester after graduating |
Summer Counseling vs. Business as usual |
1 Month |
Female; Not Hispanic or Latino;
|
93.40 |
96.00 |
No |
-- | ||
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Sample Characteristics
Characteristics of study sample as reported by study author.
-
1% English language learners -
Female: 59%
Male: 41% -
- B
- A
- C
- D
- E
- F
- G
- I
- H
- J
- K
- L
- P
- M
- N
- O
- Q
- R
- S
- V
- U
- T
- W
- X
- Z
- Y
- a
- h
- i
- b
- d
- e
- f
- c
- g
- j
- k
- l
- m
- n
- o
- p
- q
- r
- s
- t
- u
- v
- x
- w
- y
New Mexico
-
Race Asian 5% Black 2% White 85% -
Ethnicity Hispanic 50% Not Hispanic or Latino 50%
Study Details
Setting
The study was conducted in Albuquerque, New Mexico. Outreach counseling was implemented in the summer of 2012 and was based in Albuquerque Public Schools (APS) high schools and the University of New Mexico (UNM). Findings are reported separately for each setting.
Study sample
Across the full sample (n=1,602), 40.8% of students were male, 50.1% of students were Hispanic, 2.1% of students were Black, 84.8% of students were White, 4.6% of students were Asian, 28.8% of students were eligible for free or reduced price lunch, and 1.4% of students were English learners. The average high school GPA for the full sample was 3.26.
Intervention Group
The intervention was designed to support recent high school graduates, who were accepted to college, follow-through on their college plans. The researchers hired 21 school counselors, placing 8 at UNM and 13 at APS high schools where the students graduated. Counselors reached out to students via phone, email, and text message to help students complete summer tasks related to enrolling in college (e.g., finalizing financial aid, evaluating loan options, registering for orientation, and registering for academic placement tests). Supports were provided in person and over the phone and coaching contacts were documented in an online log. Overall, counselors had caseloads between 60 and 100 students, and worked 10-20 hours per week for 5-6 weeks.
Comparison Group
Students in the comparison group did not get outreach support from an intervention-trained counselor. These students reflect a 'business as usual' condition, and less than 1% of the comparison group met with a counselor.
Support for implementation
Prior to the start of the intervention, counselors participating in the study received training support from the study research staff which covered how to review financial letters and tuition bills and access to complete the necessary paperwork. College-based counselors also had a day-long training, on-campus orientation covering university-specific details on required summer tasks for students; the university also provided a staff liaison to answer questions from the college-based counselors.
An indicator of the effect of the intervention, the improvement index can be interpreted as the expected change in percentile rank for an average comparison group student if that student had received the intervention.
For more, please see the WWC Glossary entry for improvement index.
An outcome is the knowledge, skills, and attitudes that are attained as a result of an activity. An outcome measures is an instrument, device, or method that provides data on the outcome.
A finding that is included in the effectiveness rating. Excluded findings may include subgroups and subscales.
The sample on which the analysis was conducted.
The group to which the intervention group is compared, which may include a different intervention, business as usual, or no services.
The timing of the post-intervention outcome measure.
The number of students included in the analysis.
The mean score of students in the intervention group.
The mean score of students in the comparison group.
The WWC considers a finding to be statistically significant if the likelihood that the finding is due to chance alone, rather than a real difference, is less than five percent.
The WWC reviews studies for WWC products, Department of Education grant competitions, and IES performance measures.
The name and version of the document used to guide the review of the study.
The version of the WWC design standards used to guide the review of the study.
The result of the WWC assessment of the study. The rating is based on the strength of evidence of the effectiveness of the intervention. Studies are given a rating of Meets WWC Design Standards without Reservations, Meets WWC Design Standards with Reservations, or >Does Not Meet WWC Design Standards.
A related publication that was reviewed alongside the main study of interest.
Study findings for this report.
Based on the direction, magnitude, and statistical significance of the findings within a domain, the WWC characterizes the findings from a study as one of the following: statistically significant positive effects, substantively important positive effects, indeterminate effects, substantively important negative effects, and statistically significant negative effects. For more, please see the WWC Handbook.
The WWC may review studies for multiple purposes, including different reports and re-reviews using updated standards. Each WWC review of this study is listed in the dropdown. Details on any review may be accessed by making a selection from the drop down list.
Tier 1 Strong indicates strong evidence of effectiveness,
Tier 2 Moderate indicates moderate evidence of effectiveness, and
Tier 3 Promising indicates promising evidence of effectiveness,
as defined in the
non-regulatory guidance for ESSA
and the regulations for ED discretionary grants (EDGAR Part 77).