Does virtual advising increase college enrollment? Evidence from a random-assignment college access field experiment [V-SOURCE Complete vs. business as usual control]
Phillips, M., & Reber, S. (2022). American Economic Journal: Economic Policy, 14(3), 198-234.
-
examining4,087Students, grades11-12
Practice Guide
Review Details
Reviewed: April 2024
- Practice Guide (findings for Virtual Student Outreach for College Enrollment (V-SOURCE) Complete)
- Randomized Controlled Trial
- Meets WWC standards without reservations because it is a randomized controlled trial with low attrition.
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Findings
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|
|---|---|---|---|---|---|---|---|---|---|
|
Accepted to any 4 year college, self-report |
Virtual Student Outreach for College Enrollment (V-SOURCE) Complete vs. Business as usual |
0 Days |
Full sample;
|
69.00 |
67.30 |
No |
-- | ||
|
Enrolled in any college |
Virtual Student Outreach for College Enrollment (V-SOURCE) Complete vs. Business as usual |
3 Months |
Full sample;
|
71.10 |
70.50 |
No |
-- | ||
| Show Supplemental Findings | |||||||||
|
Accepted to any college in University of California system, self-report |
Virtual Student Outreach for College Enrollment (V-SOURCE) Complete vs. Business as usual |
0 Days |
Hispanic or Latino, Spanish speaking at home;
|
31.30 |
26.60 |
Yes |
|
||
|
Enrolled in any college in University of California system |
Virtual Student Outreach for College Enrollment (V-SOURCE) Complete vs. Business as usual |
3 Months |
Hispanic or Latino, Spanish speaking at home;
|
12.70 |
10.40 |
Yes |
|
||
|
Enrolled in any selective 4 year college |
Virtual Student Outreach for College Enrollment (V-SOURCE) Complete vs. Business as usual |
3 Months |
Hispanic or Latino, Spanish speaking at home;
|
11.00 |
9.40 |
No |
-- | ||
|
Accepted to any college in California State University system, self-report |
Virtual Student Outreach for College Enrollment (V-SOURCE) Complete vs. Business as usual |
0 Days |
Full sample;
|
64.20 |
61.60 |
No |
-- | ||
|
Accepted to any 4 year college, self-report |
Virtual Student Outreach for College Enrollment (V-SOURCE) Complete vs. Business as usual |
0 Days |
Hispanic or Latino, Spanish speaking at home;
|
69.10 |
66.20 |
No |
-- | ||
|
Accepted to any selective college, self-report |
Virtual Student Outreach for College Enrollment (V-SOURCE) Complete vs. Business as usual |
0 Days |
Hispanic or Latino, Spanish speaking at home;
|
23.30 |
21.00 |
No |
-- | ||
|
Accepted to any college in California State University system, self-report |
Virtual Student Outreach for College Enrollment (V-SOURCE) Complete vs. Business as usual |
0 Days |
Hispanic or Latino, Spanish speaking at home;
|
65.90 |
62.70 |
No |
-- | ||
|
Enrolled in any college |
Virtual Student Outreach for College Enrollment (V-SOURCE) Complete vs. Business as usual |
3 Months |
Hispanic or Latino, Spanish speaking at home;
|
68.20 |
65.80 |
No |
-- | ||
|
Enrolled in any 4-year college |
Virtual Student Outreach for College Enrollment (V-SOURCE) Complete vs. Business as usual |
3 Months |
Full sample;
|
44.00 |
43.30 |
No |
-- | ||
|
Enrolled in any college in University of California system |
Virtual Student Outreach for College Enrollment (V-SOURCE) Complete vs. Business as usual |
3 Months |
Full sample;
|
13.20 |
12.70 |
No |
-- | ||
|
Accepted to any selective college, self-report |
Virtual Student Outreach for College Enrollment (V-SOURCE) Complete vs. Business as usual |
0 Days |
Full sample;
|
24.20 |
23.40 |
No |
-- | ||
|
Accepted to any college in University of California system, self-report |
Virtual Student Outreach for College Enrollment (V-SOURCE) Complete vs. Business as usual |
0 Days |
Full sample;
|
30.80 |
29.50 |
No |
-- | ||
|
Enrolled in any 4-year college |
Virtual Student Outreach for College Enrollment (V-SOURCE) Complete vs. Business as usual |
3 Months |
Hispanic or Latino, Spanish speaking at home;
|
40.30 |
39.30 |
No |
-- | ||
|
Enrolled in any selective 4 year college |
Virtual Student Outreach for College Enrollment (V-SOURCE) Complete vs. Business as usual |
3 Months |
Full sample;
|
11.80 |
11.70 |
No |
-- | ||
|
Enrolled in any college in California State University system |
Virtual Student Outreach for College Enrollment (V-SOURCE) Complete vs. Business as usual |
3 Months |
Full sample;
|
25.20 |
24.90 |
No |
-- | ||
|
Enrolled in any college in California State University system |
Virtual Student Outreach for College Enrollment (V-SOURCE) Complete vs. Business as usual |
3 Months |
Hispanic or Latino, Spanish speaking at home;
|
24.60 |
26.00 |
No |
-- | ||
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|
|---|---|---|---|---|---|---|---|---|---|
|
Applied to any 4-year college, self report |
Virtual Student Outreach for College Enrollment (V-SOURCE) Complete vs. Business as usual |
0 Days |
Full sample;
|
81.30 |
77.90 |
Yes |
|
|
|
|
Submitted college applications to at least two systems, self-report |
Virtual Student Outreach for College Enrollment (V-SOURCE) Complete vs. Business as usual |
0 Days |
Full sample;
|
54.30 |
48.90 |
Yes |
|
|
|
|
Took SAT/ACT, self-report |
Virtual Student Outreach for College Enrollment (V-SOURCE) Complete vs. Business as usual |
0 Days |
Full sample;
|
85.30 |
82.90 |
No |
-- | ||
|
Completed the FAFSA |
Virtual Student Outreach for College Enrollment (V-SOURCE) Complete vs. Business as usual |
0 Days |
Full sample;
|
80.60 |
78.90 |
No |
-- | ||
|
Sought information about college application process, self-report (researcher-developed scale) |
Virtual Student Outreach for College Enrollment (V-SOURCE) Complete vs. Business as usual |
0 Days |
Full sample;
|
1.70 |
0.00 |
No |
-- | ||
| Show Supplemental Findings | |||||||||
|
Applied to any college in California State University system, self-report |
Virtual Student Outreach for College Enrollment (V-SOURCE) Complete vs. Business as usual |
0 Days |
Hispanic or Latino, Spanish speaking at home;
|
80.60 |
74.10 |
Yes |
|
||
|
Applied to any 4-year college, self report |
Virtual Student Outreach for College Enrollment (V-SOURCE) Complete vs. Business as usual |
0 Days |
Hispanic or Latino, Spanish speaking at home;
|
83.00 |
77.70 |
Yes |
|
||
|
Submitted college applications to at least two systems, self-report |
Virtual Student Outreach for College Enrollment (V-SOURCE) Complete vs. Business as usual |
0 Days |
Hispanic or Latino, Spanish speaking at home;
|
53.80 |
45.80 |
Yes |
|
||
|
Applied to any college in University of California system, self-report |
Virtual Student Outreach for College Enrollment (V-SOURCE) Complete vs. Business as usual |
0 Days |
Hispanic or Latino, Spanish speaking at home;
|
49.30 |
41.80 |
Yes |
|
||
|
Applied to any selective 4-year college, self report |
Virtual Student Outreach for College Enrollment (V-SOURCE) Complete vs. Business as usual |
0 Days |
Hispanic or Latino, Spanish speaking at home;
|
51.30 |
44.80 |
Yes |
|
||
|
Took SAT/ACT, self-report |
Virtual Student Outreach for College Enrollment (V-SOURCE) Complete vs. Business as usual |
0 Days |
Hispanic or Latino, Spanish speaking at home;
|
85.80 |
82.30 |
Yes |
|
||
|
Applied to any college in California State University system, self-report |
Virtual Student Outreach for College Enrollment (V-SOURCE) Complete vs. Business as usual |
0 Days |
Full sample;
|
76.80 |
72.70 |
Yes |
|
||
|
Registered for the SAT/ACT, self-report |
Virtual Student Outreach for College Enrollment (V-SOURCE) Complete vs. Business as usual |
0 Days |
Hispanic or Latino, Spanish speaking at home;
|
86.90 |
84.10 |
No |
-- | ||
|
Registered for the SAT/ACT, self-report |
Virtual Student Outreach for College Enrollment (V-SOURCE) Complete vs. Business as usual |
0 Days |
Full sample;
|
86.60 |
84.20 |
Yes |
|
||
|
Applied to any college in University of California system, self-report |
Virtual Student Outreach for College Enrollment (V-SOURCE) Complete vs. Business as usual |
0 Days |
Full sample;
|
48.90 |
44.50 |
Yes |
|
||
|
Applied to any selective 4-year college, self report |
Virtual Student Outreach for College Enrollment (V-SOURCE) Complete vs. Business as usual |
0 Days |
Full sample;
|
51.20 |
47.60 |
Yes |
|
||
|
Sought information about college application process, self-report (researcher-developed scale) |
Virtual Student Outreach for College Enrollment (V-SOURCE) Complete vs. Business as usual |
0 Days |
Hispanic or Latino, Spanish speaking at home;
|
0.03 |
-0.04 |
Yes |
|
||
|
Completed the FAFSA |
Virtual Student Outreach for College Enrollment (V-SOURCE) Complete vs. Business as usual |
0 Days |
Hispanic or Latino, Spanish speaking at home;
|
85.70 |
88.00 |
No |
-- | ||
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|
|---|---|---|---|---|---|---|---|---|---|
|
Persisting in college - Any college |
Virtual Student Outreach for College Enrollment (V-SOURCE) Complete vs. Business as usual |
15 Months |
Full sample;
|
65.10 |
63.30 |
No |
-- | ||
| Show Supplemental Findings | |||||||||
|
Persisting in any college in University of California system |
Virtual Student Outreach for College Enrollment (V-SOURCE) Complete vs. Business as usual |
15 Months |
Hispanic or Latino, Spanish speaking at home;
|
12.40 |
8.90 |
Yes |
|
||
|
Persisting in college - any selective 4 year college |
Virtual Student Outreach for College Enrollment (V-SOURCE) Complete vs. Business as usual |
15 Months |
Hispanic or Latino, Spanish speaking at home;
|
10.80 |
8.30 |
Yes |
|
||
|
Persisting in any college in University of California system |
Virtual Student Outreach for College Enrollment (V-SOURCE) Complete vs. Business as usual |
15 Months |
Full sample;
|
12.60 |
11.50 |
No |
-- | ||
|
Persisting in college - Any college |
Virtual Student Outreach for College Enrollment (V-SOURCE) Complete vs. Business as usual |
15 Months |
Hispanic or Latino, Spanish speaking at home;
|
62.70 |
59.50 |
No |
-- | ||
|
Persisting in college - Any 4 year college |
Virtual Student Outreach for College Enrollment (V-SOURCE) Complete vs. Business as usual |
15 Months |
Hispanic or Latino, Spanish speaking at home;
|
36.50 |
33.30 |
No |
-- | ||
|
Persisting in college - Any 4 year college |
Virtual Student Outreach for College Enrollment (V-SOURCE) Complete vs. Business as usual |
15 Months |
Full sample;
|
38.50 |
36.70 |
No |
-- | ||
|
Persisting in college - any selective 4 year college |
Virtual Student Outreach for College Enrollment (V-SOURCE) Complete vs. Business as usual |
15 Months |
Full sample;
|
11.20 |
10.80 |
No |
-- | ||
|
Persisting in any college in California State University system |
Virtual Student Outreach for College Enrollment (V-SOURCE) Complete vs. Business as usual |
15 Months |
Full sample;
|
21.00 |
20.30 |
No |
-- | ||
|
Persisting in any college in California State University system |
Virtual Student Outreach for College Enrollment (V-SOURCE) Complete vs. Business as usual |
15 Months |
Hispanic or Latino, Spanish speaking at home;
|
21.10 |
21.90 |
No |
-- | ||
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Sample Characteristics
Characteristics of study sample as reported by study author.
-
Female: 68%
Male: 32% -
Rural, Suburban, Town, Urban
-
- B
- A
- C
- D
- E
- F
- G
- I
- H
- J
- K
- L
- P
- M
- N
- O
- Q
- R
- S
- V
- U
- T
- W
- X
- Z
- Y
- a
- h
- i
- b
- d
- e
- f
- c
- g
- j
- k
- l
- m
- n
- o
- p
- q
- r
- s
- t
- u
- v
- x
- w
- y
California
-
Race Asian 11% Black 6% Other or unknown 76% Two or more races 3% White 5% -
Ethnicity Hispanic 76% Not Hispanic or Latino 24% -
Eligible for Free and Reduced Price Lunch Free or reduced price lunch (FRPL) 53% No FRPL 47%
Study Details
Setting
The study took place in 84 high schools in six counties in central and southern California. The majority of schools were in a large city (59%). The remaining schools were in a suburb (27%), small or midsize city (6%), or small town/rural area (9%).
Study sample
Students were randomly assigned to one of three groups: the V-SOURCE Complete intervention group, the V-SOURCE Milestones intervention group, and the comparison group. This review focuses on the V-SOURCE Complete and comparison groups. The researchers randomly assigned 1,564 students to the V-SOURCE Complete intervention group and 2,577 students to the comparison group. A total of 4,088 students in grades 11 and 12 were included in the study. The students attended 84 high schools in California where a majority of students were economically disadvantaged. The study recruited students who wanted to attend college and whose parents might be less able to provide support during the college application process. Approximately 68% of the sample was female, and 53% were eligible for free or reduced-price lunch. Seventy-six percent did not report race, 11% were Asian, 6% were Black, and 5% were White. Seventy-six percent were Hispanic or Latino, and 24% were non-Hispanic or Latino. About 52% spoke Spanish at home.
Intervention Group
V-SOURCE is an intervention designed to increase college enrollment among economically disadvantaged students on track to be eligible for admission to a public, 4-year college. It is an adaptation of the Student Outreach for College Enrollment (SOURCE) program, which provides in-person, peer advisors for high school students to support them in applying and enrolling in college. The V-SOURCE Complete intervention includes several components: (1) access to a website with information and guidance on the college application process, (2) access to a SAT preparation curriculum, (3) weekly reminders for specific steps in the college and financial aid application process, (4) individualized support through an advisor, and (5) incentives for completing milestones. The V-SOURCE website includes information on college and financial aid applications and allows students to track their individual progress on SAT prep and on college applications. The SAT preparation curriculum, called Ready, SAT, Go!, is available online for students to access, if they choose to do so. This curriculum is intended to assist lower-scoring students prepare for the SAT. Students receive automated electronic reminders via email and text about approaching deadlines, such as the deadline for SAT registration, college applications, or financial aid applications. Students received $20 gift cards for completing each of the four following milestones in the college application process: registering for the SAT, taking the SAT, submitting two college applications to different college systems, and completing the FAFSA. In V-SOURCE Complete, students received individualized support through an assigned advisor. All contact between the student and advisor was done electronically through the V-SOURCE website; or by phone, email, text message, or social media. The advisor provided personalized assistance, based on each student’s needs. This included SAT registration and preparation, college applications, answering questions from the student’s parents, and support in deciding which college to accept. The advisor also provided assistance around financial aid applications, including finding scholarships and helping the student choose from different financial aid offers. In this study, the V-SOURCE Complete intervention was implemented starting in March of students’ junior year and continued through June after their high school graduation. The study also provided additional services in the summer after students' senior year to a random subset of the intervention group. Specifically, one-third of the sample received automated messages and access to their advisor regarding tasks they had to complete during that summer while another one-third received one message at the beginning of the summer with all the tasks to complete for the summer. (The other one-third of the intervention group received no additional services in the summer but could continue to access the V-SOURCE website.) A separate study review under this same manuscript summarizes findings for the V-SOURCE Milestones study arm. V-SOURCE Milestones is a similar intervention to V-SOURCE Complete, but Milestones participants did not have access to an advisor.
Comparison Group
Students in the comparison group received business-as-usual counseling and support for preparing college applications.
Support for implementation
V-SOURCE staff provided advisors with six 4-hour training sessions consisting of presentations and hands-on practice with college application activities. The first two training sessions focused on the V- SOURCE program, using the SAT preparation curriculum, and preparing students to take the SAT. The third and fourth sessions focused on helping students select and apply to colleges. The fifth session focused on financial aid. The sixth session focused on how to help students choose which school to attend, scholarships, and how to ensure students attend their intended college. Advisors had a caseload of 26 students and received supervision from program coordinators. Program coordinators monitored interactions between the advisors and students and were available to answer advisors' questions. EdBoost implemented the V-SOURCE Complete program. More information on EdBoost is available online at https://www.edboost.org/.
Additional Sources
In the case of multiple manuscripts that report on one study, the WWC selects one manuscript as the primary citation and lists other manuscripts that describe the study as additional sources.
-
Phillips, M., & Reber, S. (2019). Report on the implementation and impacts of the V-SOURCE college access program. UCLA. Retrieved from https://escholarship.org/uc/item/55q9n7ws.
-
Phillips, M., & Reber, S. (2018). When "low touch" is not enough: Evidence from a random assignment college access field experiment. [V-SOURCE Complete vs. business as usual control]. California Center for Population Research, University of California Los Angeles, Population Working Paper No. PWP-CCPR-2018-008. http://papers.ccpr.ucla.edu/index.php/pwp/article/view/1213/596.
An indicator of the effect of the intervention, the improvement index can be interpreted as the expected change in percentile rank for an average comparison group student if that student had received the intervention.
For more, please see the WWC Glossary entry for improvement index.
An outcome is the knowledge, skills, and attitudes that are attained as a result of an activity. An outcome measures is an instrument, device, or method that provides data on the outcome.
A finding that is included in the effectiveness rating. Excluded findings may include subgroups and subscales.
The sample on which the analysis was conducted.
The group to which the intervention group is compared, which may include a different intervention, business as usual, or no services.
The timing of the post-intervention outcome measure.
The number of students included in the analysis.
The mean score of students in the intervention group.
The mean score of students in the comparison group.
The WWC considers a finding to be statistically significant if the likelihood that the finding is due to chance alone, rather than a real difference, is less than five percent.
The WWC reviews studies for WWC products, Department of Education grant competitions, and IES performance measures.
The name and version of the document used to guide the review of the study.
The version of the WWC design standards used to guide the review of the study.
The result of the WWC assessment of the study. The rating is based on the strength of evidence of the effectiveness of the intervention. Studies are given a rating of Meets WWC Design Standards without Reservations, Meets WWC Design Standards with Reservations, or >Does Not Meet WWC Design Standards.
A related publication that was reviewed alongside the main study of interest.
Study findings for this report.
Based on the direction, magnitude, and statistical significance of the findings within a domain, the WWC characterizes the findings from a study as one of the following: statistically significant positive effects, substantively important positive effects, indeterminate effects, substantively important negative effects, and statistically significant negative effects. For more, please see the WWC Handbook.
The WWC may review studies for multiple purposes, including different reports and re-reviews using updated standards. Each WWC review of this study is listed in the dropdown. Details on any review may be accessed by making a selection from the drop down list.
Tier 1 Strong indicates strong evidence of effectiveness,
Tier 2 Moderate indicates moderate evidence of effectiveness, and
Tier 3 Promising indicates promising evidence of effectiveness,
as defined in the
non-regulatory guidance for ESSA
and the regulations for ED discretionary grants (EDGAR Part 77).