When "low touch" is not enough: Evidence from a random assignment college access field experiment. [V-SOURCE Complete vs. business as usual control]
Phillips, M., & Reber, S. (2018). California Center for Population Research, University of California Los Angeles, Population Working Paper No. PWP-CCPR-2018-008. http://papers.ccpr.ucla.edu/index.php/pwp/article/view/1213/596 .
-
examining6,642Students, grades11-PS
Practice Guide
Review Details
Reviewed: March 2024
- Practice Guide (findings for Virtual Student Outreach for College Enrollment (V-SOURCE) Complete)
- Additional source not reviewed because it is not the primary source for the study (View primary source).
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Findings
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Sample Characteristics
Study sample characteristics were not reported.Single Study Review
Review Details
Reviewed: September 2019
- Single Study Review (findings for V-SOURCE)
- Randomized Controlled Trial
- Meets WWC standards without reservations because it is a randomized controlled trial with low attrition.
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Findings
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|
|---|---|---|---|---|---|---|---|---|---|
|
Any College_Complete |
V-SOURCE vs. Business as usual |
0 Months |
Full sample;
|
71.30 |
70.70 |
No |
-- | ||
|
Any College_Milestones |
V-SOURCE vs. Business as usual |
0 Months |
Full sample;
|
71.20 |
70.70 |
No |
-- | ||
| Show Supplemental Findings | |||||||||
|
Enrolled in a Selective College_Milestones |
V-SOURCE vs. Business as usual |
0 Months |
Full sample;
|
13.40 |
11.80 |
Yes |
|
||
|
Enrollment at a University of California College_Milestones |
V-SOURCE vs. Business as usual |
0 Months |
Full sample;
|
14.50 |
12.80 |
Yes |
|
||
|
Enrolled in a College in the University of California System_Complete |
V-SOURCE vs. Business as usual |
0 Months |
Full sample;
|
13.30 |
12.80 |
No |
-- | ||
|
Four-Year College Enrollment_Complete |
V-SOURCE vs. Business as usual |
0 Months |
Full sample;
|
44.10 |
43.50 |
No |
-- | ||
|
Four-Year College Enrollment_Milestones |
V-SOURCE vs. Business as usual |
0 Months |
Full sample;
|
44.60 |
43.50 |
No |
-- | ||
|
Enrolled in a Selective College_Complete |
V-SOURCE vs. Business as usual |
0 Months |
Full sample;
|
11.90 |
11.80 |
No |
-- | ||
|
Enrollment in a California State University_Complete |
V-SOURCE vs. Business as usual |
0 Months |
Full sample;
|
25.10 |
25.00 |
No |
-- | ||
|
Enrollment in a California State University_Milestones |
V-SOURCE vs. Business as usual |
0 Months |
Full sample;
|
23.40 |
25.00 |
Yes |
-- | ||
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|
|---|---|---|---|---|---|---|---|---|---|
|
Persistence_Any College_Complete |
V-SOURCE vs. Business as usual |
1 Year |
Full sample;
|
65.20 |
63.40 |
No |
-- | ||
|
Persistence_Any College_Milestones |
V-SOURCE vs. Business as usual |
1 Year |
Full sample;
|
64.30 |
63.40 |
No |
-- | ||
| Show Supplemental Findings | |||||||||
|
Persistence at a Selective College_Milestones |
V-SOURCE vs. Business as usual |
1 Year |
Full sample;
|
12.20 |
10.90 |
Yes |
|
||
|
Persistence in a Four-Year-College_Complete |
V-SOURCE vs. Business as usual |
1 Year |
Full sample;
|
38.50 |
36.90 |
No |
-- | ||
|
Persistence in a University of California College_Complete |
V-SOURCE vs. Business as usual |
1 Year |
Full sample;
|
12.70 |
11.60 |
No |
-- | ||
|
Persistence at a Selective College_Complete |
V-SOURCE vs. Business as usual |
1 Year |
Full sample;
|
11.30 |
10.90 |
No |
-- | ||
|
Persistence in a California State University College_Complete |
V-SOURCE vs. Business as usual |
1 Year |
Full sample;
|
21.00 |
20.50 |
No |
-- | ||
|
Persistence in a Four-Year-College_Milestones |
V-SOURCE vs. Business as usual |
1 Year |
Full sample;
|
37.20 |
36.90 |
No |
-- | ||
|
Persistence in a California State University College_Milestones |
V-SOURCE vs. Business as usual |
1 Year |
Full sample;
|
19.30 |
20.50 |
No |
-- | ||
|
Persistence at a University of California College_Milestones |
V-SOURCE vs. Business as usual |
1 Year |
Full sample;
|
13.30 |
11.60 |
Yes |
-- | ||
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Sample Characteristics
Characteristics of study sample as reported by study author.
-
Female: 68%
Male: 32% -
- B
- A
- C
- D
- E
- F
- G
- I
- H
- J
- K
- L
- P
- M
- N
- O
- Q
- R
- S
- V
- U
- T
- W
- X
- Z
- Y
- a
- h
- i
- b
- d
- e
- f
- c
- g
- j
- k
- l
- m
- n
- o
- p
- q
- r
- s
- t
- u
- v
- x
- w
- y
California
-
Race Asian 11% Black 6% White 42% -
Ethnicity Hispanic 76% Not Hispanic or Latino 24%
Study Details
Setting
This study recruited junior-level students from 84 high schools located in six California counties during the fall of the 2011-12 and 2012-13 school years. Five of these counties are located in Southern California, and one is in Central California. Targeted high schools had a majority of Hispanic/Latinx or African American students, and a majority of students were eligible for free or reduced price meals.
Study sample
The intervention was designed for students who are college ready but have parents who might not be able to provide needed support during the application process because of a lack of familiarity with higher education and do not have the economic resources needed to access paid college counselors. About 75 percent of the students in the sample reported maintaining a B average or above, and the majority (68 percent) reported having high educational aspirations, such as earning a graduate degree. About 68 percent of the sample were girls and about 75 percent reported they were Hispanic. Approximately half of the sample reported “using lunch tickets,” which is a proxy for subsidized meal eligibility. Finally, approximately 60 percent of the sample was made of up first generation college students.
Intervention Group
The Virtual Student Outreach for College Enrollment (V-SOURCE) is a 15-month college advising program designed to provide low-income students with the information, reminders, and support that higher-income students typically receive more of. The intervention spans March of students’ junior year through summer after students’ senior year. Two V-SOURCE variants were tested in the study. One version of V-Source provided access to a personal advisor (the Complete program) who met with students on-line and via text; the second version provided access to a fully automated advisor (the Milestones program). Both versions provided students with access to the V-SOURCE website, an on-line SAT study program, close to weekly reminders to complete application activities provided via email and texts, and $20 gift cards provides after students completed the following milestones: (1) registered for the SAT, (2) took the SAT, (3) submitted two college applications, and (4) submitted the Free Application for Federal Student Aid (FAFSA) on time.
Comparison Group
Comparison services are described as “business as usual” services.
Support for implementation
The study authors collaborated with the EdBoost Education Corporation to generate the V-SOURCE program to reduce its cost and simplify scaling.
An indicator of the effect of the intervention, the improvement index can be interpreted as the expected change in percentile rank for an average comparison group student if that student had received the intervention.
For more, please see the WWC Glossary entry for improvement index.
An outcome is the knowledge, skills, and attitudes that are attained as a result of an activity. An outcome measures is an instrument, device, or method that provides data on the outcome.
A finding that is included in the effectiveness rating. Excluded findings may include subgroups and subscales.
The sample on which the analysis was conducted.
The group to which the intervention group is compared, which may include a different intervention, business as usual, or no services.
The timing of the post-intervention outcome measure.
The number of students included in the analysis.
The mean score of students in the intervention group.
The mean score of students in the comparison group.
The WWC considers a finding to be statistically significant if the likelihood that the finding is due to chance alone, rather than a real difference, is less than five percent.
The WWC reviews studies for WWC products, Department of Education grant competitions, and IES performance measures.
The name and version of the document used to guide the review of the study.
The version of the WWC design standards used to guide the review of the study.
The result of the WWC assessment of the study. The rating is based on the strength of evidence of the effectiveness of the intervention. Studies are given a rating of Meets WWC Design Standards without Reservations, Meets WWC Design Standards with Reservations, or >Does Not Meet WWC Design Standards.
A related publication that was reviewed alongside the main study of interest.
Study findings for this report.
Based on the direction, magnitude, and statistical significance of the findings within a domain, the WWC characterizes the findings from a study as one of the following: statistically significant positive effects, substantively important positive effects, indeterminate effects, substantively important negative effects, and statistically significant negative effects. For more, please see the WWC Handbook.
The WWC may review studies for multiple purposes, including different reports and re-reviews using updated standards. Each WWC review of this study is listed in the dropdown. Details on any review may be accessed by making a selection from the drop down list.
Tier 1 Strong indicates strong evidence of effectiveness,
Tier 2 Moderate indicates moderate evidence of effectiveness, and
Tier 3 Promising indicates promising evidence of effectiveness,
as defined in the
non-regulatory guidance for ESSA
and the regulations for ED discretionary grants (EDGAR Part 77).