Becoming College-Ready: Early Findings from a CUNY Start Evaluation
Scrivener, Susan; Gupta, Himani; Weiss, Michael J.; Cohen, Benjamin; Cormier, Maria Scott; Brathwaite, Jessica (2018). MDRC. Retrieved from: https://eric.ed.gov/?id=ED586380
-
examining3,835Students, gradePS
Practice Guide
Review Details
Reviewed: March 2022
- Practice Guide (findings for CUNY Start)
- Randomized Controlled Trial
- Meets WWC standards without reservations because it is a randomized controlled trial with low attrition.
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Findings
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
College-level credits earned: After 1 Semester |
CUNY Start vs. Business as usual |
0 Semesters |
Full sample;
|
0.56 |
2.45 |
Yes |
|
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Sample Characteristics
Characteristics of study sample as reported by study author.
-
Female: 56%
Male: 44% -
Urban
-
- B
- A
- C
- D
- E
- F
- G
- I
- H
- J
- K
- L
- P
- M
- N
- O
- Q
- R
- S
- V
- U
- T
- W
- X
- Z
- Y
- a
- h
- i
- b
- d
- e
- f
- c
- g
- j
- k
- l
- m
- n
- o
- p
- q
- r
- s
- t
- u
- v
- x
- w
- y
New York
-
Race Black 32% Other or unknown 62% White 7% -
Ethnicity Hispanic 44% Not Hispanic or Latino 56%
Study Details
Setting
The study took place at four of the eight City University of New York (CUNY) campuses that offer CUNY Start: Borough of Manhattan Community College, Kingsborough Community College, LaGuardia Community College, and Queensborough Community College. CUNY Start was housed in each college’s Continuing Education division. All of these colleges offered associate's degree programs.
Study sample
On average, just under half (48%) were 19 years old or younger, nearly half (43.7%) were Hispanic, and just under one-third (32%) were African American. A small minority (7%) were White, and over half (56%) spoke English as a native language (22% had Spanish as their native language, and 22% had another language besides Spanish or English as their native language). Over one-third (35%) were the first in their family to attend college, and nearly half (49%) were employed at baseline. Over half (51%) required three developmental subject areas, but over half (59%) hoped to achieve a bachelor's degree, and over one-quarter hoped to achieve a postgraduate or professional degree.
Intervention Group
CUNY Start was a one-semester program for students assessed as needing remediation; it had both a full-time and part-time version. The full-time version was open to students who needed remediation in math and reading or writing (or in all three subjects), while the part-time version was also open to students who needed remediation in only one subject. CUNY Start provided up to 26.5 hours of intensive instruction following a prescribed approach in math, reading, and writing; it also provided advising, tutoring, and a weekly college skills seminar. CUNY Start's math instructional approach was student-centered, rather than lecture-centered, and integrated arithmetic and algebra and encouraged conceptual understanding, real-world learning, and the building of academic skills. CUNY Start's instructional approach to reading and writing was less substantively different than the standard developmental courses in these areas but integrates the two subjects to allow students to move more quickly through their developmental requirements. Students paid $75 to participate in CUNY Start; the cost was not covered by financial aid and therefore, does not require students to use one of their semesters of federal financial aid eligibility. The program's goal was to prepare students for college-level courses while providing all of the developmental education they need in one semester. After 12 weeks of CUNY Start, students took exit tests in the subject areas for which they needed remediation; if they passed, they were eligible to take college-level courses the next semester; if students did not pass, they received three to six additional weeks of CUNY Start instruction before being reassessed.
Comparison Group
The comparison condition was business as usual. The comparison condition cost $2,400, but students could use financial aid towards the cost. Students could still take up to 3 semesters of developmental math, reading and writing (if needed), but they were also able to enroll in college-level courses right away. Their math, reading, and writing courses were typically not taught in an integrated way, and math classes mostly tended to be lecture-based. However, the developmental reading and writing were mostly student-centered. The College Success seminar was not mandatory, but it was available to students in the comparison group. Students had advisors, but their advisors had a much heavier student load than those involved in CUNY Start. About one-third of non-CUNY Start students received tutoring.
Support for implementation
There were two main supports for implementation: (1) CUNY Start was housed in Continuing Education and managed centrally (not at each campus), while the other programs' administration is in Academic Affairs and managed within the academic departments; and (2) Many of the CUNY Start instructors participated in a semester-long apprenticeship prior to starting and continuing professional development, while most of the general instructors did not participate in additional training before teaching a course and reported fewer hours of professional development than their CUNY Start counterparts. To be hired as a CUNY Start instructor, the individual had to demonstrate both content knowledge and openness to the curriculum and pedagogy of CUNY Start.
Single Study Review
Review Details
Reviewed: January 2019
- Single Study Review (findings for CUNY Start)
- Randomized Controlled Trial
- Meets WWC standards without reservations because it is a randomized controlled trial with low attrition.
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Findings
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
College-ready in math |
CUNY Start vs. Business as usual |
0 Semesters |
Full sample;
|
57.00 |
25.00 |
Yes |
|
|
|
College-ready in reading |
CUNY Start vs. Business as usual |
0 Semesters |
Full sample;
|
70.00 |
62.00 |
Yes |
|
|
|
College-ready in writing |
CUNY Start vs. Business as usual |
0 Semesters |
Full sample;
|
61.00 |
52.00 |
Yes |
|
|
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Persistence |
CUNY Start vs. Business as usual |
1 Semester |
Full sample;
|
69.00 |
64.00 |
Yes |
|
|
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
College-level credits earned |
CUNY Start vs. Business as usual |
0 Semesters |
Full sample;
|
0.57 |
2.45 |
Yes |
|
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Sample Characteristics
Characteristics of study sample as reported by study author.
-
44% English language learners -
Female: 56%
Male: 44% -
Urban
-
- B
- A
- C
- D
- E
- F
- G
- I
- H
- J
- K
- L
- P
- M
- N
- O
- Q
- R
- S
- V
- U
- T
- W
- X
- Z
- Y
- a
- h
- i
- b
- d
- e
- f
- c
- g
- j
- k
- l
- m
- n
- o
- p
- q
- r
- s
- t
- u
- v
- x
- w
- y
New York
-
Race Black 32% -
Ethnicity Hispanic 44% Not Hispanic or Latino 56%
Study Details
Setting
The study took place at four City University of New York (CUNY) campuses: Borough of Manhattan Community College, Kingsborough Community College, LaGuardia Community College, and Queensborough Community College. CUNY Start is housed in each college’s Continuing Education division.
Study sample
56 percent of students in the study were female. Most students were Hispanic (44 percent) or Black (32 percent), and 44 percent were non-native English speakers. 35 percent were the first person in their family to attend college. 48 percent were 19 years old or younger, while 22 percent were 24 or older. Half (51 percent) required developmental instruction in three subject areas, 33 percent in two subject areas, and 15 percent in one subject area.
Intervention Group
CUNY Start is a one-semester program for students assessed as needing remediation; it has both a full-time and part-time version. The full-time version is open to students who need remediation in math and reading or writing (or in all three subjects), while the part-time version is also open to students who need remediation in only one subject. CUNY Start provides up to 26.5 hours of intensive instruction following a prescribed approach in math, reading, and writing; it also provides advising, tutoring, and a weekly college skills seminar. CUNY Start's math instructional approach is student-centered, rather than lecture-centered, and "integrates arithmetic and algebra and encourages conceptual understanding, real-world learning, and the building of academic skills..." (p. ES-6). CUNY Start's instructional approach to reading and writing is less substantively different than the standard developmental courses in these areas but integrates the two subjects to allow students to move more quickly through their developmental requirements. Students pay $75 to participate in CUNY Start (the cost is not covered by financial aid but, therefore, does not require students' to use one of their semesters of federal financial aid eligibility). The program's goal is to prepare students for college-level courses while providing all of the developmental education they need in one semester. After 12 weeks of CUNY Start, students take exit tests in the subject areas for which they needed remediation; if they pass, they are eligible to take college-level courses the next semester; if they fail, they receive three to six additional weeks of CUNY Start instruction before being reassessed.
Comparison Group
The comparison condition was "business as usual." Students could receive standard developmental or college-level courses and services.
Support for implementation
"CUNY Start instructors are expected to spend a paid apprenticeship semester observing and assisting lead instructors before they begin teaching their own classes. After the apprenticeship period, CUNY Start professional development staff members provide continuing training and support" (p. 11). The implementation evaluation indicated the CUNY Start was implemented as intended.
An indicator of the effect of the intervention, the improvement index can be interpreted as the expected change in percentile rank for an average comparison group student if that student had received the intervention.
For more, please see the WWC Glossary entry for improvement index.
An outcome is the knowledge, skills, and attitudes that are attained as a result of an activity. An outcome measures is an instrument, device, or method that provides data on the outcome.
A finding that is included in the effectiveness rating. Excluded findings may include subgroups and subscales.
The sample on which the analysis was conducted.
The group to which the intervention group is compared, which may include a different intervention, business as usual, or no services.
The timing of the post-intervention outcome measure.
The number of students included in the analysis.
The mean score of students in the intervention group.
The mean score of students in the comparison group.
The WWC considers a finding to be statistically significant if the likelihood that the finding is due to chance alone, rather than a real difference, is less than five percent.
The WWC reviews studies for WWC products, Department of Education grant competitions, and IES performance measures.
The name and version of the document used to guide the review of the study.
The version of the WWC design standards used to guide the review of the study.
The result of the WWC assessment of the study. The rating is based on the strength of evidence of the effectiveness of the intervention. Studies are given a rating of Meets WWC Design Standards without Reservations, Meets WWC Design Standards with Reservations, or >Does Not Meet WWC Design Standards.
A related publication that was reviewed alongside the main study of interest.
Study findings for this report.
Based on the direction, magnitude, and statistical significance of the findings within a domain, the WWC characterizes the findings from a study as one of the following: statistically significant positive effects, substantively important positive effects, indeterminate effects, substantively important negative effects, and statistically significant negative effects. For more, please see the WWC Handbook.
The WWC may review studies for multiple purposes, including different reports and re-reviews using updated standards. Each WWC review of this study is listed in the dropdown. Details on any review may be accessed by making a selection from the drop down list.
Tier 1 Strong indicates strong evidence of effectiveness,
Tier 2 Moderate indicates moderate evidence of effectiveness, and
Tier 3 Promising indicates promising evidence of effectiveness,
as defined in the
non-regulatory guidance for ESSA
and the regulations for ED discretionary grants (EDGAR Part 77).