A Random Assignment Evaluation of Learning Communities at Kingsborough Community College: Seven Years Later
Weiss, Michael J.; Mayer, Alexander; Cullinan, Dan; Ratledge, Alyssa; Sommo, Colleen; Diamond, John (2014). MDRC. Retrieved from: https://eric.ed.gov/?id=ED546644
-
examining1,534Students, gradePS
Practice Guide
Review Details
Reviewed: June 2021
- Practice Guide (findings for Learning Community )
- Randomized Controlled Trial
- Meets WWC standards without reservations because it is a randomized controlled trial with low attrition.
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Findings
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|
|---|---|---|---|---|---|---|---|---|---|
|
Enrolled in a 4-year college |
Learning Community vs. Business as usual |
3 Semesters |
Full sample;
|
7.20 |
6.10 |
No |
-- | ||
|
Average yearly earnings following random assignment |
Learning Community vs. Business as usual |
7 Years |
Full sample;
|
15820.00 |
14652.00 |
No |
-- | ||
|
Percentage of quarters employed |
Learning Community vs. Business as usual |
7 Years |
Full sample;
|
56.40 |
54.20 |
No |
-- | ||
|
Average yearly earnings following random assignment |
Learning Community vs. Business as usual |
3 Years |
Full sample;
|
8417.00 |
8126.00 |
No |
-- | ||
|
Percentage of quarters employed |
Learning Community vs. Business as usual |
3 Years |
Full sample;
|
54.70 |
55.80 |
No |
-- | ||
| Show Supplemental Findings | |||||||||
|
Enrolled in a 4-year college |
Learning Community vs. Business as usual |
1 Semester |
Full sample;
|
2.10 |
1.00 |
No |
-- | ||
|
Average yearly earnings following random assignment |
Learning Community vs. Business as usual |
1 Year |
Full sample;
|
4060.00 |
4611.00 |
No |
-- | ||
|
Enrolled in a 4-year college |
Learning Community vs. Business as usual |
2 Semesters |
Full sample;
|
4.40 |
4.10 |
No |
-- | ||
|
Percentage of quarters employed |
Learning Community vs. Business as usual |
5 Years |
Full sample;
|
56.30 |
53.80 |
No |
-- | ||
|
Percentage of quarters employed |
Learning Community vs. Business as usual |
6 Years |
Full sample;
|
56.20 |
54.20 |
No |
-- | ||
|
Average yearly earnings following random assignment |
Learning Community vs. Business as usual |
6 Years |
Full sample;
|
13656.00 |
12655.00 |
No |
-- | ||
|
Percentage of quarters employed |
Learning Community vs. Business as usual |
2 Years |
Full sample;
|
51.60 |
51.20 |
No |
-- | ||
|
Percentage of quarters employed |
Learning Community vs. Business as usual |
4 Years |
Full sample;
|
56.30 |
56.20 |
No |
-- | ||
|
Average yearly earnings following random assignment |
Learning Community vs. Business as usual |
4 Years |
Full sample;
|
10295.00 |
10204.00 |
No |
-- | ||
|
Average yearly earnings following random assignment |
Learning Community vs. Business as usual |
5 Years |
Full sample;
|
12103.00 |
11970.00 |
No |
-- | ||
|
Percentage of quarters employed |
Learning Community vs. Business as usual |
1 Semester |
Full sample;
|
43.50 |
44.50 |
No |
-- | ||
|
Average yearly earnings following random assignment |
Learning Community vs. Business as usual |
2 Years |
Full sample;
|
6006.00 |
6166.00 |
No |
-- | ||
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|
|---|---|---|---|---|---|---|---|---|---|
|
Earned a degree at any college |
Learning Community vs. Business as usual |
7 Years |
Full sample;
|
39.50 |
36.20 |
No |
-- | ||
| Show Supplemental Findings | |||||||||
|
Earned a degree at any college |
Learning Community vs. Business as usual |
6 Years |
Full sample;
|
35.90 |
31.50 |
No |
-- | ||
|
Earned a degree at any college |
Learning Community vs. Business as usual |
3 Years |
Full sample;
|
20.20 |
17.00 |
No |
-- | ||
|
Earned a degree at any college |
Learning Community vs. Business as usual |
5 Years |
Full sample;
|
31.90 |
28.40 |
No |
-- | ||
|
Highest degree earned: Bachelor's degree or higher |
Learning Community vs. Business as usual |
7 Years |
Full sample;
|
16.50 |
14.80 |
No |
-- | ||
|
Earned a degree at any college |
Learning Community vs. Business as usual |
4 Years |
Full sample;
|
26.50 |
23.80 |
No |
-- | ||
|
Earned a degree at any college |
Learning Community vs. Business as usual |
2 Years |
Full sample;
|
6.10 |
5.40 |
No |
-- | ||
|
Highest degree earned: Associate's degree |
Learning Community vs. Business as usual |
7 Years |
Full sample;
|
22.20 |
20.50 |
No |
-- | ||
|
Earned a degree at any college |
Learning Community vs. Business as usual |
1 Year |
Full sample;
|
0.00 |
0.00 |
-- |
-- | ||
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|
|---|---|---|---|---|---|---|---|---|---|
|
Enrolled in any college |
Learning Community vs. Business as usual |
1 Semester |
Full sample;
|
74.40 |
71.20 |
No |
-- | ||
|
Enrollment |
Learning Community vs. Business as usual |
1 Semester |
Full sample;
|
74.40 |
71.20 |
No |
-- | ||
|
Number of regular credits earned |
Learning Community vs. Business as usual |
1 Semester |
Full sample;
|
7.50 |
7.00 |
No |
-- | ||
| Show Supplemental Findings | |||||||||
|
Enrolled in any college |
Learning Community vs. Business as usual |
3 Semesters |
Full sample;
|
58.70 |
53.00 |
Yes |
|
||
|
Enrolled in any 2-year college |
Learning Community vs. Business as usual |
3 Semesters |
Full sample;
|
52.00 |
47.20 |
No |
-- | ||
|
Registered in any course |
Learning Community vs. Business as usual |
3 Semesters |
Full sample;
|
52.90 |
47.80 |
Yes |
|
||
|
Number of regular credits earned |
Learning Community vs. Business as usual |
3 Semesters |
Full sample;
|
5.60 |
5.10 |
No |
-- | ||
|
Registered in any course |
Learning Community vs. Business as usual |
1 Semester |
Full sample;
|
77.40 |
75.00 |
No |
-- | ||
|
Enrolled in any 2-year college |
Learning Community vs. Business as usual |
1 Semester |
Full sample;
|
72.30 |
70.30 |
No |
-- | ||
|
Number of regular credits earned |
Learning Community vs. Business as usual |
2 Semesters |
Full sample;
|
6.60 |
6.30 |
No |
-- | ||
|
Registered in any course |
Learning Community vs. Business as usual |
2 Semesters |
Full sample;
|
61.30 |
59.20 |
No |
-- | ||
|
Enrolled in any college |
Learning Community vs. Business as usual |
2 Semesters |
Full sample;
|
62.50 |
61.10 |
No |
-- | ||
|
Enrolled in any 2-year college |
Learning Community vs. Business as usual |
2 Semesters |
Full sample;
|
58.10 |
57.00 |
No |
-- | ||
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Sample Characteristics
Characteristics of study sample as reported by study author.
-
Female: 55%
Male: 45% -
Urban
-
- B
- A
- C
- D
- E
- F
- G
- I
- H
- J
- K
- L
- P
- M
- N
- O
- Q
- R
- S
- V
- U
- T
- W
- X
- Z
- Y
- a
- h
- i
- b
- d
- e
- f
- c
- g
- j
- k
- l
- m
- n
- o
- p
- q
- r
- s
- t
- u
- v
- x
- w
- y
New York
-
Race Asian 8% Black 38% Other or unknown 28% White 27% -
Ethnicity Hispanic 20% Not Hispanic or Latino 80%
Study Details
Setting
The study was set in Kingsborough Community College in Brooklyn, NY.
Study sample
Students were first-time, incoming freshmen who planned to attend college full time during the day, did not test into English as a Second-Language (ESL), and were between the ages of 17 and 34. Approximately 55% of the study participants were female and about 33% were the first in their families to attend college. About 20% of students in the study were Hispanic, 38% were Black/non-Hispanic, and 27% were White/non-Hispanic.
Intervention Group
Opening Doors Learning Communities (ODLC) is a one-semester learning community program. Students participating in learning communities (LC) program were expected to take three linked classes together during their first semester at the college. The ODLC was designed for full-time freshmen who did not place into the English as a Second Language (ESL) course because there was a separate program available for those students. The ODLC courses were: (1) a one-credit orientation course, (2) a developmental English course, and (3) one academic course. Class sizes within the LCs were limited to 25 students, compared with up to 35 in regular freshman classes. ODLC students received tutoring and counseling through the orientation course; tutors and case managers were assigned to each LC, and textbook vouchers were provided. In addition, students received a voucher for textbooks as part of ODLC ($225). Counselors in the LCs were responsible for 75 to 100 students and take a proactive advisement stance.
Comparison Group
Students in the comparison condition attended courses as usual and were able to take advantage of regular services offered by the college, including tutoring, counseling, and advising. They were encouraged to take the freshman orientation course that is part of the LC. Counselors in the regular program were each responsible for 500 students.
Support for implementation
Faculty teaching courses within the LC had a reduced course load, and counselors similarly had a reduced caseload.
Department-funded evaluation
Review Details
Reviewed: June 2021
- Department-funded evaluation (findings for Opening Doors Learning Communities Program)
- Randomized Controlled Trial
- Meets WWC standards without reservations because it is a randomized controlled trial with low attrition.
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Findings
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|
|---|---|---|---|---|---|---|---|---|---|
|
Received a degree |
Opening Doors Learning Communities Program vs. Business as usual |
7 Years |
Full sample;
|
39.50 |
36.20 |
No |
-- | ||
| Show Supplemental Findings | |||||||||
|
Received a degree |
Opening Doors Learning Communities Program vs. Business as usual |
3 Years |
Full sample;
|
20.20 |
17.00 |
No |
-- | ||
|
Received a degree |
Opening Doors Learning Communities Program vs. Business as usual |
6 Years |
Full sample;
|
35.90 |
31.50 |
No |
-- | ||
|
Received a degree |
Opening Doors Learning Communities Program vs. Business as usual |
5 Years |
Full sample;
|
31.90 |
28.40 |
No |
-- | ||
|
Received a degree |
Opening Doors Learning Communities Program vs. Business as usual |
4 Years |
Full sample;
|
26.50 |
23.80 |
No |
-- | ||
|
Bachelor's degree or higher earned by year 7 |
Opening Doors Learning Communities Program vs. Business as usual |
7 Years |
Full sample;
|
16.50 |
14.80 |
No |
-- | ||
|
Earning only an Associate's degree |
Opening Doors Learning Communities Program vs. Business as usual |
7 Years |
Full sample;
|
22.20 |
20.50 |
No |
-- | ||
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Received a degree |
Opening Doors Learning Communities Program vs. Business as usual |
2 Years |
Full sample;
|
6.10 |
5.40 |
No |
-- |
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|
|---|---|---|---|---|---|---|---|---|---|
|
Earnings |
Opening Doors Learning Communities Program vs. Business as usual |
7 Years |
Full sample;
|
15820.00 |
14652.00 |
No |
-- | ||
| Show Supplemental Findings | |||||||||
|
Earnings |
Opening Doors Learning Communities Program vs. Business as usual |
2 Years |
Full sample;
|
6006.00 |
6166.00 |
No |
-- | ||
|
Earnings |
Opening Doors Learning Communities Program vs. Business as usual |
3 Years |
Full sample;
|
8417.00 |
8126.00 |
No |
-- | ||
|
Earnings |
Opening Doors Learning Communities Program vs. Business as usual |
4 Years |
Full sample;
|
10295.00 |
10204.00 |
No |
-- | ||
|
Earnings |
Opening Doors Learning Communities Program vs. Business as usual |
5 Years |
Full sample;
|
12103.00 |
11970.00 |
No |
-- | ||
|
Earnings |
Opening Doors Learning Communities Program vs. Business as usual |
6 Years |
Full sample;
|
13656.00 |
12655.00 |
No |
-- | ||
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Earnings |
Opening Doors Learning Communities Program vs. Business as usual |
1 Year |
Full sample;
|
4060.00 |
4611.00 |
No |
-- |
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|
|---|---|---|---|---|---|---|---|---|---|
|
Percentage of quarters employed |
Opening Doors Learning Communities Program vs. Business as usual |
7 Years |
Full sample;
|
56.40 |
54.20 |
No |
-- | ||
| Show Supplemental Findings | |||||||||
|
Percentage of quarters employed |
Opening Doors Learning Communities Program vs. Business as usual |
5 Years |
Full sample;
|
56.30 |
53.80 |
No |
-- | ||
|
Percentage of quarters employed |
Opening Doors Learning Communities Program vs. Business as usual |
6 Years |
Full sample;
|
56.20 |
54.20 |
No |
-- | ||
|
Percentage of quarters employed |
Opening Doors Learning Communities Program vs. Business as usual |
2 Years |
Full sample;
|
51.60 |
51.20 |
No |
-- | ||
|
Percentage of quarters employed |
Opening Doors Learning Communities Program vs. Business as usual |
4 Years |
Full sample;
|
56.30 |
56.20 |
No |
-- | ||
|
Percentage of quarters employed |
Opening Doors Learning Communities Program vs. Business as usual |
3 Years |
Full sample;
|
54.70 |
55.80 |
No |
-- | ||
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Percentage of quarters employed |
Opening Doors Learning Communities Program vs. Business as usual |
1 Year |
Full sample;
|
43.50 |
44.50 |
No |
-- |
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Sample Characteristics
Characteristics of study sample as reported by study author.
-
Female: 55%
Male: 45% -
Urban
-
- B
- A
- C
- D
- E
- F
- G
- I
- H
- J
- K
- L
- P
- M
- N
- O
- Q
- R
- S
- V
- U
- T
- W
- X
- Z
- Y
- a
- h
- i
- b
- d
- e
- f
- c
- g
- j
- k
- l
- m
- n
- o
- p
- q
- r
- s
- t
- u
- v
- x
- w
- y
New York
-
Race Asian 9% Black 38% Other or unknown 26% White 27% -
Ethnicity Hispanic 20% Not Hispanic or Latino 80%
Study Details
Setting
The study took place over four semesters (fall 2003 through spring 2005) at Kingsborough Community College, a large urban college in the City University of New York (CUNY) system, located in Brooklyn, New York. Eligibility requirements included: (1) first-time incoming freshman who planned to attend college full time during the day, (2) tested into either developmental English or college-level English, and (3) between the ages of 17 and 34.
Study sample
Of the 1,534 students participating in the study, 55 percent were female. Twenty percent reported their ethnicity as Hispanic. Race was reported as 38 percent black, 27 percent white, and 9 percent Asian or Pacific Islander. Free/reduced price lunch statistics were not reported but the authors did report employment and earnings for years or quarters prior to the study. A little over half had ever been employed (54%) in the year prior to random assignment, and reported earning, on average, less than $950 in the quarter prior to random assignment. Less than half (47%) spoke a language other than English regularly at home, but as a condition of recruitment, no students tested into English as a second language.
Intervention Group
As part of the Opening Doors Learning Communities, first semester freshmen are placed into groups of 25 or fewer students that take a set of three linked classes together that are scheduled in a block: an English course (either college-level or developmental), an academic course required by major, and a 1-credit freshman orientation course. Students assigned to the intervention condition were also provided with enhanced counseling and tutoring services, as well as textbook vouchers.
Comparison Group
Students assigned to the comparison group took the college's business-as-usual classes and received the college's regular services. They were not required to take English or the freshman orientation course.
Support for implementation
The authors did not describe support for implementation.
An indicator of the effect of the intervention, the improvement index can be interpreted as the expected change in percentile rank for an average comparison group student if that student had received the intervention.
For more, please see the WWC Glossary entry for improvement index.
An outcome is the knowledge, skills, and attitudes that are attained as a result of an activity. An outcome measures is an instrument, device, or method that provides data on the outcome.
A finding that is included in the effectiveness rating. Excluded findings may include subgroups and subscales.
The sample on which the analysis was conducted.
The group to which the intervention group is compared, which may include a different intervention, business as usual, or no services.
The timing of the post-intervention outcome measure.
The number of students included in the analysis.
The mean score of students in the intervention group.
The mean score of students in the comparison group.
The WWC considers a finding to be statistically significant if the likelihood that the finding is due to chance alone, rather than a real difference, is less than five percent.
The WWC reviews studies for WWC products, Department of Education grant competitions, and IES performance measures.
The name and version of the document used to guide the review of the study.
The version of the WWC design standards used to guide the review of the study.
The result of the WWC assessment of the study. The rating is based on the strength of evidence of the effectiveness of the intervention. Studies are given a rating of Meets WWC Design Standards without Reservations, Meets WWC Design Standards with Reservations, or >Does Not Meet WWC Design Standards.
A related publication that was reviewed alongside the main study of interest.
Study findings for this report.
Based on the direction, magnitude, and statistical significance of the findings within a domain, the WWC characterizes the findings from a study as one of the following: statistically significant positive effects, substantively important positive effects, indeterminate effects, substantively important negative effects, and statistically significant negative effects. For more, please see the WWC Handbook.
The WWC may review studies for multiple purposes, including different reports and re-reviews using updated standards. Each WWC review of this study is listed in the dropdown. Details on any review may be accessed by making a selection from the drop down list.
Tier 1 Strong indicates strong evidence of effectiveness,
Tier 2 Moderate indicates moderate evidence of effectiveness, and
Tier 3 Promising indicates promising evidence of effectiveness,
as defined in the
non-regulatory guidance for ESSA
and the regulations for ED discretionary grants (EDGAR Part 77).