Brock, Thomas; Richburg-Hayes, Lashawn (2006). MDRC. Retrieved from: https://eric.ed.gov/?id=ED491719
Reviewed: February 2024
Strategies for Postsecondary Students in Developmental Education–A Practice Guide for College and University Administrators, Advisors, and Faculty Practice Guide
(findings for Louisiana Opening Doors Program)
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Cumulative GPA (2.0 and higher) through first seven semesters |
Louisiana Opening Doors Program vs. Business as usual |
7 Semesters |
Full sample-second wave;
|
0.51 |
0.44 |
Yes |
|
|
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Registered for any courses first semester |
Louisiana Opening Doors Program vs. Business as usual |
1 Semester |
Full sample-second wave;
|
0.82 |
0.77 |
Yes |
|
|
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Number of credits earned |
Louisiana Opening Doors Program vs. Business as usual |
7 Semesters |
Full sample-second wave;
|
14.90 |
12.00 |
Yes |
|
|
Characteristics of study sample as reported by study author.
Two New Orleans, Louisiana-based community colleges: Delgado Community College and Louisiana Technical College-West Jefferson
In addition to any other financial aid for which students qualified for, the Louisiana Opening Doors program offered eligible low-income parents a $1,000 monetary award for each of two semesters ($2,000 total) for maintaining at least half-time enrollment and a 2.0 GPA. Students in the intervention group also received advising and monitoring support (i.e., dedicated program advisors, who helped verify enrollment and monitored students). The program required that intervention students meet with a counselor periodically.
The comparison group was considered “business as usual.” Participants in the comparison group received the standard financial aid supports, including Pell Grants. They could access academic advising and counseling which was available to all students. They did not receive an Opening Doors scholarship and did not have counselors who monitored their academic performance.
In the case of multiple manuscripts that report on one study, the WWC selects one manuscript as the primary citation and lists other manuscripts that describe the study as additional sources.
Barrow, L., Richburg-Hayes, L., Rouse, C. E., & Brock, T. (2012). Paying for performance: The education impacts of a community college scholarship program for low-income adults (Working Paper Series: WP-09-13, 2009). Chicago, IL: Federal Reserve Bank of Chicago. https://www.chicagofed.org/publications/working-papers/2009/wp-13.
Richburg-Hayes, Lashawn; Brock, Thomas; LeBlanc, Allen; Paxson, Christina; Rouse, Cecilia Elena; Barrow, Lisa. (2009). Rewarding Persistence: Effects of a Performance-Based Scholarship Program for Low-Income Parents. MDRC.
Barrow, L., Richburg-Hayes, L., Rouse, C. E., & Brock, T. (2014). Paying for performance: The education impacts of a community college scholarship program for low-income adults. Journal of Labor Economics, 32(3), 563–599. https://www.journals.uchicago.edu/doi/10.1086/675229.
Reviewed: July 2021
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|
|---|---|---|---|---|---|---|---|---|---|
|
Term GPA, % with GPA 2.0 or greater |
Louisiana Opening Doors Program vs. Business as usual |
0 Days |
Full sample;
|
38.00 |
26.90 |
Yes |
|
|
|
| Show Supplemental Findings | |||||||||
|
Term GPA, % with GPA 2.0 or greater |
Louisiana Opening Doors Program vs. Business as usual |
0 Days |
First 2 cohorts;
|
40.40 |
19.60 |
Yes |
|
||
|
Term GPA, % with GPA 2.0 or greater |
Louisiana Opening Doors Program vs. Business as usual |
1 Semester |
First 2 cohorts;
|
27.10 |
19.20 |
Yes |
|
||
|
Term GPA, % with GPA 2.0 or greater |
Louisiana Opening Doors Program vs. Business as usual |
2 Semesters |
First 2 cohorts;
|
19.20 |
13.70 |
No |
-- | ||
|
Cumulative - Earned a 2.0 GPA or greater |
Louisiana Opening Doors Program vs. Business as usual |
5 Semesters |
First 2 cohorts;
|
45.60 |
36.90 |
Yes |
|
||
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|
|---|---|---|---|---|---|---|---|---|---|
|
Number of regular credits earned |
Louisiana Opening Doors Program vs. Business as usual |
0 Days |
Full sample;
|
3.00 |
2.10 |
Yes |
|
|
|
| Show Supplemental Findings | |||||||||
|
Registered for any course |
Louisiana Opening Doors Program vs. Business as usual |
1 Semester |
First 2 cohorts;
|
49.40 |
37.60 |
Yes |
|
||
|
Registered for any course |
Louisiana Opening Doors Program vs. Business as usual |
2 Semesters |
First 2 cohorts;
|
30.10 |
22.90 |
Yes |
|
||
|
Cumulative - Registered for any course |
Louisiana Opening Doors Program vs. Business as usual |
5 Semesters |
First 2 cohorts;
|
84.70 |
79.30 |
No |
-- | ||
|
Cumulative - Number of semesters registered |
Louisiana Opening Doors Program vs. Business as usual |
5 Semesters |
First 2 cohorts;
|
2.50 |
2.00 |
Yes |
-- | ||
|
Cumulative - Number of regular credits earned |
Louisiana Opening Doors Program vs. Business as usual |
5 Semesters |
First 2 cohorts;
|
11.20 |
7.90 |
Yes |
-- | ||
|
Number of regular credits earned |
Louisiana Opening Doors Program vs. Business as usual |
0 Days |
First 2 cohorts;
|
2.40 |
1.50 |
Yes |
-- | ||
|
Number of regular credits earned |
Louisiana Opening Doors Program vs. Business as usual |
1 Semester |
First 2 cohorts;
|
2.50 |
1.80 |
Yes |
-- | ||
|
Number of courses passed |
Louisiana Opening Doors Program vs. Business as usual |
0 Days |
First 2 cohorts;
|
1.20 |
0.70 |
Yes |
-- | ||
|
Number of courses passed |
Louisiana Opening Doors Program vs. Business as usual |
1 Semester |
First 2 cohorts;
|
1.10 |
0.80 |
Yes |
-- | ||
|
Number of regular credits earned |
Louisiana Opening Doors Program vs. Business as usual |
2 Semesters |
First 2 cohorts;
|
1.40 |
1.20 |
No |
-- | ||
Characteristics of study sample as reported by study author.
| Asian | 0% | |
| Black | 85% | |
| Other or unknown | 4% | |
| White | 11% |
| Hispanic | 3% | |
| Not Hispanic or Latino | 97% |
The study was set in two community colleges in New Orleans, Louisiana: Delgado Community College (City Park and West Bank campuses) and Louisiana Technical College-West Jefferson (LTC). It is worth noting that the study began enrolling participants in Fall 2003, and Hurricane Katrina hit in Fall 2005, devastating campus buildings at all three campuses as well as the homes of students. Enrollment at the two Delgado campuses, where 80% of the sample were enrolled, began to bounce back after the hurricane, reaching pre-Katrina levels at the West Bank campus but only about two-thirds of the prior number at the City Park, which was even more adversely affected.
The majority of the study participants were female (94.5% Delgado, 84.2% Louisiana Tech), unmarried and not living with a partner (77.0% Delgado, 65.5% Louisiana Tech), and Black (84.8% Delgado, 85.2% Louisiana Tech). The average age was 24.9 at Delgado and 27.0 at Louisiana Tech. Few participants were dependent on their parents (17.9% Delgado, 14.4% Louisiana Tech). Just over half were currently employed (51.4% Delgado, 52.5% Louisiana Tech). These characteristics are based on the full 1,019 participants in the study (reported in 2009 study). For the 2006 report, program effects were only analyzed for the first two cohorts who entered the study in spring and summer 2004, which represents 53% of the full sample (537 students).
Students in the Opening Doors program received an initial payment of $250 after the program counselors confirmed with the registrar that they had enrolled at least half time. After midterms, those who were verified by counselors to be enrolled at least half time and were still earning a 2.0 GPA received a second payment of $250. A final payment of $500 was made at the end of the semester, after verification that the students had passed their courses and earned a GPA of at least 2.0. The Opening Doors counselors provided treatment participants with personalized attention and oversight by monitoring students’ compliance with program requirements. Counselors were also encouraged to help students resolve problems that interfered with their academic performance, either directly or by referring them to other campus or community resources. The incremental payment of the scholarship gave counselors at least two or three opportunities to talk with students each semester.
The comparison group was considered "business as usual," receiving whatever regular financial aid, including Pell Grants, and counseling was available to all students. They did not receive an Opening Doors scholarship and did not have counselors who monitored their academic performance.
This study was a collaboration between MDRC and its partners in the Network on Transitions to Adulthood, funded by the MacArthur Foundation. The Louisiana Department of Social Services and the Louisiana Workforce Commission agreed to fund and oversee the program, while MDRC assisted the colleges with program implementation and conducted all evaluation activities, including random assignment of students to program and control groups. Program funding came through the Temporary Assistance for Needy Families (TANF) program, and limited the ages of students to 18-34 year olds, and participants had to provide documentation required by state agencies.
Reviewed: March 2017
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|
|---|---|---|---|---|---|---|---|---|---|
|
Cumulative GPA (2.0 and higher) |
Louisiana Opening Doors Program vs. Business as usual |
7 Semesters |
Full sample;
|
50.90 |
44.00 |
Yes |
|
|
|
| Show Supplemental Findings | |||||||||
|
Second semester GPA (2.0 and higher) |
Louisiana Opening Doors Program vs. Business as usual |
1 Semester |
First two cohorts;
|
40.40 |
19.60 |
Yes |
|
||
|
First semester GPA (2.0 and higher) |
Louisiana Opening Doors Program vs. Business as usual |
1 Semester |
First two cohorts;
|
49.00 |
36.90 |
Yes |
|
||
|
Third semester GPA (2.0 and higher) |
Louisiana Opening Doors Program vs. Business as usual |
1 Semester |
First two cohorts;
|
27.10 |
19.20 |
Yes |
|
||
|
Cumulative GPA (2.0 and higher) |
Louisiana Opening Doors Program vs. Business as usual |
4 Semesters |
First two cohorts;
|
47.50 |
36.50 |
Yes |
|
||
|
Fourth semester GPA (2.0 and higher) |
Louisiana Opening Doors Program vs. Business as usual |
1 Semester |
First two cohorts;
|
19.20 |
13.70 |
No |
-- | ||
|
Cumulative GPA (2.0 and higher) |
Louisiana Opening Doors Program vs. Business as usual |
7 Semesters |
First two cohorts;
|
45.60 |
36.90 |
Yes |
|
||
|
Cumulative GPA (2.0 and higher) |
Louisiana Opening Doors Program vs. Business as usual |
2 Semesters |
Full sample;
|
55.00 |
46.90 |
Yes |
|
||
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|
|---|---|---|---|---|---|---|---|---|---|
|
Registered for any courses first semester |
Louisiana Opening Doors Program vs. Business as usual |
1 Semester |
Full sample;
|
82.20 |
76.80 |
Yes |
|
|
|
| Show Supplemental Findings | |||||||||
|
Registered for any course: Second semester |
Louisiana Opening Doors Program vs. Business as usual |
1 Semester |
Delgado Community College: First two cohorts;
|
65.20 |
46.40 |
Yes |
|
||
|
Registered for any course: Third semester |
Louisiana Opening Doors Program vs. Business as usual |
1 Semester |
Delgado Community College: First two cohorts;
|
61.20 |
46.00 |
Yes |
|
||
|
Registered for any course: Second semester |
Louisiana Opening Doors Program vs. Business as usual |
1 Semester |
Delgado Community College;
|
69.50 |
56.70 |
Yes |
|
||
|
Registered for any courses first semester |
Louisiana Opening Doors Program vs. Business as usual |
1 Semester |
Delgado Community College: First two cohorts;
|
78.00 |
72.30 |
No |
-- | ||
|
Registered for any courses first semester |
Louisiana Opening Doors Program vs. Business as usual |
1 Semester |
First two cohorts;
|
74.40 |
69.80 |
No |
-- | ||
|
Registered for any courses first semester |
Louisiana Opening Doors Program vs. Business as usual |
1 Semester |
Delgado Community College;
|
84.50 |
81.00 |
No |
-- | ||
|
Registered for any course: Fourth semester |
Louisiana Opening Doors Program vs. Business as usual |
1 Semester |
Delgado Community College: First two cohorts;
|
35.80 |
30.10 |
No |
-- | ||
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|
|---|---|---|---|---|---|---|---|---|---|
|
Number of regular credits earned |
Louisiana Opening Doors Program vs. Business as usual |
7 Semesters |
Full sample;
|
11.50 |
9.10 |
Yes |
|
|
|
| Show Supplemental Findings | |||||||||
|
Number of regular credits earned |
Louisiana Opening Doors Program vs. Business as usual |
2 Semesters |
Full sample;
|
7.10 |
5.50 |
Yes |
|
||
|
Number of regular credits earned |
Louisiana Opening Doors Program vs. Business as usual |
4 Semesters |
First two cohorts;
|
9.50 |
6.70 |
Yes |
|
||
|
Number of credits earned during first semester |
Louisiana Opening Doors Program vs. Business as usual |
1 Semester |
First two cohorts;
|
3.20 |
2.20 |
Yes |
|
||
|
Number of regular credits earned |
Louisiana Opening Doors Program vs. Business as usual |
7 Semesters |
First two cohorts;
|
11.20 |
7.90 |
Yes |
|
||
|
Number of credits earned during the second semester |
Louisiana Opening Doors Program vs. Business as usual |
1 Semester |
First two cohorts;
|
2.40 |
1.50 |
Yes |
|
||
|
Number of credits earned during third semester |
Louisiana Opening Doors Program vs. Business as usual |
1 Semester |
First two cohorts;
|
2.50 |
1.80 |
Yes |
|
||
|
Number of credits earned during fourth semester |
Louisiana Opening Doors Program vs. Business as usual |
1 Semester |
First two cohorts;
|
1.40 |
1.20 |
No |
-- | ||
Characteristics of study sample as reported by study author.
The study took place at two New Orleans, Louisiana-based community colleges: Delgado Community College and Louisiana Technical College-West Jefferson.
The majority of the study participants were female (94.5% Delgado, 84.2% Louisiana Tech), unmarried and not living with a partner (77.0% Delgado, 65.5% Louisiana Tech), and Black (84.8% Delgado, 85.2% Louisiana Tech). The average age was 24.9 at Delgado and 27.0 at Louisiana Tech. Few participants were dependent on their parents (17.9% Delgado, 14.4% Louisiana Tech). Just over half were currently employed (51.4% Delgado, 52.5% Louisiana Tech). These characteristics are based on the full 1,019 participants in the study (reported in 2009 study). For the 2006 report, program effects were only analyzed for the first two cohorts who entered the study in spring and summer 2004, which represents 53% of the full sample (537 students).
Students in the Opening Doors program received an initial payment of $250 after the program counselors confirmed with the registrar that they had enrolled at least half time. After midterms, those who were verified by counselors to be enrolled at least half time and were still earning a 2.0 GPA, received a second payment of $250. A final payment of $500 was made at the end of the semester, after verification that the students had passed their courses and earned a GPA of at least 2.0. The Opening Doors counselors provided treatment participants with personalized attention and oversight by monitoring students’ compliance with program requirements. Counselors were also encouraged to help students resolve problems that interfered with their academic performance, either directly or by referring them to other campus or community resources. The incremental payment of the scholarship gave counselors at least two or three opportunities to talk with students each semester.
The control group was considered "business as usual." Participants in the control group received whatever regular financial aid, including Pell Grants, and counseling was available to all students. They did not receive an Opening Doors scholarship and did not have counselors who monitored their academic performance.
This study was a collaboration between MDRC and its partners in the Network on Transitions to Adulthood, funded by the MacArthur Foundation (p. 1, 2009 report). "The Louisiana Department of Social Services and the Louisiana Workforce Commission agreed to fund and oversee the program, while MDRC assisted the colleges with program implementation and conducted all evaluation activities, including random assignment of students to program and control groups," (P. 11, 2009 report). Due to the effects of Hurricane Katrina during the study, outcomes are studied for both Pre-Katrina and Pre to Post Katrina (p. 11, 2009 report). Program funding came through the Temporary Assistance for Needy Families (TANF) program, and limited the ages of students to 18-34 year olds, and participants had to provide documentation required by state agencies (p. 16). The program began as a small pilot program in spring semester-summer 2005 just before Hurricane Katrina struck, and each college appointed an administrator to oversee Opening Doors and hired counselors to work with students. Delgado's City Park campus had two counselors and the other two campuses had one counselor and one administrative assistant (p. 31, 2009 report). The MDRC 2009 reported that there was a lack of leadership support for the Opening Doors program (p. 34), hypothesized to be due to the short-lived nature of the program and the pilot nature of the program.
In the case of multiple manuscripts that report on one study, the WWC selects one manuscript as the primary citation and lists other manuscripts that describe the study as additional sources.
Barrow, L., Richburg-Hayes, L., Rouse, C. E., & Brock, T. (2014). Paying for performance: The education impacts of a community college scholarship program for low-income adults. Journal of Labor Economics, 32(3), 563–599. https://www.journals.uchicago.edu/doi/10.1086/675229.
Barrow, L., Richburg-Hayes, L., Rouse, C. E., & Brock, T. (2012). Paying for performance: The education impacts of a community college scholarship program for low-income adults (Working Paper Series: WP-09-13, 2009). Chicago, IL: Federal Reserve Bank of Chicago. https://www.chicagofed.org/publications/working-papers/2009/wp-13.
Richburg-Hayes, Lashawn; Brock, Thomas; LeBlanc, Allen; Paxson, Christina; Rouse, Cecilia Elena; Barrow, Lisa. (2009). Rewarding Persistence: Effects of a Performance-Based Scholarship Program for Low-Income Parents. MDRC.
An indicator of the effect of the intervention, the improvement index can be interpreted as the expected change in percentile rank for an average comparison group student if that student had received the intervention.
For more, please see the WWC Glossary entry for improvement index.
An outcome is the knowledge, skills, and attitudes that are attained as a result of an activity. An outcome measures is an instrument, device, or method that provides data on the outcome.
A finding that is included in the effectiveness rating. Excluded findings may include subgroups and subscales.
The sample on which the analysis was conducted.
The group to which the intervention group is compared, which may include a different intervention, business as usual, or no services.
The timing of the post-intervention outcome measure.
The number of students included in the analysis.
The mean score of students in the intervention group.
The mean score of students in the comparison group.
The WWC considers a finding to be statistically significant if the likelihood that the finding is due to chance alone, rather than a real difference, is less than five percent.
The WWC reviews studies for WWC products, Department of Education grant competitions, and IES performance measures.
The name and version of the document used to guide the review of the study.
The version of the WWC design standards used to guide the review of the study.
The result of the WWC assessment of the study. The rating is based on the strength of evidence of the effectiveness of the intervention. Studies are given a rating of Meets WWC Design Standards without Reservations, Meets WWC Design Standards with Reservations, or >Does Not Meet WWC Design Standards.
A related publication that was reviewed alongside the main study of interest.
Study findings for this report.
Based on the direction, magnitude, and statistical significance of the findings within a domain, the WWC characterizes the findings from a study as one of the following: statistically significant positive effects, substantively important positive effects, indeterminate effects, substantively important negative effects, and statistically significant negative effects. For more, please see the WWC Handbook.
The WWC may review studies for multiple purposes, including different reports and re-reviews using updated standards. Each WWC review of this study is listed in the dropdown. Details on any review may be accessed by making a selection from the drop down list.
Tier 1 Strong indicates strong evidence of effectiveness,
Tier 2 Moderate indicates moderate evidence of effectiveness, and
Tier 3 Promising indicates promising evidence of effectiveness,
as defined in the
non-regulatory guidance for ESSA
and the regulations for ED discretionary grants (EDGAR Part 77).
Resources
Newsflash
Contact Us
Facebook
Twitter
YouTube