Shining the Spotlight on Those outside Florida's Reform Limelight: The Impact of Developmental Education Reform for Nonexempt Students [Post-2014 Program]
Mokher, Christine G.; Park-Gaghan, Toby; Hu, Shouping (2021). Grantee Submission. Retrieved from: https://eric.ed.gov/?id=ED611562
-
examining11,440Students, gradePS
Single Study Review
Review Details
Reviewed: January 2024
- Single Study Review (findings for Florida's developmental math education program after the 2014 reform—Mokher et al. (2021))
- Regression Discontinuity Design
- Meets WWC standards with reservations because it is a regression discontinuity design that partially satisfies the integrity of the forcing variable, continuity, and functional form and bandwidth standards and completely satisfies the attrition standard.
This review may not reflect the full body of research evidence for this intervention.
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Findings
|
Outcome measure |
Comparison | Period | Sample |
Intervention mean |
Comparison mean |
Significant? |
Improvement index |
Evidence tier |
|---|---|---|---|---|---|---|---|---|
|
Completion of an advanced math course |
Florida's developmental math education program after the 2014 reform—Mokher et al. (2021) vs. Business as usual |
1 Year |
Students near the threshold of being assigned to upper developmental education or college-level business-as-usual education;
|
28.50 |
34.80 |
Yes |
|
|
|
Completion of Intermediate Algebra |
Florida's developmental math education program after the 2014 reform—Mokher et al. (2021) vs. Other intervention |
1 Year |
Students near the threshold of being assigned to lower developmental education or upper-level developmental education;
|
19.00 |
24.40 |
Yes |
|
Evidence Tier rating based solely on this study. This intervention may achieve a higher tier when combined with the full body of evidence.
Sample Characteristics
Characteristics of study sample as reported by study author.
-
Female: 53%
Male: 47% -
Rural, Suburban, Urban
-
- B
- A
- C
- D
- E
- F
- G
- I
- H
- J
- K
- L
- P
- M
- N
- O
- Q
- R
- S
- V
- U
- T
- W
- X
- Z
- Y
- a
- h
- i
- b
- d
- e
- f
- c
- g
- j
- k
- l
- m
- n
- o
- p
- q
- r
- s
- t
- u
- v
- x
- w
- y
Florida
-
Race Black 20% Other or unknown 39% White 41% -
Ethnicity Hispanic 32%
Study Details
Setting
The study took place in all 28 institutions of the Florida College System.
Study sample
A total of 11,440 first-time-in-college students in Florida were included in the study. Specifically, the data are from the 2014 and 2015 entering cohorts in all 28 institutions of the Florida College System. The sample consists of students who were eligible for the Florida developmental education program according to eligibility requirements as they existed in 2014 and later. In particular, students must have met at least one of three eligibility criteria to be included in the analysis: (1) not have a transcript record from a Florida public high school, (2) not have completed a standard high school diploma, or (3) have a standard high school diploma prior to 2006-2007. The study excluded students who did not have placement scores on the Postsecondary Education Readiness Test (PERT) or had incomplete high school administrative records. Sample characteristics were not available for the analytic sample. Among all non-exempt students across four student cohorts, 53% were female. Forty-one percent were White, 20% were Black, and 39% did not report race. Thirty-two percent were Hispanic or Latino, and the rest did not report ethnicity.
Intervention Group
This study examines Florida's developmental math education program, which required entering college students to take developmental math courses if they scored below college-ready in a placement exam. This review focuses on the findings for the reformed version of the program that was in place starting in 2014. Similar to the pre-reform version of the program, the post-reformed version assigned students to take either lower- or upper- developmental education courses for one semester depending on one's placement score. The lower-level course is the first course in a college-preparatory two-course sequence emphasizing fundamental operations with applications to beginning algebra. The upper-level course is the second course in the college preparatory sequence which provides algebraic background on topics including fundamental operations with polynomials, linear equalities, factoring, and an introduction to radicals. Students assigned to the intervention were required to take these development courses before they could take college-level math courses. The post-reform version of the program required colleges to offer development courses in a way that was intended to allow students to complete them more quickly, such as shorter courses. The post-reform version of the program also included access to additional advising services and academic supports, such as online orientation resources, longer advising sessions, and online tutoring. The outcomes were measured after students' first year in college. The manuscript also includes findings that measure the impact of the pre-reform version of the program, and another set of findings that compare the post-reform version of the program to the pre-reform version. The WWC examined these findings and reported its conclusions in two other separate reviews of this manuscript. The WWC reviewed them separately because they focus on measuring the impact of a different intervention or use a different study design and analysis method.
Comparison Group
The comparison condition differed for the two levels of the developmental education intervention: (1) students assigned to upper-developmental education courses were compared to students who were placed into the business-as-usual college-level curriculum without developmental education courses, and (2) students assigned to lower-developmental education courses were compared to students who were assigned to upper-developmental education courses.
Support for implementation
Schools provided additional advising and academic support services to all incoming students. This took the form of more online orientation resources and advising sessions. These were implemented by adopting online tutoring programs and adding or increasing faculty time in success centers or tutoring. Schools also identified and spent additional time with at-risk students.
An indicator of the effect of the intervention, the improvement index can be interpreted as the expected change in percentile rank for an average comparison group student if that student had received the intervention.
For more, please see the WWC Glossary entry for improvement index.
An outcome is the knowledge, skills, and attitudes that are attained as a result of an activity. An outcome measures is an instrument, device, or method that provides data on the outcome.
A finding that is included in the effectiveness rating. Excluded findings may include subgroups and subscales.
The sample on which the analysis was conducted.
The group to which the intervention group is compared, which may include a different intervention, business as usual, or no services.
The timing of the post-intervention outcome measure.
The number of students included in the analysis.
The mean score of students in the intervention group.
The mean score of students in the comparison group.
The WWC considers a finding to be statistically significant if the likelihood that the finding is due to chance alone, rather than a real difference, is less than five percent.
The WWC reviews studies for WWC products, Department of Education grant competitions, and IES performance measures.
The name and version of the document used to guide the review of the study.
The version of the WWC design standards used to guide the review of the study.
The result of the WWC assessment of the study. The rating is based on the strength of evidence of the effectiveness of the intervention. Studies are given a rating of Meets WWC Design Standards without Reservations, Meets WWC Design Standards with Reservations, or >Does Not Meet WWC Design Standards.
A related publication that was reviewed alongside the main study of interest.
Study findings for this report.
Based on the direction, magnitude, and statistical significance of the findings within a domain, the WWC characterizes the findings from a study as one of the following: statistically significant positive effects, substantively important positive effects, indeterminate effects, substantively important negative effects, and statistically significant negative effects. For more, please see the WWC Handbook.
The WWC may review studies for multiple purposes, including different reports and re-reviews using updated standards. Each WWC review of this study is listed in the dropdown. Details on any review may be accessed by making a selection from the drop down list.
Tier 1 Strong indicates strong evidence of effectiveness,
Tier 2 Moderate indicates moderate evidence of effectiveness, and
Tier 3 Promising indicates promising evidence of effectiveness,
as defined in the
non-regulatory guidance for ESSA
and the regulations for ED discretionary grants (EDGAR Part 77).