Using Computer-Assisted Instruction to Accelerate Students through Developmental Math: An Impact Study of Modularization and Compression

Pre-registration site: The ModMath study's pre-analysis plan can be found at https://sreereg.icpsr.umich.edu/sreereg/subEntry/2351/pdf?section=all&action=download.

Main findings of the ModMath study (Weiss and Headlam, 2019)

• ModMath was well-implemented.
• Researchers did not find evidence that ModMath was superior to the "traditional" non-modularized direct-instruction math course.

Main findings from the ABE study (Boatman et al., 2020)

• Regardless of remedial course referral type (traditional developmental or ABE alternative), 50 percent of enrolled students did not attempt any math remediation upon entering college.
• Compliance with course referrals was low. The full population of test-takers with math skills ranging from 4th grade to 8th grade were less likely to be enrolled at HCC and less likely to enroll in their assigned math class than students with math skills ranging from 9th grade to 12th grade, both in the first semester, but also up to two years after taking the placement test.
• Among enrolled students, few enrolled in college-level math by the end of four semesters. Only 17 percent of students assigned to developmental math enrolled in a college-level math course by the end of 4 semesters, compared to 9 percent of students assigned to ABE.
• Approximately 50 percent of students assigned to ABE enrolled in the ABE course within 4 semesters. However, only 32 percent of students eventually passed the course within that time frame. Of the students assigned to developmental math, 63 percent enrolled in the course by the end of 4 semesters, and 33 percent of these students had passed the course within 4 semesters. There was a five percent difference in the percent of ABE students compared to developmental math students passing college-level math by the end of four semesters.

The second intervention consisted of a set of policy guidelines developed by the Texas Higher Education Coordinating Board. They required colleges to students with low scores on the statewide math placement exam into ABE offerings, which vary by college, rather than developmental education. Scores on the new placement test in Texas, the Texas Success Initiative Assessment (TSIA), first divided students into two groups: college-ready and below college-ready. Those who score below the college-ready cutoff were given a second set of items, the Adult Basic Education Diagnostic. Students who tested into the top two levels (Levels 6 and 5) were referred to developmental education courses; students who score at the middle two levels (4 and 3) were referred to ABE programs associated with the community college; and students at the bottom two levels (1 and 2) were referred to other programs, e.g., either federally funded ABE programs in the community, workforce programs on campus, or other alternatives.

For the ModMath Study, the team carried out a randomized control trial to estimate the impact of ModMath on student outcomes and to determine if impacts vary by student characteristics. Researchers also gathered data on the implementation of ModMath to determine the degree to which it was implemented with fidelity and to describe how the ModMath experience differs from "business as usual" —lecture-based math instruction.

For the ABE Study, researchers attempted a regression discontinuity design (RDD) to compare students whose scores on the ABE Diagnostic place them at the cusp between developmental education and ABE. Conducting an RDD analysis in this context requires a continuous test score used to independently assign students to the treatment (ABE) or control condition (developmental math). However, due to missing data, the researchers had to calculate a "predicted test score" in instances where the actual test score was missing. Because the predicted scores failed empirically testable assumptions of the regression discontinuity design (many laid out in the What Works Clearinghouse (WWC) standards for RDD validity), the attempted RDD did not prove fruitful, and descriptive analyses of students' progression through ABE and developmental math classes were conducted.

For the ABE Study, researchers attempted a regression discontinuity design (RDD) to compare students whose scores on the ABE Diagnostic place them at the cusp between developmental education and ABE. Conducting an RDD analysis in this context requires a continuous test score used to independently assign students to the treatment (ABE) or control condition (developmental math). However, due to missing data, the researchers had to calculate a "predicted test score" in instances where the actual test score was missing. Because the predicted scores failed empirically testable assumptions of the regression discontinuity design (many laid out in the What Works Clearinghouse (WWC) standards for RDD validity), the attempted RDD did not prove fruitful, and descriptive analyses of students' progression through ABE and developmental math classes were conducted.