|Title:||Using Computer-Assisted Instruction to Accelerate Students through Developmental Math: An Impact Study of Modularization and Compression|
|Principal Investigator:||Weiss, Michael||Awardee:||MDRC|
|Program:||Postsecondary and Adult Education [Program Details]|
|Award Period:||4 years (7/1/13-6/30/17)||Award Amount:||$3,289,513|
|Type:||Efficacy and Replication||Award Number:||R305A130125|
Purpose: The purpose of this project is to evaluate the impact of two distinct interventions for community college students whose math skills are below college level. The first intervention is a modularized, computer-assisted curriculum, called ModMath, that allows students to move at their own pace through the math curriculum. The second intervention is a state-wide course-placement policy that routes students who score very low on a placement test into adult basic education (ABE) services at the community college rather than the lowest level of developmental math.
Both of these interventions are gaining popularity in community colleges and state systems. Self-paced, modularization programs, such as the ModMath program, are found in many colleges, and more states are joining Texas in the effort to redefine the floor for developmental math by simply referring students to ABE programs. However, the impact of these innovations is unclear. Over the course of this project, researchers will study their implementation and gather data to provide evidence about their impact on student outcomes such as academic progress as measured by credit attainment and passing the first college-level math course.
Project Activities: Researchers are using a randomized controlled trial to study the impact of ModMath and a regression discontinuity design to study the impact of Texas policy on college’s use of ABE referrals for low-scoring students. Researchers will track students in both studies for at least two semesters, and they will collect implementation data for each as well.
Products: Findings from both studies will be reported periodically in MDRC publications that will be available on the MDRC website and authors will submit an article to a peer-reviewed journal.
Setting: This project will take place in publicly funded community colleges in Texas (one community college system for the ModMath study, and two community college systems for the ABE study).
Sample: Approximately 1,200 students will participate in the ModMath Study, and approximately 4,000 students will participate in the ABE Study.
Intervention: The first intervention, ModMath, is a computer-assisted instructional approach that has the potential of accelerating students’ progress through the developmental math sequence. ModMath divides the developmental math curriculum into discrete computerized modules, allowing students to learn at their own pace and spend less time on skills they either already know or can master quickly. Students work through modules identified by an assessment and then advance as quickly as they are capable of learning the material. ModMath classrooms are set up like computer labs, with an instructor who circulates among the students to offer assistance when needed.
The second intervention consists of a set of rules and guidelines developed by the Texas Higher Education Coordinating Board and is scheduled to go into full effect for the fall 2015 cohort. They require colleges to route low-scoring students into ABE offerings, which vary by college, rather than developmental education. Scores on the new placement test in Texas, the Texas Success Initiative Assessment (TSIA), first divide students into two groups: college-ready and below college-ready. Those who score below the college-ready cutoff are immediately given a second test, the Adult Basic Education Diagnostic. Students who test into the top two levels (Levels 6 and 5) are referred to developmental education courses; students who score at the middle two levels (4 and 3) are referred to ABE programs associated with the community college; and students at the bottom two levels (1 and 2) are referred to other programs, e.g., either federally funded ABE programs in the community, workforce programs on campus, or other alternatives.
Research Design and Methods: The research team will conduct two separate studies of these two interventions, using different methodologies.
For the ModMath Study, the team is carrying out a randomized control trial to estimate the impact of ModMath on student outcomes and to determine if impacts vary by student characteristics. Researchers are also gathering data on the implementation of ModMath to determine the degree to which it is implemented with fidelity and to describe how the ModMath experience differs from "business as usual" —lecture based math instruction.
For the ABE Study, researchers will use a regression discontinuity design to compare students whose scores on the ABE Diagnostic place them at the cusp between developmental education and ABE. Students just below the cut score are referred to ABE and students just above are referred to developmental math. The two groups are assumed to be similar in their math abilities and, therefore, any difference in their outcomes can be attributed to their different post-placement experiences. This study will also gather information about the implementation of the ABE interventions, and the enrollment behaviors of students.
Control condition: For the ModMath Study, the control group students are referred to traditional, lecture-based developmental math classes. These semester-long classes meet throughout the semester. For the ABE Study, students who are referred to developmental math serve as the control group while students who are referred to ABE serve as the program group.
Key Measures: Researchers will obtain primary outcome information for both studies through college transcript records. Key measures include students’ academic progress in mathematics and non-mathematics courses (e.g., completing the ABE or developmental math course, passing the first college-level math course, credit accumulation in mathematics and overall). The research team will also collect data on implementation fidelity and the treatment contrast through interviews with faculty and students and classroom observations. For the ModMath Study, researchers will also use student and faculty surveys as implementation measures.
Data Analytic Strategy: For both the ModMath Study and the ABE Study, researchers will compute intent-to-treat estimates by comparing average outcomes of the program group and control group members and those of selected subgroups (e.g., part-time versus full-time status, high versus low familiarity with technology, gender, minority status). Because the ABE study will take place in two community college systems, the researchers will analyze the data from each system separately.
Weiss, M. J. & Headlam, C. (2019). A randomized controlled trial of a modularized, computer-assisted, self-paced approach to developmental math. Journal of Research on Educational Effectiveness, 12(3), 484-513. doi: 10.1080/19345747.2019.1631419.
Nongovernment report, issue brief, or practice guide
Boatman, A., Cerna, O., Reiman, K., Diamond, J., Visher, M.G., & Zachry-Rutschow, E. (in press). Building A New “Bridge” to Math: A Study of a Transition Program Serving Students with Low Math Skills at a Community College. MDRC, NY.
Cerna, O. (in press). Building Basic Math Skills: Boot Camp at Tarrant County College. MDRC, NY.
Gardenhire, A., Diamond, J., Headlam, C., & Weiss, M. J. (2016). At Their Own Pace: Interim Findings from an Evaluation of a Computer-Assisted, Modular Approach to Developmental Math. MDRC.
Weiss, M.J., Headlam, C. (2018). A Randomized Controlled Trial of a Modularized, Computer-Assisted, Self-Paced Approach to Developmental Math. MDRC, NY. Full Text
Visher, M., Cerna, O., Diamond, J., & Rutschow, E. Z. (2017). Raising the Floor: New Approaches to Serving the Lowest-Skilled Students at Community Colleges in Texas and Beyond. MDRC. Full Text