Co-Principal Investigators: Raymond Flemming, Laura Pedrick, Rodney Swain, Simone Conceicao, Cindy Walker

Purpose: The University of Wisconsin at Milwaukee has developed an instructional approach, known as U-PACE, for its introductory psychology course that combines self-paced online learning with mastery learning and amplified assistance (in which the instructor initiates support to student on both course material and to increase student motivation). The goal of this instructional approach is to improve student achievement in these courses and increase student retention. This project will evaluate U-PACE through experiments carried out in introductory courses within psychology, sociology, and political science.

Project Activities: The project will implement experiments in three introductory undergraduate courses (psychology, sociology, and political science) to determine if U-PACE leads to greater academic success and postsecondary retention. Previous pilot work with promising findings has been done in the introductory psychology course. Separate experiments will be done in each course. The first experiment will be done in psychology to determine both whether U-PACE has beneficial impacts, and, if so, which components of U-PACE are the causes of these impacts. Two follow-up studies will be done in sociology and political science to determine if the best form of U-PACE, identified in the first experiment, creates greater student achievement and retention in introductory courses in these two other disciplines.

Products: The evaluation will provide evidence on the efficacy of the U-PACE program and report this evidence in peer reviewed journals.

Structured Abstract

Setting: The project will take place at the University of Wisconsin at Milwaukee (UWM), the major urban public doctoral university in the University of Wisconsin System with over 28,000 undergraduates.

Population: The population is undergraduate students at the University of Wisconsin at Milwaukee. The UWM undergraduate population includes more students who are more likely to be first-generation college students, employed, members of a minority, and require remediation than other University of Wisconsin institutions. Sample selection criteria include students taking introductory course in psychology, sociology, or political science who are over 18 years of age, have not previously failed the introductory course, have not participated in a U-Pace pilot study, and who agree to take part in the study.

Intervention: U-Pace combines self-paced online learning with mastery learning and amplified assistance. In a U-Pace course, students cover the same text chapters as a conventionally taught class but cover them through a self-paced module approach (each module is about half a chapter) and must display mastery of the material in a module before moving to the next one. After each module, students take an electronic, 10-item multiple choice quiz and must score 90% before they can move on to the next module. Students receive their scores and instructor feedback on which concepts they have not learned (but not which questions they missed). Students can retake the quiz an unlimited number of times but they must wait at least one hour between quizzes (to allow time to study) and the questions are changed each time. Instructors initiate the amplified assistance by monitoring student activity in the course quizzes. Instructors then send electronic messages identifying the concepts needed to be learned to pass a quiz and encouragement to continue working on mastering the material.

Control Condition: A "business-as-usual" control group will receive conventional instruction in each course that covers the same material and uses the same textbook and review activities. Its exams will draw from the same pool of questions as used with the treatment group. The control group's instruction will differ in that it will be provided through three 50 minute lectures per week at a pace set by the instructor that does not include determining student mastery of material before allowing them to move to the next topic. It will also not include instructor-initiated personalized feedback on exams (though students may initiate such feedback) or instructor-sent motivational messages.

Research Design and Methods: The researchers will conduct three experiments in which students are randomly assigned to either conventional instruction or U-Pace instruction. This randomization will occur so that academically underprepared students (defined as having a composite ACT score below 19 or an undergraduate GPA below 2.0) and disadvantaged students (defined as having a federal Pell grant or belong to a racial/ethnic group with low postsecondary success) are distributed evenly across the instructional conditions. The instruction will take place in introductory psychology, sociology, and political science courses. Each experiment will have two instructors who will teach all the conditions in both the fall and spring semesters. All instructional conditions will use the same curricular materials. To minimize contamination of instructional method to the other condition(s), instructors will receive intensive training, follow explicit instructional manuals for each instructional approach, and fidelity will be monitored throughout the semesters.

The first experiment will be carried out in Year 1 with 960 first-year undergraduates in introductory psychology assigned to one of four instructional conditions: conventional instruction, U-Pace instruction with the amplified assistance component only, U-Pace instruction with the mastery component only, and U-Pace instruction with both the amplified assistance component and the mastery component. The results of this experiment will determine which form of U-Pace, if any, leads to the highest student gains. This form will be used in the two subsequent experiments. In Year 2, the second experiment will randomly assign 480 students in the introductory sociology course to either the U-Pace condition identified in Experiment 1 or to the control group. In Year 3, the third experiment will randomly assign 480 students in the introductory political science courses to either the U-Pace condition or the control condition.

Key Measures: University and class administration data will be used to collect three measures of student academic success: (1) final course grades, (2) scores on a 50-item multiple-choice cumulative exam that does not contribute to students' grades and is taken by a randomly selected subsample of 120 students in each discipline, and (3) one-year retention rates.

Data Analytic Strategy: For Experiment 1, the outcome on the 50-item multiple choice test will be analyzed through an analysis of variance with pairwise follow-up tests to identify the most efficacious U-PACE treatment. In all three experiments, ordinal logistic regression will be used to analyze the grade outcomes, binary logistic regression will be used to analyze retention, and ANOVA will be used to analyze the multiple-choice test scores. Subgroup analyses will be done for gender, academically unprepared students, and disadvantaged students.

Products and Publications

Book chapter

Reddy, D.M., Fleming, R., Jirovec, D.L., Pedrick, L.E., Pfeiffer, H.M., and Stoiber, L.C. (in press). Increasing Student Success in Higher Education Through UPace Instruction. Higher Learning Commission 28th Annual Collection of Papers on Self-study and Institutional Improvement.

Journal article, monograph, or newsletter

Reddy, D.M., Fleming, R., Pedrick, L.E., Jirovec, D.L., Pfeiffer, H.M., Ports, K.A., Barnack-Tavlaris, J.L., Helion, A.M., and Swain, R.A. (2013). U-Pace Instruction: Improving Student Success by Integrating Content Mastery and Amplified Assistance. Journal of Asynchronous Learning Networks, 17(1): 147–154.

Nongovernment report, issue brief, or practice guide

Reddy, D.M., Fleming, R., and Pedrick, L.E. (2012). Increasing Student Success: Evaluating the Effectiveness of U-Pace Instruction at UWM. Milwaukee, WI: EDUCAUSE.