Skip Navigation
Funding Opportunities | Search Funded Research Grants and Contracts

IES Grant

Title: Impact of an Orientation Course on Online Students' Completion Rates
Center: NCER Year: 2017
Principal Investigator: Zweig, Jacqueline Awardee: Education Development Center, Inc.
Program: Low-Cost, Short-Duration Evaluation of Education Interventions      [Program Details]
Award Period: 2 years (9/1/2017-8/31/2019) Award Amount: $238,867
Award Number: R305L170008

Co-Principal Investigator(s):  Stephen Best (Michigan Department of Education) and Makoto Hanita (EDC)

Partners: EDC, Michigan Department of Education, and Michigan Virtual Schools

Purpose: Michigan high school students are required to complete one online course before graduating, and may take up to two online courses per term from a statewide catalog of courses. In 2014-15, 91,000 students took an online course with 9,200 enrolling in the Michigan Virtual School (MVS) - a state-supported school offering online courses to 500 schools. Previous research found that MVS online courses have lower completion rates (77%) than in-person courses (91%), and that completion rates are lower for core courses (73% for math, English language arts, science, social studies, and languages) than for non-core courses (84% for courses aligned with career pathways as well as general elective credits). MVS has adapted an existing online orientation tool into an online course to better prepare students for online learning, provide strategies that help students stay on track in their online courses, and increase their engagement in those courses. MVS and the Michigan Department of Education (MDE) want to determine if the new orientation course improves student course completion rates.

Project Activities:  This partnership will carry out a randomized controlled trial to evaluate the impacts of an online orientation course to prepare high school students for taking online courses on course completion. MVS will implement the online orientation course; MVS and MDE will provide course data and student administrative data, respectively; and EDC will perform the analysis. The partners will work together to disseminate the results.

Products: Researchers will provide evidence of the efficacy of the MVS orientation course for high school online courses. A briefing on the results will be held at the MDE. The findings will also be provided to other practitioners, online course providers, and researchers through a written brief made available online, and through presentations and publications for both practitioner and academic professional organizations and journals.

Structured Abstract

Setting: The project will take place in Michigan.

Sample:  The project includes Michigan high school students taking their first online course offered by MVS in the spring semester of school year 2017-18 (about 3300 students in over 350 school districts).

Intervention: The two-hour orientation course to online learning contains four modules: (1) technical skills, (2) learning skills, (3) strategies for self-regulation and engagement, and (4) basic organization including familiarity with the online environment. The course will be offered on Blackboard (the MVS learning management system) and include interactive assessments of motivation, self-regulation, and preparation for online learning.

Research Design and Methods: A randomizedcontrolled trial will be used. High school students enrolling for their first MVS online course in the spring semester of 2017-18 will be randomly assigned to receive the two-hour orientation before starting their online course (treatment group) or to start their online course without the orientation (control group). Randomization will be done within two blocks (students taking online core courses and students taking online non-core courses) with the majority expected to be in the core course block.

Control Condition:  Control students will take their online courses without having access to the online orientation course. They will have access to an orientation tool that has traditionally been available to all online students to help them navigate online courses which they can use or ignore.

Key Measures: The key outcome measure is student completion of their MVS online course. Completion is defined by MVS as earning 60% or more of total course points. For the online orientation course, student logins, module completion, and overall completion will be collected in order to examine student use of and successful completion of the orientation course.

Data Analytic Strategy: Researchers will check baseline equivalence of the treatment and control groups will be checked using student administrative data. A fidelity of implementation analysis regarding student use of the online course will address logging into the orientation course (number of times overall and if weekly), completing the assessment for each module of the course, and receiving a completion certificate for the course. The impact analysis will be based on a logistic regression estimating the impact of the orientation course on online course completion overall and for core versus non-core course taking. An interaction term between treatment status and a binary indicator for core course will be added to the model to test whether the impact differs for students in core and non-core courses. Also, impacts will be examined by grade. If the fidelity of implementation study shows lower than expected online course completion, a treatment-on-the-treated analysis will be carried out.