Skip Navigation
archived information
Skip Navigation

Back to Ask A REL Archived Responses

REL Midwest Ask A REL Response

Online Courses

January 2019

Question:

What does the research say about practices to authorize, monitor, and oversee virtual schools?



Response:

Following an established Regional Educational Laboratory (REL) Midwest protocol, we conducted a search for research reports and descriptive studies on practices for authorizing, monitoring, and overseeing virtual schools. For details on the databases and sources, keywords, and selection criteria used to create this response, please see the Methods section at the end of this memo.

Below, we share a sampling of the publicly accessible resources on this topic. References are listed in alphabetical order, not necessarily in order of relevance. The search conducted is not comprehensive; other relevant references and resources may exist. For each reference, we provide an abstract, excerpt, or summary written by the study’s author or publisher. We have not evaluated the quality of these references, but provide them for your information only.

Research References

Gemin, B., & Pape, L. (2016). Keeping pace with K–12 online learning. Evergreen, CO: Evergreen Education Group. Retrieved from https://eric.ed.gov/?id=ED576762

From the ERIC abstract: “‘Keeping Pace with K–12 Online Learning 2016’ marks the thirteenth consecutive year Evergreen has published its annual research of the K-12 education online learning market. The thirteen years of researching, writing and publishing this report represents a time of remarkable change. There has been a constant presence that has become the backbone, supporting the growth and success of online learning—the array of organizations that supply online courses, online teachers, digital content and tools to schools. The number and breadth of types of suppliers has changed and grown as the demand for broader and deeper services has increased. Suppliers range from schools that supply regions or whole states, to stand-alone ‘intermediate’ suppliers that provide online courses and related services to schools, to vendors who develop courses and content and deliver their courses directly to schools or distribute them through intermediates. ‘Keeping Pace with K-12 Online Learning 2016’ focuses on these suppliers of online learning and reports on levels and types of activity, including online course enrollments, types of enrollments and number of students involved in online learning.”

Gill, B., Walsh, L., Wulsin, C., Matulewicz, H., Severn, V., Grau, E., et al. (2015). Inside online charter schools. A report of the National Study of Online Charter Schools. Cambridge, MA: Mathematica Policy Research. Retrieved from https://eric.ed.gov/?id=ED560967

From the ERIC abstract: “Online charter schools—also known as virtual charters or cyber charters—are publicly funded schools of choice that eschew physical school buildings and use technology to deliver education to students in their own homes. These schools typically provide students with computers, software, and network-based resources, while also providing access to teachers via email, telephone, web, and/or teleconference. Online charter schools deliver instruction using a radically different approach than conventional public schools. Nonetheless, critics of online charter schools worry that they might not be effective in promoting student learning. This report and its companion volumes describe the findings of the most ambitious and comprehensive study of online charter schools to date, conducted jointly by Mathematica Policy Research, the Center for Research on Education Outcomes (CREDO) at Stanford University, and the Center on Reinventing Public Education (CRPE) at the University of Washington. This volume begins with a snapshot of online charter schools operating across the country, describing their numbers, the states in which they operate, and the students they serve. The report then describes the instructional programs of online charter schools; methods used to engage students and parents, along with expectations of parental involvement; the teachers and principals of online charter schools; and their management and governance. Key findings include: (1) Student-driven, independent study is the dominant mode of learning in online charter schools, with 33 percent of online charter schools offering only self-paced instruction; (2) Online charter schools typically provide students with less live teacher contact time in a week than students in conventional schools have in a day; (3) Maintaining student engagement in this environment of limited student-teacher interaction is considered the greatest challenge by far, identified by online charter school principals nearly three times as often as any other challenge; (4) Online charter schools place significant expectations on parents, perhaps to compensate for limited student-teacher interaction, with 43, 56, and 78 percent of online charters at the high school, middle, and elementary grade levels, respectively, expecting parents to actively participate in student instruction; and (5) These findings suggest reason for concern about whether the online charter school sector is likely to be effective in promoting the achievement of its students.”

Jack, J., Sludden, J., & Schott, A. (2013). An analysis of Pennsylvania’s cyber charter schools (Issue Brief). Philadelphia, PA: Research for Action. Retrieved from https://eric.ed.gov/?id=ED553143

From the ERIC abstract: “Pennsylvania’s first cyber charter school opened in 1998, enrolling 44 full-time students. From this modest beginning, Pennsylvania’s cyber charter sector has grown to 16 schools enrolling 35,000 students from all but one school district in the Commonwealth. Pennsylvania has one of the nation’s most extensive cyber charter sectors, and six additional proposed schools are before the Department of Education (PDE) for review. The recent growth in the sector coincides with increasing attention from state policymakers. As of this writing, there are at least 12 legislative proposals pending in the Pennsylvania General Assembly addressing cyber authorization, funding, or oversight. To help situate cyber charter schools within a state context, Research for Action (RFA) examined the state’s recently-issued School Performance Profile (SPP) scores for the 11 cyber charters for which complete data are available. RFA then compared these scores to all public schools statewide, including traditional public schools and brick-and-mortar charters. RFA’s analysis is based on publicly-available data from PDE’s School Performance Profile. Also, given the relationship between student mobility and academic achievement, RFA examined enrollment into and out of five cyber charters for which data were available during the 2010–11 and 2011–12 school years. RFA reviewed the demographics of the student populations in cyber charter schools as compared to traditional public schools and brick-and-mortar charters.”

Lin, M. (2011). School quality in the cloud: Guidelines for authorizing virtual charter schools (Issue Brief). Chicago, IL: National Association of Charter School Authorizers. Retrieved from https://eric.ed.gov/?id=ED544280

From the ERIC abstract: “This Issue Brief, an update of ‘Authorizing Virtual Charter Schools: Rules of the Road on the Digital Highway’ by Gregg Vanourek, is part of NACSA’s Cyber series, which addresses issues in policy and practice that concern authorizing online schools and blended learning. It aims to improve authorizer understanding and oversight of online charter schools generally, with particular focus on strengthening authorizer practices in evaluating proposals for virtual charter schools. The brief aims to improve authorizer understanding and oversight of online charter schools generally, with particular focus on strengthening authorizer practices in evaluating proposals for virtual charter schools. Provided is guidance to authorizers in evaluating proposals for online charter schools—including identifying key application questions and review practices to evaluate virtual charter applicants’ plans and capacities. The brief concludes with general recommendations for overseeing and evaluating online charter schools. A list of selected sources is included.”

Molnar, A., Miron, G., Gulosino, C., Shank, C., Davidson, C., Barbour, M., et al. (2017). Virtual schools in the U.S. 2017. Boulder, CO: National Education Policy Center. Retrieved from https://eric.ed.gov/?id=ED574702

From the ERIC abstract: “In the five years since the first National Education Policy Center (NEPC) ‘Annual Report on Virtual Education’ was released in 2013, virtual education has continued to be a focal point for policymakers. Proponents argue that virtual education can expand student choices and improve the efficiency of public education. In particular, full-time virtual schools (also sometimes referred to as virtual charter schools, virtual academies, online schools or cyber schools) have attracted a great deal of attention. Many believe that online curriculum can be tailored to individual students more effectively than curriculum in traditional classrooms, giving it the potential to promote greater student achievement than can be realized in traditional brick-and-mortar schools. Further, the promise of lower costs—primarily for instructional personnel and facilities—makes virtual schools financially appealing to both policymakers and for-profit providers. The assumption that virtual schools are cost effective and educationally sound, coupled with policies expanding school choice and providing market incentives attractive to for-profit companies, continue to help fuel virtual school growth in the U.S. There is, however, little high-quality systematic evidence that the rapid expansion of the past several years is wise. Indeed, evidence presented in the NEPC annual reports argues for caution. Nevertheless, the movement toward virtual schools continues to gather steam, often supported by weak or even dishonest data. For example, as a part of the confirmation hearings for the current Secretary of Education, National Public Radio reported that Secretary Betsy DeVos responded to a written question from Senator Patty Murray using performance data provided by a for-profit corporation that inflated the four-year graduation rates of virtual schools—in some cases by as much as 300%. The 2017 NEPC Annual Report contributes to the existing evidence related to virtual education, and so to debates surrounding it. It provides objective analysis of the characteristics and performance of full-time, publicly funded K–12 virtual schools; available research on virtual school practices and policy; and an overview of recent state efforts to craft new policy.”

National Alliance for Public Charter Schools. (2016). A call to action: To improve the quality of full-time virtual charter public schools. Washington, DC: Author. Retrieved from https://eric.ed.gov/?id=ED568917

From the ERIC abstract: “Currently, more than 180,000 students attend 135 full-time virtual charter schools in 23 states and the District of Columbia. While some students do well in a full-time virtual charter school environment, too many of these schools are not providing a quality educational program to the vast majority of their students, while enrolling too many who are simply not a good fit for attending a fully online school. In this report, ‘A Call to Action to Improve the Quality of Full-Time Virtual Charter Public Schools,’ the National Alliance for Public Charter Schools, the 50-State Campaign for Achievement Now (50CAN) and the National Association of Charter School Authorizers (NACSA) propose specific policy recommendations to help states better hold full-time virtual charter schools accountable for student results.”

Pazhouh, R., Lake, R., & Miller, L. (2015). The policy framework for online charter schools. Seattle, WA: Center on Reinventing Public Education. Retrieved from https://eric.ed.gov/?id=ED560949

From the ERIC abstract: “Online charter schools, charter schools that primarily utilize remote online instruction, have been both popular and controversial. As of October 2015, fully online charter schools operate in 26 (soon to be 27) states and enrolled approximately 200,000 students in the 2013–14 school year, comprising over 8 percent of all public charter school enrollments nationwide. Despite their growing popularity with students and families, online charter schools have received their share of bad press, thanks to reports of lackluster student outcomes and high-profile controversies over contracting for school management services with for-profit providers. This paper looks at the question of how state policy shapes the online charter school landscape. The authors reviewed the legal framework affecting online schools that operate as charter schools (as opposed to online schools operating on contract with a district or the state). The authors wanted to know: (1) In what ways do state laws vary?(2) How might that policy variation influence the ability of online charter schools to operate efficiently and effectively?and (3) What are emerging trends in state policy and how might charter online laws be improved?Overall, it was found that states’ charter laws and administrative regulations create varied policy environments for online charter schools. With few exceptions, the regulatory framework targeting online charter schools is layered on to the overarching state charter law, which falls under the general education code. Moreover, the analysis shows that existing regulation arises in response to problems and concerns, not as a proactive set of policies to guide the unique opportunities and challenges of online charter schools. The following are appended: (1) State Law Resources; and (2) Major Laws Passed.”

Pazzaglia, A. M., Clements, M., Lavigne, H. J., & Stafford, E. T. (2016). An analysis of student engagement patterns and online course outcomes in Wisconsin (REL 2016-147). Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Midwest. Retrieved from https://eric.ed.gov/?id=ED566960

From the ERIC abstract: “Student enrollment in online courses has increased in the past 15 years and continues to grow. However, little is known about students’ education experiences or online course outcomes. These are areas of particular interest to the Midwest Virtual Education Research Alliance, whose goal is to understand how to support student success in online courses. Members of the alliance partnered with Regional Educational Laboratory Midwest to develop and conduct this study on how students engage in online learning and how student engagement patterns are associated with online course outcomes. Findings from this study may help inform policymakers, state and local education agencies, and online learning providers as they seek ways to support student success in online courses. This study analyzed learning management system data and student information system data for all core, elective, and Advanced Placement online high school course enrollments during the fall 2014 semester. The data were collected by Wisconsin Virtual School, a state-level online learning program that partnered with 194 Wisconsin districts to serve 5,511 student enrollments in 256 supplemental online courses during the 2014/15 school year. Analyses looked for student engagement patterns in online courses and the percentage of student enrollments that followed each pattern; differences among student engagement groups (groups of student enrollments that followed a given pattern) in course type taken, gender, or grade level; and associations between student engagement in online learning and online course outcomes. Engagement refers to behavioral engagement and was defined as the amount of time a student was logged in to the online course each week. Course outcomes were measured by the percentage of possible points earned in the course (which students’ home schools use to assign a letter grade based on the local grading scale) and the percentage of course activities completed. Key findings include: (1) Student enrollments in online courses followed one of six engagement patterns, with average engagement ranging from 1.5 hours to 6 or more hours per week; (2) Most students (77 percent) steadily engaged in their online course for 1.5 or 2.5 hours per week; (3) Students who engaged in their online course for at least 1.5 hours per week typically earned a high enough percentage of possible points to pass the course; and (4) Students who engaged in their online course for two or more hours per week had better course outcomes than students who engaged for fewer than two hours per week.”

Queen, B., & Lewis, L. (2011). Distance education courses for public elementary and secondary school students, 2009–10 (NCES 2012-008). U.S. Department of Education, National Center for Education Statistics. Washington, DC: Government Printing Office. Retrieved from https://eric.ed.gov/?id=ED526879

From the ERIC abstract: “This report provides national estimates about student enrollment in distance education courses in public school districts. The estimates presented in this report are based on a district survey about distance education courses offered by the district or by any of the schools in the district during the 12-month 2009–10 school year. For this survey, distance education courses were defined as courses offered to elementary and secondary school students regularly enrolled in the district that meet all of the following criteria: (1) are credit granting; (2) are technology delivered; and (3) have the instructor in a different location than the students and/or have course content developed in, or delivered from, a different location than that of the students. Findings include: (1) Fifty-five percent of public school districts reported having students enrolled in distance education courses in 2009–10 (table 1); (2) Districts reported an estimated 1,816,400 enrollments in distance education courses for 2009–10 (table 2); (3) Seventy-nine percent of districts with students enrolled in distance education courses reported enrollments of 100 or fewer students, with 25 percent reporting 1 to 10 enrollments, 27 percent reporting 11 to 30 enrollments, and 27 percent reporting 31 to 100 enrollments (table 3); (4) Eighty-seven percent of districts reported tracking all distance education courses that students completed with a passing grade, 79 percent reported tracking all distance education courses that students completed with a failing grade, and 65 percent reported tracking all distance education courses where students withdrew prior to completing the course (table 4); (5) Ninety-eight percent of districts reported monitoring student progress in distance education courses using a final grade report (table 5); (6) Twelve percent of districts reported having written policies specifying that a student cannot enroll in another distance education course when a distance education course was not successfully completed, while 6 percent of districts reported having policies specifying that a student must wait a specified time before enrolling in another distance education course (table 6); (7) Twenty-two percent of districts with students enrolled in distance education courses reported that students enrolled in regular high school programs could take a full course load in an academic term using only distance education courses, while 12 percent reported that students could fulfill all high school graduation requirements using only distance education courses (table 7); (8) Fifty percent of districts reported that a postsecondary institution in the United States delivered distance education courses in which students in their district were enrolled (table 8); (9) Districts reported that the types of distance education courses in which students enrolled were credit recovery (62 percent), dual enrollment (47 percent), Advanced Placement (29 percent), career and technical education (27 percent), and other types of academic courses (65 percent) (table 9); (10) The majority of districts reported that providing courses not otherwise available at the school (64 percent) and providing opportunities for students to recover course credits from classes missed or failed (57 percent) were very important reasons for having distance education courses in their district (table 10); (11) Fifty-nine percent of districts reported having students enrolled in courses that used the Internet with asynchronous (not simultaneous) instruction to a large extent, with an additional 27 percent reporting having students enrolled in courses that used this technology to a small or moderate extent (table 11); (12) The technology most frequently cited by districts as the primary mode of instructional delivery for the greatest number of distance education courses was the Internet using asynchronous instruction (63 percent) (table 12); (13) Ninety percent of districts with distance education enrollments reported having students enrolled in distance education courses delivered over the Internet (table 13); and (14) About three-quarters (74 percent) of the districts with distance education enrollments in 2009–10 indicated that they planned to expand the number of distance education courses offered in the next 3 years (table 14).”

Watson, J., & Gemin, B. (2009). Management and operations of online programs: Ensuring quality and accountability (Promising Practices in Online Learning). Vienna, VA: International Association for K–12 Online Learning. Retrieved from https://eric.ed.gov/?id=ED509622

From the ERIC abstract: “Online learning is growing rapidly as states and districts are creating new online schools, and existing programs are adding new courses and students. The growth reflects the spreading understanding that online courses and programs can serve a wide variety of students and needs. These include: (1) Creating opportunities for small and rural school districts to offer varied course subjects and highly qualified teachers to their students; (2) Allowing students to blend high school and post-secondary learning options; (3) Reducing class size; (4) Helping students recover credits in an alternative learning environment; (5) Providing individualized instruction and unique learning options; (6) Allowing students the opportunity to interact with students far beyond their school or town boundaries; and (7) Meeting the needs and expectations of today’s millennial students. Many school leaders are excited about the possibilities of online learning. When they start an online school, however, they quickly confront all the challenges of managing a high-quality, successful online program: creating online courses; finding, hiring, and managing teachers; supporting students; managing technology; and evaluating their programs to determine if they are successful. Fortunately, many online schools have years of operating experience, have developed and revised formal operations and management structures, and provide examples of successful management. This paper explores emerging practices in online program management and operations that can be used by many people working with an online learning program, from executive-level school leaders to department managers to teachers trying to find ways to improve their effectiveness with online students. Although it does not address state or district policy issues, legislators and policymakers will find it useful to understand the varied approaches that online schools are embracing to ensure quality as they determine the best ways to create oversight while allowing innovation to meet the needs of students and schools.”

Watson, J., & Rapp, C. (2011). Quality authorizing for online and blended-learning charter schools (NACSA Monograph). Chicago, IL: National Association of Charter School Authorizers. Retrieved from https://eric.ed.gov/?id=ED544290

From the ERIC abstract: “Online charter schools are one key subset of the K–12 online education landscape, which also includes state virtual schools, district-level online programs, private providers of both individual courses and entire schools, and others. As of late 2010, online and blended charter schools existed in more than 20 states, serving more than 100,000 students. Growth has been rapid, in the range of 15 percent to 25 percent annually, as parents and students seek new educational options. Despite the presence and growth of online charter schools, little research has been done regarding the practices of online charter school authorizers. Policymakers recognize that online charter schools present unique challenges for authorizers, but few best practices for authorizing online charter schools have emerged. Many online charter schools have been operating for five years or less, and some states are just beginning to open their first few online charter schools. In many cases, authorizing processes have not kept up with the rate at which online charter schools have opened, as most states are still in the early stages of developing authorizing procedures for online schools. To address this gap, the National Association of Charter School Authorizers (NACSA) has begun a multi-phase project to review online-school authorizing practices and make recommendations to authorizers. The purpose of this initial stage of research was to conduct a preliminary environmental scan in order to identify key issues that authorizers of online charter schools experience. NACSA engaged with the Evergreen Education Group and Donnell-Kay Foundation to accomplish the first stage of work, which included in-depth interviews with seven key authorizers from across the country to identify similar key issues they are working to address. The key findings generated from the interviews are outlined and explained in greater detail throughout this paper. They are: (1) Charter school authorizing is still in its early stages of development; (2) Online charter schools present opportunities and challenges for oversight; (3) Accountability for student achievement in online environments can be unique; (4) Governing board expertise may be lacking; (5) Online special education can be a serious challenge; (6) Building process and capacity in authorizing offices is critical; and (7) Appropriate funding levels are still under debate. Online and blended learning have created new opportunities for hundreds of thousands of students across the United States. Online learning, however, is still new enough that practices are outpacing policies and, in many cases, oversight. The sustainable, long-term growth of online and blended schools requires that policy frameworks keep pace with educational practice.”

Methods

Keywords and Search Strings

The following keywords and search strings were used to search the reference databases and other sources:

  • Accountability descriptor:“online classes”

  • Authorization descriptor:“Distance Education”

  • Authorization oversight online

  • Authorization descriptor:“virtual*”

  • Authorizing descriptor:“virtual classrooms”

  • Monitoring descriptor:“online classes”

  • Monitoring descriptor:“virtual classroom”

  • Oversight descriptor:“Distance Education”

  • Oversight descriptor:“Online Courses”

  • Oversight descriptor:“Virtual classroom”

  • Virtual schools authorization oversight

Databases and Search Engines

We searched ERIC for relevant resources. ERIC is a free online library of more than 1.6 million citations of education research sponsored by the Institute of Education Sciences (IES). Additionally, we searched IES and Google Scholar.

Reference Search and Selection Criteria

When we were searching and reviewing resources, we considered the following criteria:

  • Date of the publication: References and resources published over the last 15 years, from 2003 to present, were included in the search and review.

  • Search priorities of reference sources: Search priority is given to study reports, briefs, and other documents that are published or reviewed by IES and other federal or federally funded organizations.

  • Methodology: We used the following methodological priorities/considerations in the review and selection of the references: (a) study types—randomized control trials, quasi-experiments, surveys, descriptive data analyses, literature reviews, policy briefs, and so forth, generally in this order, (b) target population, samples (e.g., representativeness of the target population, sample size, volunteered or randomly selected), study duration, and so forth, and (c) limitations, generalizability of the findings and conclusions, and so forth.
This memorandum is one in a series of quick-turnaround responses to specific questions posed by educational stakeholders in the Midwest Region (Illinois, Indiana, Iowa, Michigan, Minnesota, Ohio, Wisconsin), which is served by the Regional Educational Laboratory (REL Midwest) at American Institutes for Research. This memorandum was prepared by REL Midwest under a contract with the U.S. Department of Education’s Institute of Education Sciences (IES), Contract ED-IES-17-C-0007, administered by American Institutes for Research. Its content does not necessarily reflect the views or policies of IES or the U.S. Department of Education nor does mention of trade names, commercial products, or organizations imply endorsement by the U.S. Government.