Skip Navigation
Stay Up-to-Date:
Skip Navigation

REL Southwest Ask A REL Response

Online Courses:

Comparing Student Outcomes for Online and Traditional Curriculum Materials

November 2021

Question:

Is there any difference in student performance when using online curriculum materials versus using a textbook?

Response:

Print-friendly version (438 KB) PDF icon

Thank you for the questions you submitted to our REL Reference Desk. We have prepared the following memo with research references to help answer your questions. For each reference, we provide an abstract, excerpt, or summary written by the study’s author or publisher. Following an established Regional Educational Laboratory (REL) Southwest research protocol, we conducted a search for research reports as well as descriptive study articles on whether there is any difference in student performance when using online curriculum materials versus using a textbook.

We have not evaluated the quality of references and the resources provided in this response. We offer them only for your reference. Also, we searched the references in the response from the most commonly used resources of research, but they are not comprehensive, and other relevant references and resources may exist. References provided are listed in sections with sources in each section in alphabetical order, not necessarily in order of relevance. We do not include sources that are not freely available to the requestor.

Research References

Bledsoe, C., Pilgrim, J., Vasinda, S., & Martinez, E. (2019). Making Connections: An analogy between traditional and online text. Texas Journal of Literacy Education, 7(1), 10–24. https://eric.ed.gov/?id=EJ1221631

From the ERIC abstract: “Today’s student must be able to work in an online environment. This often presents unique challenges for elementary students as they develop and extend fundamental literacy skills to various media. Many text features such as titles, headings, authors, and copyrights are easily transferable from paper to an online format; however, orientation and navigation of text differ in online environments where text offers dynamic features not possible in traditional print resources. We present a parallel comparison of traditional and online text to illustrate how online texts mimic their analog counterparts and highlight ways in which they differ. This offers teachers a way to make learning about text features of both formats more concrete for their students.”

Chang, I. (2020). Open versus traditional textbooks: A comparison of student engagement and performance. International Journal of Teaching and Learning in Higher Education, 32(3), 486–498. https://eric.ed.gov/?id=EJ1300061

From the ERIC abstract: “This study compared student engagement and performance in both open educational resources (OER) (n[open textbook users fall 2018] = 72) and traditional textbook (n[traditional textbook users fall 2017] = 66) classes. Data were drawn from the Learning Management System (LMS). Results show (1) final grades in the OER class were on a par with the traditional textbook class, and (2) OER equalize student engagement and performance by narrowing the dispersions of page views, on-time assignment submissions (OTAS), attendance, and final grades. (3) OER increased attendance and lessened excessive dependence on LMS course materials recorded in the traditional class. (4) The indirect effect of attendance on final grades was stronger than the direct effect of OTAS in the OER class. Attendance provided the opportunity for the instructor and students to be on the ‘same page,’ which helps students better assimilate course content and comprehend lectures. (5) The availability of textbooks appears to be a factor influencing student course success. However, it remains unknown how much of the variance was explained by OER. It is apparent that OER are more important than ever in elevating overall student academic success.”

Colvard, N. B., Watson, C., & Park, H. (2018). The impact of open educational resources on various student success metrics. International Journal of Teaching and Learning in Higher Education, 30(2), 262–276. https://eric.ed.gov/?id=EJ1184998

From the ERIC abstract: “There are multiple indicators which suggest that completion, quality, and affordability are the three greatest challenges for higher education today in terms of students, student learning, and student success. Many colleges, universities, and state systems are seeking to adopt a portfolio of solutions that address these challenges. This article reports the results of a large-scale study (21,822 students) regarding the impact of course-level faculty adoption of Open Educational Resources (OER). Results indicate that OER adoption does much more than simply save students money and address student debt concerns. OER improve end-of-course grades and decrease DFW (D, F, and Withdrawal letter grades) rates for all students. They also improve course grades at greater rates and decrease DFW rates at greater rates for Pell recipient students, part-time students, and populations historically underserved by higher education. OER address affordability, completion, attainment gap concerns, and learning. These findings contribute to a broadening perception of the value of OERs and their relevance to the great challenges facing higher education today.”

Cuttler, C. (2019). Students’ use and perceptions of the relevance and quality of open textbooks compared to traditional textbooks in online and traditional classroom environments. Psychology Learning and Teaching, 18(1), 65–83. https://eric.ed.gov/?id=EJ1207859

From the ERIC abstract: “The bulk of previous research on students’ perceptions of open educational resources have lacked a control group of students rating traditional textbooks. Moreover, few studies have examined differences in the perceptions of online students and those taking classes in the classroom. A 2 × 2 cross-sectional design was used in which 925 students, assigned either a traditional textbook or an open textbook in either an online or classroom environment, were recruited to complete an online survey. Students assigned open textbooks were almost twice as likely to report using their textbooks, they used them more frequently, and for more time per week overall. Students assigned open textbooks also perceived a greater degree of overlap between the textbook, lecture, and quiz material than did students assigned traditional textbooks. Finally, ratings of the open textbooks were significantly higher than ratings of the traditional textbooks overall and on 11 of 15 different dimensions. Few differences in the online and classroom students were detected, suggesting both groups experienced similar benefits of the open textbooks. These findings demonstrate that replacing traditional textbooks with open textbooks may help to offset some of the financial hardships students face while improving students’ engagement and satisfaction with their assigned textbook.”
REL Southwest Note: To access the full article, click on “Direct link” on the ERIC landing page.

Grissett, J. O., & Huffman, C. (2019). An open versus traditional psychology textbook: Student performance, perceptions, and use. Psychology Learning and Teaching, 18(1), 21–35. https://eric.ed.gov/?id=EJ1207847

From the ERIC abstract: “In the current study we examined students’ course performance, perceptions, and self-reported use of an open textbook compared to a traditional publisher’s textbook in an introductory psychology course. Sixty students from two course sections used either an open textbook or a traditional textbook. To minimize the effect of confounding variables, each section was taught by the same instructor, at the same time of day, using identical in-class materials, syllabi, and sequencing, on alternating days of the week. Course performance was measured by analyzing exam scores and final course grades. A 22-item survey was utilized at the end of the term to measure students’ textbook perceptions and use. Findings revealed no significant differences in student course performance or textbook use across the two sections. Participants in both groups perceived cost, weight, and convenience as the biggest advantages of an open textbook and perceived ease of reading, convenience, ability to highlight and take notes, ability to quickly find a topic, and ability to keep as a reference as the biggest advantages of a traditional textbook. Finally, when asked to select which type of textbook they preferred, students in each section selected the textbook they were using. The implications of these findings are discussed.”
REL Southwest Note: To access this article from the ERIC landing page, click “Direct link” to get to the publication page and download the article.

Reich, S. M., Yau, J. C., Xu, Y., Muskat, T., Uvalle, J., & Cannata, D. (2019). Digital or print? A comparison of preschoolers’ comprehension, vocabulary, and engagement from a print book and an e-book. AERA Open, 5(3), 1–16. https://eric.ed.gov/?id=EJ1229658

From the ERIC abstract: “Increasingly, children are engaging in early literacy experiences through digital devices. This raises questions about how electronic reading compares to print reading. To assess this, we randomly assigned 200 children (3-5 years) to be read the same book (1) with auto-narration on a tablet or (2) by a researcher from a print book. Reading was recorded and coded for behavioral and emotional engagement and vocalizations. Children were also tested on their story comprehension and vocabulary. Children had slightly higher posttest scores in the print condition. Older children and females also scored higher. There was an interaction between weekly tablet use and book platform. Children were equally engaged with the e-book and print book, but vocalized more about the device in the e-book condition. Findings suggest that e-books offer many of the same, but not all, of the educational affordance as print books. Additionally, novelty might be important in supporting comprehension.”

Robinson, T. J., Fischer, L., Wiley, D., & Hilton, J., III. (2014). The impact of open textbooks on secondary science learning outcomes. Educational Researcher, 43(7), 341–351. https://eric.ed.gov/?id=EJ1042753

From the ERIC abstract: “Given the increasing costs associated with commercial textbooks and decreasing financial support of public schools, it is important to better understand the impacts of open educational resources on student outcomes. The purpose of this quantitative study is to analyze whether the adoption of open science textbooks significantly affects science learning outcomes for secondary students in earth systems, chemistry, and physics. This study uses a quantitative quasi-experimental design with propensity score matched groups and multiple regression to examine whether student learning was influenced by the adoption of open textbooks instead of traditional publisher produced textbooks. Students who used open textbooks scored 0.65 points higher on end-of-year state standardized science tests than students using traditional textbooks when controlling for the effects of 10 student and teacher covariates. Further analysis revealed statistically significant positive gains for students using the open chemistry textbooks, with no significant difference in student scores for earth systems of physics courses. Although the effect size of the gains were relatively small, and not consistent across all textbooks, the finding that open textbooks can be as effective or even slightly more effective than their traditional counterparts has important considerations in terms of school district policy in a climate of finite educational funding.”
REL Southwest Note: To access the full article, click on “Direct link” on the ERIC landing page.

Ruggieri, C. (2020). Students’ use and perception of textbooks and online resources in introductory physics. Physical Review Physics Education Research, 16(2), Article 020123, 1–25. https://eric.ed.gov/?id=EJ1275059

From the ERIC abstract: “In this mixed-methods study of large enrollment introductory physics service courses, I investigated students’ perception and use of online education resources as supplements to course-provided materials and activities. Specifically, I focused on the increasing use of popular free online media resources such as YouTube and Khan Academy, and fee-based textbook solution repository services such as Chegg. In the quantitative portion of this study, I surveyed students from three courses on their textbook and online resource usage and found that most students relied primarily on online resources as they navigated the courses, and comparatively few used the textbook regularly. In the qualitative portion, I investigated the patterns and culture of textbook and online resource usage via semistructured interviews and found that students reported using online resources as supplements to, or in place of, the course-provided materials when engaging with online homework or studying for an exam. Students reported using online resources productively to guide learning efforts, but also acknowledged unproductive uses such as copying solutions to mitigate loss of assignment points. I provide suggestions for changes in course materials, practices, and expectations to better engage students in the course and in their learning.”
REL Southwest Note: To access this article from the ERIC landing page, click “Direct link” to get to the Physical Review Physics Education Research page. Then, click on the button of the preferred version of the article—PDF or HTML.

Sansom, R. L., Clinton-Lisell, V., & Fischer, L. (2021). Let students choose: Examining the impact of open educational resources on performance in general chemistry. Journal of Chemical Education, 98(3), 745–755. https://eric.ed.gov/?id=EJ1290370

From the ERIC abstract: “General chemistry is a gateway course for most STEM majors, so student success is a priority for chemistry faculty. Providing quality information resources for students, including textbooks, is one way that instructors can support student learning. However, these resources can be prohibitively expensive for some students, causing them to opt out of purchasing a textbook or incur stress from the costs of time and money to obtain a textbook. Open educational resources (OERs) are no-cost materials, available in the public domain, that students and instructors can use to reduce the financial burden of college coursework. However, as chemistry instructors consider adopting OERs, they may be concerned about the time cost to reframe their courses around different materials, the risk of negatively impacting student learning, and whether the benefits to students outweigh those costs and risks. Although the financial benefits to students have been established, and the evidence suggests minimal risk of poor academic outcomes, the cost to instructors continues to be prohibitively high. In this study, the instructor used a commercial text as the official resource for the course and offered students a choice to use either the commercial textbook or an OER textbook. This soft adoption of OERs dramatically reduced the time cost associated with using the OER for the instructor, while providing financial benefits to students who chose the OER. To address the risk that using an OER might negatively impact student performance, we investigated the impact of student textbook choice on student performance in general chemistry, controlling for relevant academic and affective variables. We found that students using OER performed as well as students using the commercial textbook. With minimal effort, chemistry instructors can provide a no-cost alternative for students, with confidence that it will not detrimentally affect their learning.”
REL Southwest Note: To access the full article, click on “Direct link” on the ERIC landing page.

Singer T., Lauren M., Alexander, P. A., & Berkowitz, L. E. (2017). Effects of processing time on comprehension and calibration in print and digital mediums. Journal of Experimental Education, 87(1), 101–115. https://eric.ed.gov/?id=EJ1214848. Retrieved from https://www.researchgate.net/publication/321991419

From the ERIC abstract: “This study explored the effects of processing texts in print or digitally on readers’ comprehension, processing time, and calibration. Eighty-six undergraduates read print and digital versions of book excerpts about childhood ailments presented in counterbalanced order. Comprehension was tested at three levels (i.e., main idea, key points, and other relevant information). Direct comparisons between print and digital reading demonstrated a significant advantage for reading in print on students’ recall of key points and other relevant information but not the main idea. When processing time was added as a mediator variable, it significantly affected the relation between medium and comprehension for all question levels. In terms of calibration, students read more quickly and judged their performance higher when engaged digitally, although their actual performance was much better when reading in print. Implications of these findings for subsequent research are considered.”

Methods

Keywords and Search Strings

The following keywords and search strings were used to search the reference databases and other sources:

  • [(“online materials” OR “digital texts” OR “Open Educational Resources”) AND (“textbooks” OR “printed materials” OR “printed texts”) AND (K–12)]
  • [(“online materials” OR “digital texts” OR “Open Educational Resources”) AND (“textbooks” OR “printed materials” OR “printed texts”)]
  • textbooks versus online curriculum materials
  • printed materials versus online materials
  • print versus digital texts
  • impact of Open Educational Resources in K–12 education

Databases and Resources

We searched ERIC for relevant, peer-reviewed research references. ERIC is a free online library of more than 1.7 million citations of education research sponsored by the Institute of Education Sciences (IES). Additionally, we searched the What Works Clearinghouse.

Reference Search and Selection Criteria

When we were searching and reviewing resources, we considered the following criteria:

  • Date of the publication: References and resources published from 2006 to present, were include in the search and review.
  • Search priorities of reference sources: Search priority is given to study reports, briefs, and other documents that are published and/or reviewed by IES and other federal or federally funded organizations, academic databases, including ERIC, EBSCO databases, JSTOR database, PsychInfo, PsychArticle, and Google Scholar.
  • Methodology: The following methodological priorities/considerations were given in the review and selection of the references: (a) study types—randomized control trials, quasi-experiments, surveys, descriptive data analyses, literature reviews, policy briefs, and so forth, generally in this order; (b) target population, samples (representativeness of the target population, sample size, volunteered or randomly selected, and so forth), study duration, and so forth; and (c) limitations, generalizability of the findings and conclusions, and so forth.
This memorandum is one in a series of quick-turnaround responses to specific questions posed by stakeholders in the Southwest Region (Arkansas, Louisiana, New Mexico, Oklahoma, and Texas), which is served by the Regional Educational Laboratory (REL) Southwest at AIR. This memorandum was prepared by REL Southwest under a contract with the U.S. Department of Education’s Institute of Education Sciences (IES), Contract ED-IES-91990018C0002, administered by AIR. Its content does not necessarily reflect the views or policies of IES or the U.S. Department of Education nor does mention of trade names, commercial products, or organizations imply endorsement by the U.S. Government.