Research References

Ahn, J., Beck, A., Rice, J., & Foster, M. (2016). Exploring issues of implementation, equity, and student achievement with educational software in the DC public schools. AERA Open, 2(4), 1–10. https://eric.ed.gov/?id=EJ1194390

In this article, we present analyses from a researcher-practitioner partnership in the District of Columbia Public Schools, where we are exploring the impact of educational software on students' academic achievement. We analyze a unique data set that combines student-level information from the district with data of student usage of a mathematics game platform: First in Math (FIM). These data offer a window into long-standing issues in the educational technology literature around implementation, equity, and student achievement. We show that time spent in FIM was correlated with improved future performance on standardized math assessments for students in Grades 4–8. However, student time spent using FIM was highly related to factors such as race, gender, and prior achievement. Such observations from data are helpful for school districts and researchers to inform equitable implementation of new technologies and maximize benefits to learners.

Cavalluzzo, L., Lowther, D. L., Mokher, C., & Fan, X. (2012). Effects of the Kentucky Virtual Schools' hybrid program for algebra I on grade 9 student math achievement. Final Report. NCEE 2012–4020 National Center for Education Evaluation and Regional Assistance. https://eric.ed.gov/?id=ED530803

The 2006–11 Regional Educational Laboratory Appalachia at CNA conducted a rigorous evaluation of the Kentucky Virtual Schools hybrid algebra I curriculum. The curriculum combines traditional face-to-face instruction with an online program. This study used a two-cohort sample with 25 high schools in year 1 (SY 07/08: 13 treatment and 12 control) and 22 in year 2 (SY 08/09: 11 and 11), the randomized sample included 6,908 students, 61.4 percent of whom were in rural schools. As reported in the study, ‘Effects of the Kentucky Virtual Schools Hybrid Program for Algebra I on Grade 9 Student Math Achievement,’ researchers found that the hybrid class format was no more effective at increasing student achievement and future course-taking in math than algebra offered in the traditional face-to-face format.

Pane, J. F., Griffin, B. A., McCaffrey, D. F., & Karam, R. (2014). Effectiveness of Cognitive Tutor Algebra I at scale. Educational Evaluation and Policy Analysis, 36(2), 127–144. https://eric.ed.gov/?id=EJ1024233

This article examines the effectiveness of a technology-based algebra curriculum in a wide variety of middle schools and high schools in seven states. Participating schools were matched into similar pairs and randomly assigned to either continue with the current algebra curriculum for 2 years or to adopt Cognitive Tutor Algebra I (CTAI), which uses a personalized, mastery-learning, blended-learning approach. Schools assigned to implement CTAI did so under conditions similar to schools that independently adopt it. Analysis of posttest outcomes on an algebra proficiency exam finds no effects in the first year of implementation, but finds evidence in support of positive effects in the second year. The estimated effect is statistically significant for high schools but not for middle schools; in both cases, the magnitude is sufficient to improve the median student's performance by approximately eight percentile points.

Pyzdrowski, L. J., Butler, M. B., Walker, V. L., Pyzdrowski, A. S., & Mays, M. E. (2011). Exploring the feasibility of dual-credit mathematics courses in high school via a web-enhanced, blended model. Journal of General Education, 60(1), 43–60. https://eric.ed.gov/?id=EJ942482

The ‘WVEB’ Mathematics Project is a collaborative program offering college-level courses to high school students. Results of a matched-pair study comparing on- and off-campus students in algebra indicate that both sections show a significant gain in score on the math ‘ACT’ test, with no difference in gain found between sections.

Roschelle, J., Feng, M., Murphy, R. F., & Mason, C. A. (2016). Online mathematics homework increases student achievement. AERA Open, 2(4), 1–12. https://eric.ed.gov/?id=EJ1194398

In a randomized field trial with 2,850 seventh-grade mathematics students, we evaluated whether an educational technology intervention increased mathematics learning. Assigning homework is common yet sometimes controversial. Building on prior research on formative assessment and adaptive teaching, we predicted that combining an online homework tool with teacher training could increase learning. The online tool ASSISTments (a) provides timely feedback and hints to students as they do homework and (b) gives teachers timely, organized information about students' work. To test this prediction, we analyzed data from 43 schools that participated in a random assignment experiment in Maine, a state that provides every seventh-grade student with a laptop to take home. Results showed that the intervention significantly increased student scores on an end-of-the-year standardized mathematics assessment as compared with a control group that continued with existing homework practices. Students with low prior mathematics achievement benefited most. The intervention has potential for wider adoption. [For the corresponding grantee submission, see ED575159.]

Snipes, J., Huang, C.-W., Jaquet, K., Finkelstein, N. (2015). The effects of the Elevate Math summer program on math achievement and algebra readiness (REL 2015–096). Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory West. https://eric.ed.gov/?id=ED558157

The Effects of the Elevate Math summer program on math achievement and algebra readiness: This randomized trial examined the effects of the Elevate Math summer program on math achievement and algebra readiness, as well as math interest and self-efficacy, among rising 8th grade students in California's Silicon Valley. The Elevate Math summer math program targets students who score in the range between ‘high basic’ and ‘low proficient’ on state math tests. It consists of 19 days of mathematics instruction, consisting of three hours per day in traditional classroom instruction and one hour per day using Khan Academy (a free online learning system). During summer 2014, students were randomly assigned to a treatment group that received access to the program at the beginning of the summer or to a control group that received access to the program later in the summer. End-of-program test scores and survey responses of students in the treatment group were compared with those of students in the control group prior to their exposure to the program. Treatment group students scored significantly higher than the control group (4 points or 0.7 standard deviation) on a test of algebra readiness. They were also significantly more likely (29 percent versus 12 percent) to reach achievement thresholds associated with success in algebra I. However, treatment and control groups did not show significant differences in terms of math interest or self-efficacy. The results show that the Elevate Math summer program can significantly improve student math achievement and algebra readiness; however, 70 percent of program participants were still not ready for algebra I content. This suggests that summer math programs such as Elevate Maths may be important tools for improving math achievement among rising eighth grade students, but most targeted students will need additional support in order to ensure success in algebra. The following are appended: (1) Data, outcomes, and methodology; and (2) Sensitivity analyses.

Additional Organizations to Consult

National Council of Teachers of Mathematics (NCTM), Free Resources for Teaching Math Online: https://www.nctm.org/freeresources/

NCTM is here to support the mathematics education community, not just during challenging times like these but 365 days a year. With so many teachers currently transitioning to online teaching, we offer these free resources to support teachers and their students.

What Works Clearinghouse, “Studies of Distance Learning:” https://ies.ed.gov/ncee/wwc/Distance LearningStudy

On March 26, NCEE announced a call for nominations of rigorous research they are aware of or have conducted that evaluates the effectiveness of specific distance education practices or products on student outcomes. This page includes all nominations from the field, as well as relevant entries from ERIC and the What Works Clearinghouse Reviews of Individual Studies Database. We are providing this list to the field for educators and researchers to see the range of publicly available evidence on the effectiveness of distance learning.

Inclusion in this list is provided to the field as a way to highlight the breadth of research available. It is not an endorsement of the intervention or the study by the WWC, IES, or the US Department of Education. If the studies have been reviewed by the WWC, a link to the study review page is included in the list. Note that not all of the studies listed here are eligible for WWC review or would necessarily meet WWC standards.

Methods

The following keywords and search strings were used to search the reference databases and other sources:

• “distance learning” AND “math”
• “online learning” AND “math”

Databases and Resources

We searched ERIC, a free online library of over 1.6 million citations of education research sponsored by the Institute of Education Sciences, for relevant resources.

Reference Search and Selection Criteria

REL Pacific searched ERIC for studies that were published in English-language peer-reviewed research journals within the last 10 years. We also utilized resource harvesting to find relevant research. Sources included in this document were last accessed in July 2020.

REL Pacific prioritized documents that are accessible online and publicly available, and prioritized references that provide practical information based on peer-reviewed research for the education system stakeholders who requested this Ask A REL. Because this request specifically requested rigorous research, we prioritized studies that used randomized control trials and quasi-experiments. We also considered aspects such as target population, sample size, study duration, limitations, and generalizability of the findings and conclusions.

This memorandum is one in a series of quick-turnaround responses to specific questions posed by educational stakeholders in the Pacific Region (American Samoa, the Commonwealth of the Northern Mariana Islands, the Federated States of Micronesia, Guam, Hawai'i, the Republic of the Marshall Islands, and the Republic of Palau), which is served by the Regional Educational Laboratory (REL Pacific) at McREL International. This memorandum was prepared by REL Pacific under a contract with the U.S. Department of Education's Institute of Education Sciences (IES), Contract ED-IES-17-C-0010, administered by McREL International. Its content does not necessarily reflect the views or policies of IES or the U.S. Department of Education, nor does mention of trade names, commercial products, or organizations imply endorsement by the U.S. Government.