Skip Navigation
Stay Up-to-Date:
Skip Navigation

REL Southwest Ask A REL Response

Education Technology:

Impact of Devices on Student Achievement in Middle School

October 2019

Question:

What is the impact of devices in the classroom on student achievement in middle school?

Response:

Print-friendly version (489 KB) PDF icon

Thank you for the question you submitted to our REL Reference Desk. We have prepared the following memo with research references to help answer your question. For each reference, we provide an abstract, excerpt, or summary written by the study’s author or publisher. Following an established Regional Educational Laboratory (REL) Southwest research protocol, we conducted a search for research reports as well as descriptive study articles on the impact of devices in the classroom on student achievement in middle school.

We have not evaluated the quality of references and the resources provided in this response. We offer them only for your reference. Also, we searched the references in the response from the most commonly used resources of research, but they are not comprehensive, and other relevant references and resources may exist. References provided are listed in alphabetical order, not necessarily in order of relevance. We do not include sources that are not freely available to the requestor.

Research References

Bartholomew, S. R., Reeve, E., Veon, R., Goodridge, W., Lee, V., & Nadelson, L. (2017). Relationships between access to mobile devices, student self-directed learning, and achievement. Journal of Technology Education, 29(1), 2–24. https://eric.ed.gov/?id=EJ1164687

From the ERIC abstract: “Today’s students are growing up in a world of constant connectivity, instant information, and ever-changing technological advancements. The increasingly ubiquitous nature of mobile devices among K-12 students has led many to argue for and against the inclusion of these devices in K-12 classrooms. Arguments in favor cite instant access to information and collaboration with others as positive affordances that enable student self-directed learning. In this study, 706 middle school students from 18 technology and engineering education classes worked in groups of 2-3 to complete an open-ended engineering design challenge. Students completed design portfolios and constructed prototypes in response to the design challenge. Classes were divided with some allowing access to mobile devices during the study and others not allowing access. Additionally, randomly assigned classes completed the design portfolio electronically, and others completed the portfolio on paper. Final student portfolios and products were assessed and assigned a rank order using a method of assessment called adaptive comparative judgment. Thirty student interviews were conducted as well as 6 teacher interviews. Statistical analyses between student access, portfolio type, student self-directed learning, and student achievement were conducted. Findings showed that student self-directed learning was independent of mobile device access during the study. Mobile device access was significantly correlated with higher student scores on the design portfolio, but mobile device access was independent of student scores on design products.”

Blankenship, M. U., & Margarella, E. E. (2014). Technology and secondary writing: A review of the literature. Contemporary Educational Technology, 5(2), 146–160. https://eric.ed.gov/?id=EJ1105503

From the ERIC abstract: “This article reports a review of the literature that focused on relationship between writing instruction and technology in the secondary classroom since the passing of the No Child Left Behind Act over the past two decades. Based on the search, six themes have emerged across the fields of writing instruction and assessment. Within writing instruction, it was found that researchers often focused on a third space (Bhabha, 1994) where writing can take place in meaningful ways. Also, technology often served as a motivator during the instructional process of writing and worked to engage students in varied lessons. Finally, researchers found an increase in the amount of writing for secondary students when technology was introduced into the instructional classroom. Within writing assessment, the research focused on special populations including special education students, minorities, economically disadvantaged and English language learners.”

Callaghan, M. N., Long, J. J., van Es, E. A., Reich, S. M., & Rutherford, T. (2018). How teachers integrate a math computer game: Professional development use, teaching practices, and student achievement. Journal of Computer Assisted Learning, 34(1), 10–19. https://eric.ed.gov/?id=EJ1165838. Full text retrieved from https://escholarship.org/content/qt1c77v2tn/qt1c77v2tn.pdf

From the ERIC abstract: “As more attention is placed on designing digital educational games to align with schools’ academic aims (e.g., Common Core), questions arise regarding how professional development (PD) may support teachers’ using games for instruction and how such integration might impact students’ achievement. This study seeks to (a) understand how teachers use PD resources (e.g., technology personnel and game-use workshops) for integration; (b) determine how teachers integrate games into their instruction; and (c) examine how those teaching practices are associated with student achievement. This mixed method study used survey and interview responses from elementary school teachers (n = 863) with access to PD resources for implementing a math game intervention and standardized math-test scores from their second- through sixth-grade students (n = 10,715). Findings showed few teachers sought PD assistance for integration, but many desired such support. Some reported using integrative practices (i.e., referencing game and using game-generated progress reports) to identify struggling students, whereas several found integration challenging. Teachers’ reordering of game objectives to align with lessons and viewing of game-based PD videos were associated with increased student math achievement in our OLS-analysis. However, this result was no longer statistically significant within a school fixed-effects model, suggesting school differences may influence how strongly teachers’ practices are associated with student achievement.”

Cheung, A. C. K., & Slavin, R. E. (2013). The effectiveness of educational technology applications for enhancing mathematics achievement in K-12 classrooms: A meta-analysis. Educational Research Review, 9, 88–113. https://eric.ed.gov/?id=EJ999451. Full text retrieved from http://sttechnology.pbworks.com/w/file/fetch/67600623/Cheung_(2013)_The%20effectiv eness%20of%20ed%20tech%20applications%20in%20K12%20classrooms.pdf

From the ERIC abstract: “The present review examines research on the effects of educational technology applications on mathematics achievement in K-12 classrooms. Unlike previous reviews, this review applies consistent inclusion standards to focus on studies that met high methodological standards. In addition, methodological and substantive features of the studies are investigated to examine the relationship between educational technology applications and study features. A total of 74 qualified studies were included in our final analysis with a total sample size of 56,886 K-12 students: 45 elementary studies (N = 31,555) and 29 secondary studies (N = 25,331). Consistent with the more recent reviews, the findings suggest that educational technology applications generally produced a positive, though modest, effect (ES = +0.15) in comparison to traditional methods. However, the effects may vary by educational technology type. Among the three types of educational technology applications, supplemental CAI had the largest effect with an effect size of +0.18. The other two interventions, computermanagement learning and comprehensive programs, had a much smaller effect size, +0.08 and +0.07, respectively. Differential impacts by various study and methodological features are also discussed.”

Regan, K., Evmenova, A. S., Good, K., Legget, A., Ahn, S. Y., Gafurov, B., et al. (2018). Persuasive writing with mobile-based graphic organizers in inclusive classrooms across the curriculum. Journal of Special Education Technology, 33(1), 3–14. https://eric.ed.gov/?id=EJ1168629. Full text retrieved from https://journals.sagepub.com/doi/pdf/10.1177/0162643417727292

From the ERIC abstract: “As writing instruction expands beyond the language arts classroom, students with disabilities, English language learners, and others who struggle with writing continue to need support with written expression. A timely practice to support student writing is the use of technology. This study used a quasi-experimental group design to examine the effects of a mobile-based graphic organizer (MBGO) with embedded self-regulated learning strategies and strategy instruction on the persuasive writing of middle school students in an inclusive classroom. After given opportunities to practice writing in the content areas, students with and without disabilities who used the MBGO on an iPad to compose persuasive essays significantly outperformed students with and without disabilities in the control group for number of transition words and writing quality. Limitations of the study and future research suggestions are discussed.”

Robinson, K. (2016). The effect of technology integration on high school students’ literacy achievement. Teaching English with Technology, 16(3), 3–16. https://eric.ed.gov/?id=EJ1135683

From the ERIC abstract: “This literature review presents a critical appraisal of current research on the role technology integration plays in high school students’ literacy achievement. It identifies the gaps within the research through comprehensive analysis. The review develops an argument that the use of laptops in secondary English classrooms has a significant impact upon students’ literacy achievement in both a positive and negative manner. The literature review begins by exploring early research and finds that there is a lack of longitudinal studies regarding laptop integration. This is a result of the trend at the time, which was to focus on the impact on student and teacher attitudes rather than the impact on literacy. Through the critical appraisal of current research, it is revealed that the attitudes and beliefs of individual teachers to laptop integration is the leading cause of student literacy achievement. The literature review progresses to explore the challenges facing educators and the concerns for educators.”

Shapley, K. S., Sheehan, D., Maloney, C., & Caranikas-Walker, F. (2010). Evaluating the implementation fidelity of technology immersion and its relationship with student achievement. Journal of Technology, Learning, and Assessment, 9(4). https://eric.ed.gov/?id=EJ873678

From the ERIC abstract: “In a pilot study of the Technology Immersion model, highneed middle schools were ‘immersed’ in technology by providing a laptop for each student and teacher, wireless Internet access, curricular and assessment resources, professional development, and technical and pedagogical support. This article examines the fidelity of model implementation and associations between implementation indicators and student achievement. Results across three years for 21 immersion schools show that the average levels of school support for Technology Immersion and teachers’ Classroom Immersion increased slightly, while the level of Student Access and Use declined. Implementation quality varied across schools and classrooms, with a quarter or fewer of schools and core-content classrooms reaching ‘substantial’ implementation. Using hierarchical linear modeling, we found that teacher-level implementation components (Immersion Support, Classroom Immersion) were inconsistent and mostly not statistically significant predictors of student achievement, whereas students’ use of laptops outside of school for homework and learning games was the strongest implementation predictor of achievement.”

Sternberg, B. J., Kaplan, K. A., & Borck, J. E. (2007). Enhancing adolescent literacy achievement through integration of technology in the classroom. Reading Research Quarterly, 42(3), 416–420. https://eric.ed.gov/?id=EJ767777 Full text retrieved from https://lispsd2010.pbworks.com/f/adolit4.pdf

From the ERIC abstract: “Adolescent literacy achievement across the United States is in crisis. More than eight million students in grades 4 to 12 are identified as struggling readers. These students, who perform below grade level in reading and writing, are at high risk for failure in all content subjects and ultimately for dropping out of school. Professionals in the field must pursue additional research around technology integration to enhance adolescent literacy achievement so that states across the nation can best create and promote the necessary programs to reverse the adolescent literacy achievement crisis. In this article, the authors take the state of Connecticut as an example that is expanding its focus by seeking sound research to inform the preparation of adolescents for success in further education and training through integration of technology in the classroom. Connecticut continues to explore key elements in programs designed to improve adolescent literacy achievement in middle and high schools, such as those outlined by the Alliance for Excellent Education, the National Council of Teachers of English, the International Reading Association, and the National Association of Secondary School Principals. Recognizing that important research has already been completed in the area of educational technology, this article suggests seven areas for further research that are of interest to state policymakers, focusing particularly on enhancing adolescent literacy achievement through the integration of technology across all content areas. Empirical research in these areas can be used to inform future practice in Connecticut and across the nation: (1) state-offered virtual courses and delivery systems, (2) communication tools, (3) artificial intelligence, (4) word processors, (5) new literacies practices, (6) professional development, and (7) technology for parents.”

Young, J., Hamilton, C., & Cason, M. (2017). Interactive whiteboards in mathematics spaces: An examination of technology integration in an urban middle school. Contemporary Educational Technology, 8(4), 303–318. https://eric.ed.gov/?&id=EJ1157955

From the ERIC abstract: “The purpose of this study was to examine the effects of integrating Interactive Whiteboard (IWB) technology on middle school mathematics achievement in an urban school. Propensity score matching was used to create a comparable control group in order to isolate the effects of IWB technology on mathematics achievement. An initial experimental group (n = 716) of ethnically diverse urban students receiving IWB instruction was matched to a control population (n = 856) based on propensity scores generated from demographic and ability data. Student achievement data were analyzed with 2 × 4 ANOVA to access treatment main effects and the effects of demographic variables such as gender, ethnicity, and ability. Ethnicity was a significant moderator of the effects. Specifically, a positive effect size was observed for White students, and the achievement gap was also reduced for Hispanic students. Implications for mathematics pedagogy with an IWB are provided based on these conclusions.”

Additional Organizations to Consult

Future Ready Schools (FRS) — https://futureready.org/

From the website: “Future Ready Schools® (FRS) helps innovative educators ensure that each student graduates from high school with the agency, passion, and skills to be a productive, successful, and responsible citizen.
Through FRS’s tools and resources, district and school leaders collaborate to advance evidence-based practices and create rigorous and engaging student-centered learning environments.
The FRS network emphasizes vision, culture, leadership, strong pedagogy, goal setting, and the critical role of ‘people’ in personalized learning. Technology is a necessary tool that enables these new systems to perform efficiently with equity for every child.”

International Association for K-12 Online Learning (iNACOL) — https://www.inacol.org/

From the website: “The mission of iNACOL is to drive the transformation of education systems and accelerate the advancement of breakthrough policies and practices to ensure high-quality learning for all.
iNACOL is the leading advocate for transforming learning. We leverage the power of personalized, competency-based learning models to accelerate the shift to studentcentered learning. …
Working collaboratively with experts in the field, iNACOL publishes reports and related resources on key topics and tough issues that equip and empower educators and leaders to catalyze and scale personalized, next generation learning models.”

International Society for Technology in Education (ISTE) — https://www.iste.org/

From the website: “The ISTE Standards for Education Leaders support the implementation of the ISTE Standards for Students and the ISTE Standards for Educators and provide a framework for guiding digital age learning. These standards target the knowledge and behaviors required for leaders to empower teachers and make student learning possible. They’re focused on some of the most timely, yet enduring, topics in education today—equity, digital citizenship, visioneering, team and systems building, continuous improvement and professional growth.”

Office of Educational Technology (OET), U.S. Department of Education — https://tech.ed.gov/

From the website: “The U.S. Department of Education Office of Educational Technology (OET) develops national educational technology policy and establishes the vision for how technology can be used to transform teaching and learning and how to make everywhere, all-the-time learning possible for early learners through K-12, higher education, and adult education.”
REL Southwest note: OET provides multiple professional learning resources including:

PowerUp What Works — https://powerupwhatworks.org/

From the website: “PowerUp provides resources and tools that represent best practices in instruction, the principles of Universal Design for Learning and the integration of technology to enhance teaching and learning.”

Project Tomorrow — http://www.tomorrow.org/

From the website: “The vision of Project Tomorrow is to ensure that today’s students are well prepared to be tomorrow’s innovators, leaders and engaged citizens of the world. We believe that by supporting the innovative uses of science, math and technology resources in our K-12 schools and communities, students will develop the critical thinking, problem solving and creativity skills needed to compete and thrive in the 21st century.
We approach our mission through national research projects, the replication of model excellence projects in schools and communities, online tools and resources for students, teachers and parents, and national and regional advocacy efforts.”

Methods

Keywords and Search Strings

The following keywords and search strings were used to search the reference databases and other sources:

  • [(“technology integration”) AND “middle schools”)]
  • [(technology) AND (achievement) AND (“middle schools”)]
  • [(“technology integration”) AND (“student achievement”) AND (schools)]

Databases and Resources

We searched ERIC for relevant, peer-reviewed research references. ERIC is a free online library of more than 1.7 million citations of education research sponsored by the Institute of Education Sciences (IES). Additionally, we searched the What Works Clearinghouse.

Reference Search and Selection Criteria

When we were searching and reviewing resources, we considered the following criteria:

  • Date of the publication: References and resources published from 2004 to present, were include in the search and review.
  • Search priorities of reference sources: Search priority is given to study reports, briefs, and other documents that are published and/or reviewed by IES and other federal or federally funded organizations, academic databases, including ERIC, EBSCO databases, JSTOR database, PsychInfo, PsychArticle, and Google Scholar.
  • Methodology: The following methodological priorities/considerations were given in the review and selection of the references: (a) study types—randomized control trials, quasi-experiments, surveys, descriptive data analyses, literature reviews, policy briefs, and so forth, generally in this order; (b) target population, samples (representativeness of the target population, sample size, volunteered or randomly selected, and so forth), study duration, and so forth; and (c) limitations, generalizability of the findings and conclusions, and so forth.
This memorandum is one in a series of quick-turnaround responses to specific questions posed by stakeholders in the Southwest Region (Arkansas, Louisiana, New Mexico, Oklahoma, and Texas), which is served by the Regional Educational Laboratory (REL) Southwest at AIR. This memorandum was prepared by REL Southwest under a contract with the U.S. Department of Education’s Institute of Education Sciences (IES), Contract ED-IES-91990018C0002, administered by AIR. Its content does not necessarily reflect the views or policies of IES or the U.S. Department of Education nor does mention of trade names, commercial products, or organizations imply endorsement by the U.S. Government.