NCES Blog

National Center for Education Statistics

What National and International Assessments Can Tell Us About Technology in Students’ Learning: Eighth-Graders’ Readiness to Use Technology

Across the country in 2020, students, teachers, and parents have had to adapt to changes in the delivery of education instruction due to the coronavirus pandemic and turn to information and communication technologies (ICT) to learn, interact, and assess progress. It is important that we are able to assess students’ abilities to understand and use technology now more than ever. This post, the first in a three-part series, discusses the results of two technology-focused assessments, NAEP TEL and ICILS, which NCES administered in 2018 (see textbox for more information about these assessments).

Students’ performance on NAEP TEL and ICILS

According to the 2018 NAEP TEL, 46 percent of U.S. eighth-grade students scored at or above the NAEP Proficient level, meaning that they were able to demonstrate the selection and use of an appropriate range of tools and media (figure 1). According to the 2018 ICILS, 25 percent of eighth-grade students scored at or above proficiency level 3 for computer and information literacy—that is, they demonstrated the capacity to work independently when using computers as information-gathering and management tools. In addition, 20 percent of eighth-grade students scored in the upper region for computational thinking, meaning that they demonstrated an understanding of computation as a problem-solving framework.


Figure 1. Percentage of eighth-grade students identified as at or above proficient, by assessment: 2018


There are many factors that may affect performance on these assessments, such as access to technology, devices, hardware, and software; access to learning opportunities using technology; amount of experience using technology; and attitudes toward technology.

Student’s participation in technology-related classes

For NAEP TEL, students were asked if they were currently taking a technology-related class or if they had taken one in the past. In 2018, about 57 percent of U.S. eighth-grade students were either currently enrolled in or had taken at least one technology-related class, such as industrial technology, engineering, or a class that involved learning to use, program, or build computers (table 1). In addition, a higher percentage of students who had completed a technology-related class before or during eighth-grade scored at or above the NAEP Proficient level, compared with students who had not completed such a class (table 2).




Overall, both assessments show that a portion of students demonstrated the knowledge and skills necessary to meet the assessment’s defined proficiency measures, although more than half reported taking technology-related classes before or during eighth grade.

It should be noted that the data presented here were collected in 2018; any changes since then due to the coronavirus pandemic or other factors are not reflected in the results reported here. The NAEP TEL and ICILS samples both include public and private schools, but the two assessments use different methods for reporting student performance. Data from these assessments do not support any causal inferences as they are not experimental studies. Data reported in the text and figures are rounded to the nearest integer.

 

Resources for more information:

By Mary Ann Fox, AIR, and Taslima Rahman, NCES

Teaching with Technology: U.S. Teachers’ Perceptions and Use of Digital Technology in an International Context

The coronavirus pandemic forced teachers across the world to immediately transition instruction to a virtual setting in early 2020. To understand U.S. teachers’ level of preparedness for this shift in an international context, this blog examines recent international data from U.S. teachers’ responses to questions on the following topics:

  • Their perceptions of information and communications technologies (ICT) resources
  • Their use of ICT for instruction prior to the pandemic

In general, the results suggest that U.S. teachers are more resourced in ICT than their international peers, and they use ICT at a similar frequency at school when teaching.

 

Teachers’ perceptions of ICT resources at their school

The quantity and quality of ICT resources available in school systems prior to the coronavirus pandemic may impact teachers’ access to such resources for instructional purposes while classrooms are functioning in a virtual format. The United States participated in the 2018 International Computer and Information Literacy Study (ICILS), which asked questions about ICT resources to a nationally representative sample of eighth-grade teachers from 14 education systems.

The results from this study show that 86 percent of eighth-grade teachers both in the United States and across ICILS 2018 education systems “strongly agreed” or “agreed” that ICT is considered a priority for use in teaching (figure 1). Compared with the ICILS 2018
averages,[1] higher percentages of U.S. eighth-grade teachers “strongly agreed” or “agreed” with various statements about the use of ICT.

While 86 percent of U.S. eighth-grade teachers “strongly agreed” or “agreed” that “ICT is considered a priority for use in teaching,” only 61 percent “strongly agreed” or “agreed” that “there is sufficient opportunity for me to develop expertise in ICT” (figure 1). Additionally, 62 percent of U.S. eighth-grade teachers “strongly agreed” or “agreed” that “there is enough time to prepare lessons that incorporate ICT.” These disparities may have had an impact on teacher capacity during the sudden shift to 100 percent online learning as a result of the coronavirus pandemic, which would be a good topic for future research and analyses.  


Figure 1. Percentage of eighth-grade teachers who reported that they “strongly agree” or “agree” with statements about using ICT in teaching at school, by statement: 2018

p < .05. Significantly different from the U.S. estimate at the .05 level of statistical significance.

¹ Did not meet the guidelines for a sample participation rate of 85 percent and not included in the international average.
² National Defined Population covers 90 to 95 percent of National Target Population.
NOTE: ICT = information and communications technologies. The ICILS 2018 average is the average of all participating education systems meeting international technical standards, with each education system weighted equally. Statements are ordered by the percentages of U.S. teachers reporting “strongly agree” or “agree” from largest to smallest.
SOURCE: International Association for the Evaluation of Educational Achievement (IEA), The International Computer and Information Literacy Study (ICILS), 2018. Modified reproduction of figure 17 from U.S. Results from the 2018 ICILS Web Report.


Teachers’ perceptions of the use of ICT for instruction

Teachers’ views on the role of ICT in virtual instruction during the coronavirus pandemic are not yet clear. However, in 2018, when instruction was conducted in physical classrooms, most U.S. eighth-grade teachers participating in ICILS expressed positive perceptions about “using ICT in teaching and learning at school,” as did many teachers internationally.

Among eighth-grade teachers in the United States, 95 percent agreed that ICT “enables students to access better sources of information,” 92 percent agreed that ICT “helps students develop greater interest in learning,” and 92 percent agreed that ICT “helps students work at a level appropriate to their learning needs.” On average across other education systems participating in ICILS, at least 85 percent of teachers agreed with each of these statements (Fraillon et al. 2019).

Seventy-five percent of U.S. eighth-grade teachers in 2018 agreed that ICT “improves academic performance of students,” which was higher than the ICILS international average of 71 percent. The percentages of teachers who agreed with this statement varied across education systems, from three-quarters or more of teachers in Chile, Denmark, Kazakhstan, and Portugal to less than half of teachers in Finland and North Rhine-Westphalia (Germany).

 

Frequency of teachers’ use of ICT

Teachers’ reported use of ICT for instruction in physical classroom settings may provide insight into their level of experience as they transition to virtual settings during the coronavirus pandemic.

In 2018, half of U.S. eighth-grade teachers reported “using ICT at school when teaching” every day, which was not significantly different from the ICILS average of 48 percent. However, the U.S. percentage was lower than the percentages of teachers in Moscow (76 percent), Denmark (72 percent), and Finland (57 percent) (figure 2).


Figure 2. Percentage of eighth-grade teachers who reported using ICT at school every day when teaching, by education system: 2018

p < .05. Significantly different from the U.S. estimate at the .05 level of statistical significance.
¹ Met guidelines for sample participation rates only after replacement schools were included.
² National Defined Population covers 90 to 95 percent of National Target Population.
³ Did not meet the guidelines for a sample participation rate of 85 percent and not included in the international average.
⁴ Data collected at the beginning of the school year.
NOTE: ICT = information and communications technologies. The ICILS 2018 average is the average of all participating education systems meeting international technical standards, with each education system weighted equally. Education systems are ordered by their percentages of teachers reporting using ICT at school when teaching from largest to smallest. Italics indicate the benchmarking participants.
SOURCE: International Association for the Evaluation of Educational Achievement (IEA), The International Computer and Information Literacy Study (ICILS), 2018. Modified reproduction of figure 15 from U.S. Results from the 2018 ICILS Web Report.


For more information on teachers and technology, check out NCES’s ICILS 2018 website, the international ICILS website, and the earlier NCES blog “New Study on U.S. Eighth-Grade Students’ Computer Literacy.”

 

By Amy Rathbun, AIR, and Stephen Provasnik, NCES

 


[1] The ICILS average is the average of all participating education systems meeting international technical standards, with each education system weighted equally. The United States did not meet the guidelines for a sample participation rate of 85 percent, so it is not included in the international average.

 

Reference

Fraillon, J., Ainley, J., Schulz, W., Friedman, T., and Duckworth, D. (2019). Preparing for Life in a Digital World: IEA International Computer and Information Literacy Study 2018 International Report. Amsterdam: International Association for the Evaluation of Educational Achievement.

New Education Data from the Household Pulse Survey

Recognizing the extraordinary information needs of policymakers during the coronavirus pandemic, NCES joined a partnership with the Census Bureau and four other federal statistical agencies to quickly develop a survey to gather key indicators of our nation’s response to the global pandemic. Thus, the experimental 2020 Household Pulse Survey began development on March 23, 2020, and data collection began on April 23, 2020. This new survey provides weekly national and state estimates, which are released to the public in tabular formats one week after the end of data collection.

The Household Pulse Survey gathers information from adults about employment status, spending patterns, food security, housing, physical and mental health, access to health care, and educational disruption. The education component includes questions about the following:

  • The weekly time spent on educational activities by students in public and private elementary and secondary schools
  • The availability of computer equipment and the Internet for instructional purposes
  • The extent to which computer equipment and the Internet for students were provided or subsidized

Since this survey is designed to represent adults 18 years old and over, the responses to the education questions concern students within the households of adults 18 years old and over, not the percentage of students themselves.

In the Household Pulse Survey during the weeks of April 23 through May 5, adults reported that their average weekly time spent on teaching activities with elementary and secondary students in their household was 13.1 hours. These results differed by educational attainment: adults who had not completed high school reported a weekly average of 9.9 hours in teaching activities with children, whereas adults with a bachelor’s or higher degree reported 13.9 hours (figure 1). In terms of the average weekly time spent on live virtual contact between students in their household and their teachers, adults reported a lower average of 4.1 hours.



Adults’ reports about the school instruction model need to be interpreted carefully because respondents could choose multiple types of approaches. A higher percentage of adults with a bachelor’s or higher degree (84 percent) reported that classes for elementary and secondary students in their household had moved to a format using online resources than did adults who had completed some college or an associate’s degree (74 percent), adults who had completed only high school (64 percent), or adults who had not completed high school (57 percent).

Higher percentages of adults with higher levels of education than of adults with lower levels of education reported that computers and the Internet were always available for educational purposes for elementary and secondary students in their households (figure 2).



The percentage of adults who reported that the school district provided a computer or digital device for children in their households to use at home for educational purposes was higher for adults who had not completed high school (44 percent) than for adults with a bachelor’s or higher degree (33 percent). Also, a higher percentage of adults who had not completed high school than of adults with higher levels of educational attainment reported financial assistance for student Internet access.

It is important to note that the speed of the survey development and the pace of the data collection efforts have led to policies and procedures for the experimental Household Pulse Survey that are not always consistent with traditional federal survey operations. Data should be interpreted with proper caution.  

More information on the Household Pulse Survey, detailed statistical tables, and microdata sets are available at https://www.census.gov/householdpulsedata. The Household Pulse Survey site includes breakouts of the data by other characteristics, such as race/ethnicity. In addition to participating in the development of this new survey, NCES has also generated new analyses based on existing data that respond to new needs for policy information, such as the availability of the Internet for student learning.

 

By Xiaolei Wang, AIR

The NCES Ed Tech Equity Initiative Framework

In a recent blog post, NCES announced the groundbreaking work of the NCES Ed Tech Equity Initiative. The Center’s efforts for this initiative focus on working with stakeholders to identify how NCES data collection, reporting, and dissemination efforts can better inform the relationship between technology and K–12 students’ educational experiences and outcomes. 

THE FRAMEWORK

As part of these efforts, NCES developed a framework to better understand the various facets that influence technology in K–12 education, as well as how these facets interact. The framework was created through extensive research and is designed to be revised over time to align with changes in the ed tech equity space.

The NCES Ed Tech Equity Framework, included below, is comprised of four critical components—Indicators (located in the center of the framework), Dimensions and Environments (the green and purple circle), and Change Agents (shown in the outer gray circle).

HOW IT WORKS

The interaction of the framework elements informs ed tech equity and NCES data collection:

  • Indicators represent the broad categories used to measure or assess education technology—relevant NCES survey questions will fit within at least one of the Indicator categories.
  • Dimensions are the key perspectives through which NCES focuses its ed tech equity data collection efforts.
  • Environments are the settings that facilitate educational experiences.
  • Finally, Change Agents are factors that impact or influence students’ educational experiences and outcomes.

Below, a few existing NCES items are mapped to the framework to illustrate how it will be used in NCES data collection:

  • TECHNOLOGY RESOURCES AND SUPPORT
  • TEACHING IN-SCHOOL: In this school year, did your school offer training for teachers on how to use computers or other digital devices?  —NAEP, 2017

  • TECHNOLOGY KNOWLEDGE, SKILLS, AND ATTITUDES
  • TEACHING OUT-OF-SCHOOL: During the last 12 months, which of the following activities have you or another family member done with [your 9th grader]?

- Worked or played on a computer together  —HSLS, 2009

  • INTEGRATION OF TECHNOLOGY
  • LEARNING IN SCHOOLDo you use the Internet to do any of the following tasks for schoolwork (including classroom tasks, homework, studying outside of class)?

- c) Collaborate with classmates on assignments or projects  —TIMSS, 2015

NEXT STEPS

NCES recently convened an expert panel to assist with evaluating NCES’ existing technology-related efforts and provide recommendations on priorities for future NCES data collection, reporting, and dissemination. Feedback from the panel will assist us in our efforts to provide greater focus on the relationship between technology and K–12 students’ educational experiences and outcomes. We plan to share insights from the expert panel meeting in an upcoming blog post.

 

By Halima Adenegan, NCES, and Emily Martin, Hager Sharp

 

The Digital Divide: Differences in Home Internet Access

The expanding use of technology affects the lives of students both inside and outside the classroom. While exposure to learning technology inside schools and classrooms is important, access can also differ once those students are in their homes. It’s important for educators to be aware of the potential barriers to technology and internet access that students may face. A recent report from NCES, Student Access to Digital Learning Resources Outside the Classroom, highlighted some differences in home internet access for students.

The percentage of 5- to 17-year-old students with either no internet access or only dial-up access differed by students’ race/ethnicity.

Access also differed geographically. Remote rural locales had the highest percentage of students with either no internet access or only dial up access at home. Within these remote rural areas, the percentage of students lacking access differed by students’ race/ethnicity. Forty-one percent of Black students and 26 percent of Hispanic students living in remote rural areas had either no internet access or only dial up access at home. This was higher than the percentage of White students (13 percent) and Asian students (11 percent) living in remote rural areas who had either no internet access or only dial up access at home.   

The percentage of students who had no access to the Internet or only dial-up access was higher for students living below the poverty threshold (26 percent) than for students living between 100 and 185 percent of the poverty threshold (15 percent) and at greater than 185 percent of the poverty threshold (4 percent).

In 2015, the two most common main reasons for children ages 3 to 18 to not have home internet access were that it was too expensive or that the family did not believe they needed it/ were not interested in having it (38 percent each). Other main reasons for not having home internet access included that the home lacked a computer or a computer adequate for internet use (8 percent), internet service was not available in the area (5 percent), the Internet could be used somewhere else (3 percent), and privacy and security concerns (i.e., online privacy and cybersecurity and personal safety concerns) (2 percent). 

Browse the full report for more data on additional topics relating to differences in access to technology and the internet.

 

By Lauren Musu