IES Blog

Institute of Education Sciences

What National and International Assessments Can Tell Us About Technology in Students’ Learning: Eighth-Graders’ Readiness to Use Technology

Across the country in 2020, students, teachers, and parents have had to adapt to changes in the delivery of education instruction due to the coronavirus pandemic and turn to information and communication technologies (ICT) to learn, interact, and assess progress. It is important that we are able to assess students’ abilities to understand and use technology now more than ever. This post, the first in a three-part series, discusses the results of two technology-focused assessments, NAEP TEL and ICILS, which NCES administered in 2018 (see textbox for more information about these assessments).

Students’ performance on NAEP TEL and ICILS

According to the 2018 NAEP TEL, 46 percent of U.S. eighth-grade students scored at or above the NAEP Proficient level, meaning that they were able to demonstrate the selection and use of an appropriate range of tools and media (figure 1). According to the 2018 ICILS, 25 percent of eighth-grade students scored at or above proficiency level 3 for computer and information literacy—that is, they demonstrated the capacity to work independently when using computers as information-gathering and management tools. In addition, 20 percent of eighth-grade students scored in the upper region for computational thinking, meaning that they demonstrated an understanding of computation as a problem-solving framework.


Figure 1. Percentage of eighth-grade students identified as at or above proficient, by assessment: 2018


There are many factors that may affect performance on these assessments, such as access to technology, devices, hardware, and software; access to learning opportunities using technology; amount of experience using technology; and attitudes toward technology.

Student’s participation in technology-related classes

For NAEP TEL, students were asked if they were currently taking a technology-related class or if they had taken one in the past. In 2018, about 57 percent of U.S. eighth-grade students were either currently enrolled in or had taken at least one technology-related class, such as industrial technology, engineering, or a class that involved learning to use, program, or build computers (table 1). In addition, a higher percentage of students who had completed a technology-related class before or during eighth-grade scored at or above the NAEP Proficient level, compared with students who had not completed such a class (table 2).




Overall, both assessments show that a portion of students demonstrated the knowledge and skills necessary to meet the assessment’s defined proficiency measures, although more than half reported taking technology-related classes before or during eighth grade.

It should be noted that the data presented here were collected in 2018; any changes since then due to the coronavirus pandemic or other factors are not reflected in the results reported here. The NAEP TEL and ICILS samples both include public and private schools, but the two assessments use different methods for reporting student performance. Data from these assessments do not support any causal inferences as they are not experimental studies. Data reported in the text and figures are rounded to the nearest integer.

 

Resources for more information:

By Mary Ann Fox, AIR, and Taslima Rahman, NCES

Listening to Schools: The National Teacher and Principal Survey (NTPS) Shares Educators’ Perspectives on Coronavirus Impacts on Education

As the 2020–21 school year gets underway, many are considering the tremendous impact the coronavirus will have on classrooms—whether in person or virtual—across the United States. What can be done to support policymakers and education sector leaders as they strive to address, amongst other concerns, potentially unequitable learning opportunities and mental health challenges?

The National Teacher and Principal Survey (NTPS) will gather critical information from teachers and principals about the changes implemented and lessons learned by schools and their staff during the 2020–21 school year. The NTPS is a nationwide system of related surveys that collect data on elementary and secondary education in the United States, including teaching and working conditions in schools and characteristics of public and private school teachers and principals at the state level.

Conducted every 2 to 3 years, NTPS provides critical representative data to policymakers and researchers on school organization, staff evaluations, teacher and principal preparation and professional development, classes taught, school characteristics, demographics of the teacher and principal labor force, and other important education topics. These data serve to inform those who set funding and other priorities, including Congress, the U.S. Department of Education, state education agencies, and public school districts. The data also allow important comparative analyses of key education personnel in public and private educational settings.

Unlike many other studies capturing information about the coronavirus and related education issues, NTPS allows for comparisons at the national level, between states for public schools and by affiliation for private schools, and over time. It is important that NTPS questionnaires reach selected teachers, principals, and other staff during these changing times. The information they provide will help decision makers evaluate the effects of school workplace conditions, salaries, and training opportunities on the educational workforce and aid in the U.S Department of Education’s program planning in the areas of teacher recruitment and retention, teaching policies, and teacher education.

But participation is key! If school staff do not participate in NTPS when selected, the data will provide an incomplete and possibly misleading description of the impact of the coronavirus on school communities, potentially affecting funding and other policy decisions.  

 

What do we know about the coronavirus and the 202021 school year?

By May 2020, in just the United States alone, at least 50.8 million public school students were affected by ordered or recommended K–12 school closures. According to Education Week reports, school districts unveiled a variety of reopening plans for the 2020–21 school year that include remote, hybrid or partial in-person, or full in-person learning approaches. As of September 2, 73 percent of the 100 largest school districts had announced they wiould resume with remote learning only. Information from the Census Bureau’s experimental weekly Household Pulse Survey suggests that students in one of every six households do not usually have access to the internet for education purposes.

Regardless of each school district’s decision for the beginning of the school year, local and state education leaders are responsible for numerous decisions on behalf of their schools, students, teachers, and school staff that rely heavily on the availability of reliable data. But most of the existing information on the education sector’s response to the coronavirus is at the district or state level and does not typically include information about experiences of teachers and principals directly from these critical education providers. Also, because of varying reporting resources and practices across the nation’s 130,000 public and private K–12 schools, a consistent national-level understanding of coronavirus-related education problems is not readily available. As a result, key policymakers and other decision makers currently have little information from individual teachers or principals about coronavirus-related problems specifically and education issues more generally. NTPS provides an opportunity for the voices of teachers, principals, and other school staff to be included in the conversation.

 

How can NTPS show what is happening in K–12 schools across the United States?

Last administered in 2017–18, the NTPS is set to resume in the 2020–21 school year and new questions have been added to reflect changes that may have occurred due to the coronavirus. This offers an opportunity for sampled teachers, principals, and schools to provide valuable data that explain their experience as educators during the coronavirus pandemic. In addition to new items, data gleaned from recurring questions will capture changes over time and yield important insights into areas of success and areas in need of further support. Data from prior school years has already been used during the pandemic to highlight differences in the number of health staff, such as school nurses, and mental health staff, such as counselors, psychologists, and social workers. Gathering more responses to these and other questions will allow for trend analyses, giving policymakers and other decision makers a better understanding of changes occurring at the teacher, principal, and school levels.

 

How will the survey be conducted?

The NTPS data collection process is both voluntary and self-administered, meaning all questionnaires can be completed without any in-person contact and without interruption for staff who may be fully working remotely. Teachers, principals, and schools who have been sampled to participate in the survey will be contacted by mail and e-mail and invited to complete the questionnaires online. Sampled participants will also receive paper surveys at their school mailing addresses.

 

Why is this survey important?

Responses from sampled schools ensure that NTPS estimates are reliable and accurately reflect the activities of all U.S. public and private schools. These data are vital as policymakers, researchers, families, and school staff strive to understand and respond to the effects of the current pandemic and build a better, stronger education sector for the future—including improved response options for potential future pandemics. NTPS responses during the 2020–21 school year can be compared to data from future NTPS cycles to understand possible longer-term impacts of the current significant changes to education delivery in the country and across the states. These data provide national and state policymakers with a distinct understanding of the condition of K–12 education in their communities and will remain important as leaders monitor changes in the education sector in future years.

For more information about the National Teacher and Principal Survey (NTPS), please visit https://nces.ed.gov/surveys/ntps/.

 

By Julia Merlin, NCES

Research on Adult Literacy: A History of Investment in American Adults

Reading is fundamental, but it is also difficult to master, taking thousands of hours of instruction and practice. Roughly 52 percent of U.S. adults over the age of 16 may struggle with everyday literacy tasks. Of these adults, approximately 20 percent may perform at very low levels of literacy. For adults who are still mastering of this skill, the task can seem overwhelming.

Luckily, IES-funded researchers have been working towards solutions for adults with low basic reading skills and are creating and refining assessments, curricula, and software. These innovations aim to help adult learners, the instructors and tutors who work with them, and the programs that support them.

As part of our commemoration of National Adult Education and Family Literacy Week (September 20-26, 2020), we would like to recognize the history of adult literacy research at IES and its National Center for Education Research.  

Since 2004, IES-funded researchers have been developing assessments to help identify the needs of adults struggling with literacy and working on solutions to build adult literacy skills. This work fed into the measurement component of IES’s Reading for Understanding Initiative in 2010 and later returned back to addressing adult basic literacy measurement in 2016.

In 2012, IES funded the Center for the Study of Adult Literacy (CSAL), which developed a curriculum and technology for adults reading between the 3rd- and 8th-grade levels. CSAL demonstrates how adult literacy research benefits by integrating research conducted with students with disabilities and those in K-12 and postsecondary settings. In fact, the researchers pulled upon findings from eight prior IES grants funded by NCER and NCSER.

Our researchers are also developing a clearer picture of the adults who fall into the broad category of those with low literacy. They are leveraging the PIAAC data set to conduct exploratory work that informs both our understanding of those at the very low ends of literacy and also of whether basic skills may predict success in postsecondary career and technical education programs.

In 2020, IES funded additional development research to help refine an interactive, online reading comprehension program, AutoTutor for Adult Reading Comprehension (AT-ARC). Another project will recruit and train postdoctoral fellows to cultivate the next generation of researchers who can continue to build a research base for improving adult literacy outcomes.

Although IES researchers are making great strides to build knowledge, the field needs more information, and adult learners deserve tools and innovations developed for their specific needs and goals. IES hopes to continue to support such work.

 


To learn more about IES-wide efforts to understand and improve adult learners’ outcomes, visit the Adult Basic Skills topic page. Contact Dr. Meredith Larson for more information about the research supported by NCER.

 

Building Knowledge About Adult Education

 

Adult education programs aim to support the millions of American adults who wish to strengthen their basic skills, earn a high school degree or equivalent, or become U.S. citizens. During the National Adult Education Family and Literacy Week (September 20-26, 2020), IES is highlighting the ongoing research it supports to help expand our knowledge of and innovations for the adult education system and the communities they support.

Roughly 30 million U.S. adults may lack a high school degree or equivalent. And according to an assessment conducted in 2017, approximately 30 percent of adults may have very low numeracy skills, and approximately 20 percent may have very low literacy skills. Adults with low levels of academic attainment or low basic skills may face barriers to full economic or civic engagement.

Adult education programs aim to help such adults. These programs often run on lean budgets and need to meet the needs of a highly diverse population with a multitude of learning goals.

In order to help understand and improve the ability of the adult education system to provide services, IES researchers have been conducting various research projects in partnership with adult education providers.

For example, Career Pathways Programming for Lower-Skilled Adults and Immigrants conducted mixed-methods research in partnership with Chicago, Houston, and Miami. This work helped each city understand what types of career pathways adult education programs were offering and who was participating in such programing. This type of information helps programs, cities, and the system more broadly understand and adjust to the needs of learners and communities.

The New York State Literacy Zone Researcher-Practitioner Partnership focused on improving the ability of case managers to help adult learners leverage wrap-around services and access and succeed in adult education and training programs. This project helped develop tools and training for case managers and conduct an exploratory pilot study of these tools to see if they predicted learner outcomes, such as persistence in a program.

The Georgia Partnership for Adult Education and Research (GPAER) is a collaboration among researchers at Georgia State University and leadership at the Georgia Office of Adult Education: Technical College System of Georgia to help understand adult literacy programs across the state. This ongoing work is conducting mixed-methods studies to understand program features, learner characteristics, and indicators of beneficial learner outcomes.

There is still much to learn to help improve the ability of programs to find and support adult learners. IES encourages additional research to further help us understand the landscape of adult education, the needs and interests of adult learners and their instructors, and the outcomes and impacts of improving adult basic skills.


For more information about adult education research at the National Center of Education Research, contact Dr. Meredith Larson.

 

 

Back to School During COVID19: Developers and Researchers Continue to Respond to Support In-Class and Remote Teaching and Learning

 

Many programs across the Federal government, such as the ED/IES Small Business Innovation Research (SBIR) and the IES Research Grants programs, fund projects to develop and evaluate new forms of education technology and interventions that can be implemented to support instruction and learning at schools and for remote learning. More than 150 of these technologies were demoed in January 2020 at the ED Games Expo, a showcase for learning games and technologies developed with support from IES and more than 30 other Federal programs.

Since the global outbreak of COVID19 and the closure of schools across the United States and the world, a group of government-supported developers and researchers responded to provide resources to educators, students, and families to facilitate remote learning. More than 50 developers and researchers offered 88 learning games and technologies at no cost through the end of the school year for use in distance learning settings with internet access (see this blog for the list). In addition, many of the developers and researchers provided technical assistance directly to individual teachers to support implementation at a distance, and many created new materials and worked to refine and adapt their products to optimize usability and feasibility for fully remote use. More than a million students and thousands of educators used these learning technologies during the spring.

In April and May 2020, more than 70 developers and researchers partnered to produce and participate in a series of free day-long virtual events, which were called “unconferences.” The events featured presentations on innovative models and approaches to teaching and learning remotely and provided an in-depth look at the learning games and technologies created by the presenters. More than 25,000 educators attended these virtual events in real-time, hundreds asked questions and made comments through chats during the events, and many thousands more have accessed these videos after the events. See this blog for the list of archived videos.

A New Resource: Guides to Education Technologies that are Ready Now

As schools begin re-opening for the new school year, a group of 70 developers and researchers have collaborated to produce a new series of Guides to Education Technologies. The guides present information on government-supported education technology products that are ready now for in-class and remote learning. All the resources are web-based and can be used on either computers, tablets, or personal devices. The resources in the guides include a mix of no-cost products as well as ones that are fee-based.  

With awards from government programs, all of the resources were developed through an iterative process with feedback from teachers and students, and most were evaluated through small pilot studies to measure the promise of the technologies to support improvements in student learning and relevant educational outcomes. All the products were used and demonstrated to be feasible for use in remote settings in the spring after the onset of the pandemic.

The guides present resources appropriate for young children through postsecondary students in education and special education, for English learners, and for teachers in education and special education across a wide range of educational topics. Many of the technologies personalize learning by adjusting content to students as they go and present information to educators to inform instruction.

The Guides focus on the following areas and can be accessed below:

 

Stay tuned to the Inside IES Blog for more information and resources about the response to the COVID-19 in education.


Edward Metz (Edward.Metz@ed.gov) is a research scientist and the program manager for the Small Business Innovation Research Program at the US Department of Education’s Institute of Education Sciences.