IES Blog

Institute of Education Sciences

Program for the International Assessment of Adult Competencies (PIAAC) 2022–23 Data Collection Begins

Last month, the National Center for Education Statistics (NCES) kicked off a major survey of adults (ages 16–74) across the nation to learn about their literacy skills, education, and work experience. Information collected through this survey—officially known as Cycle 2 of the Program for the International Assessment of Adult Competencies (PIAAC) in the United States—is used by local, state, and national organizations, government entities, and researchers to learn about adult skills at the state and local levels (explore these data in the PIAAC Skills Map, shown below).


Image of PIAAC Skills Map on state and county indicators of adult literacy and numeracy


Specifically, these data are used to support educational and training initiatives organized by local and state programs. For example, the Houston Mayor’s Office for Adult Literacy has used the PIAAC Skills Map data in developing the Adult Literacy Blueprint, a comprehensive plan for coordinated citywide change to address the systemic crisis of low literacy and numeracy in the city. In addition, the Kentucky Career and Technical College System developed a comprehensive data-driven app for workforce pipeline planning using the county-level PIAAC Skills Map data as one of the education pipeline indicators.

This is not the first time NCES is administering PIAAC. NCES collected PIAAC data three times between 2011 and 2017, when the first cycle of this international study was administered in 39 countries. Developed by the Organization for Economic Cooperation and Development (OECD), PIAAC measures fundamental cognitive and workplace skills needed for individuals to participate in society and for economies to prosper. Among these fundamental skills are literacy, numeracy, and digital problem-solving. Data from the first cycle of PIAAC (2011–17) provided insights into the relationships between adult skills and various economic, social, and health outcomes—both across the United States as a whole and for specific populations of interest (e.g., adults who are women, immigrants, older, employed, parents, or incarcerated). The OECD and NCES have published extensively using these data.

The current cycle (Cycle 2) of PIAAC will resemble the first cycle in that interviewers will visit people’s homes to ask if they are willing to answer background questionnaire and take a self-administered test of their skills. However, unlike the first cycle when respondents could respond to the survey on paper or on a laptop, this cycle will be conducted entirely on a tablet. PIAAC is completely voluntary, but each respondent is specifically selected to provide invaluable information that will help us learn about the state of adult skills in the country (participants can also receive an incentive payment for completing the survey).

PIAAC’s background questionnaire includes questions about an individual’s demographics, family, education, employment, skill use, and (new in Cycle 2 and unique to the United States) financial literacy. The PIAAC test or “direct assessment” measures literacy, numeracy, and (new in Cycle 2) adaptive problem-solving skills of adults.1

Each sampled person’s response is not only kept confidential but also “anonymized” before the data are released (so that no one can ever definitively identify an individual from personal characteristics in the datafile).

The international report and data for PIAAC Cycle 2 is scheduled to be released by the OECD in December 2024.

Be sure to follow NCES on TwitterFacebookLinkedIn, and YouTube and subscribe to the NCES News Flash to stay up-to-date on PIAAC report and data releases and resources.

 

By Saida Mamedova, AIR, Stephen Provasnik, NCES, and Holly Xie, NCES


[1] Data is collected from adults ages 16–74 in the United States and ages 16–65 in the other countries.

Rescaled Data Files for Analyses of Trends in Adult Skills

In January 2022, NCES released the rescaled data files for three adult literacy assessments conducted several decades earlier: the 1992 National Adult Literacy Survey (NALS), the 1994 International Adult Literacy Survey (IALS), and the 2003 Adult Literacy and Lifeskills Survey (ALL). By connecting the rescaled data from these assessments with data from the current adult literacy assessment, the Program for the International Assessment of Adult Competencies (PIAAC), researchers can examine trends on adult skills in the United States going back to 1992. This blog post traces the history of each of these adult literacy assessments, describes the files and explains what “rescaling” means, and discusses how these files can be used in analyses in conjunction with the PIAAC files. The last section of the post offers several example analyses of the data.

A Brief History of International and National Adult Literacy Assessments Conducted in the United States

The rescaled data files highlighted in this blog post update and combine historical data from national and international adult literacy studies that have been conducted in the United States.

NALS was conducted in 1992 by NCES and assessed U.S. adults in households, as well as adults in prisons. IALS—developed by Statistics Canada and ETS in collaboration with 22 participating countries, including the United States—assessed adults in households and was administered in three waves between 1994 and 1998. ALL was administered in 11 countries, including the United States, and assessed adults in two waves between 2003 and 2008.

PIAAC seeks to ensure continuity with these previous surveys, but it also expands on their quality assurance standards, extends the definitions of literacy and numeracy, and provides more information about adults with low levels of literacy by assessing reading component skills. It also, for the first time, includes a problem-solving domain to emphasize the skills used in digital (originally called “technology-rich”) environments.

How Do the Released Data Files From the Earlier Studies of Adult Skills Relate to PIACC?

All three of the released restricted-use data files (for NALS, IALS, and ALL) relate to PIAAC, the latest adult skills assessment, in different ways.

The NALS data file contains literacy estimates and background characteristics of U.S. adults in households and in prisons in 1992. It is comparable to the PIAAC data files for 2012/14 and 2017 through rescaling of the assessment scores and matching of the background variables to those of PIAAC.

The IALS and ALL data files contain literacy (IALS and ALL) and numeracy (ALL) estimates and background characteristics of U.S. adults in 1994 (IALS) and 2003 (ALL). Similar to NALS, they are comparable to the PIAAC restricted-use data (2012/14) through rescaling of the literacy and numeracy assessment scores and matching of the background variables to those of PIAAC. These estimates are also comparable to the international estimates of skills of adults in several other countries, including in Canada, Hungary, Italy, Norway, the Netherlands, and New Zealand (see the recently released Data Point International Comparisons of Adult Literacy and Numeracy Skills Over Time). While the NCES datasets contain only the U.S. respondents, IALS and ALL are international studies, and the data from other participating countries can be requested from Statistics Canada (see the IALS Data Files/Publications and ALL Data pages for more detail). See the History of International and National Adult Literacy Assessments page for additional background on these studies. 

Table 1 provides an overview of the rescaled NALS, IALS, and ALL data files.


Table 1. Overview of the rescaled data files for the National Adult Literacy Survey (NALS), International Adult Literacy Survey (IALS), and Adult Literacy and Lifeskills Survey (ALL) 

Table showing overview of the rescaled data files for the National Adult Literacy Survey (NALS), International Adult Literacy Survey (IALS), and Adult Literacy and Lifeskills Survey


What Does “Rescaled” Mean?

“Rescaling” the literacy (NALS, IALS, ALL) and numeracy (ALL) domains from these three previous studies means that the domains were put on the same scale as the PIAAC domains through the derivation of updated estimates of proficiency created using the same statistical models used to create the PIAAC skills proficiencies. Rescaling was possible because PIAAC administered a sufficient number of the same test questions used in NALS, IALS, and ALL.1 These rescaled proficiency estimates allow for trend analysis of adult skills across the time points provided by each study.

What Can These Different Files Be Used For?

While mixing the national and international trend lines isn’t recommended, both sets of files have their own distinct advantages and purposes for analysis.

National files

The rescaled NALS 1992 files can be used for national trend analyses with the PIAAC national trend points in 2012/2014 and 2017. Some potential analytic uses of the NALS trend files are to

  • Provide a picture of the skills of adults only in the United States;
  • Examine the skills of adults in prison and compare their skills with those of adults in households over time, given that NALS and PIAAC include prison studies conducted in 1992 and 2014, respectively;
  • Conduct analyses on subgroups of the population (such as those ages 16–24 or those with less than a high school education) because the larger sample size of NALS allows for more detailed breakdowns along with the U.S. PIAAC sample;
  • Focus on the subgroup of older adults (ages 66–74), given that NALS sampled adults over the age of 65, similar to PIAAC, which sampled adult ages 16–74; and
  • Analyze U.S.-specific background questions (such as those on race/ethnicity or health-related practices).

International files

The rescaled IALS 1994 and ALL 2003 files can be used for international trend analyses among six countries with the U.S. PIAAC international trend point in 2012/2014: Canada, Hungary, Italy, Norway, the Netherlands, and New Zealand. Some potential analytic uses of the IALS and ALL trend files are to

  • Compare literacy proficiency results internationally and over time using the results from IALS, ALL, and PIAAC; and
  • Compare numeracy proficiency results internationally and over time using the results from ALL and PIAAC.

Example Analyses Using the U.S. Trend Data on Adult Literacy

Below are examples of a national trend analysis and an international trend analysis conducted using the rescaled NALS, IALS, and ALL data in conjunction with the PIAAC data.

National trend estimates

The literacy scores of U.S. adults increased from 269 in NALS 1992 to 272 in PIAAC 2012/2014. However, the PIAAC 2017 score of 270 was not significantly different from the 1992 or 2012/2014 scores.


Figure 1. Literacy scores of U.S. adults (ages 16–65) along national trend line: Selected years, 1992–2017

Line graph showing literacy scores of U.S. adults (ages 16–65) along national trend line for NALS 1992, PIAAC 2012/2014, and PIAAC 2017

* Significantly different (p < .05) from NALS 1992 estimate.
SOURCE: U.S. Department of Education, National Center for Education Statistics, National Adult Literacy Survey (NALS), NALS 1992; and Program for the International Assessment of Adult Competencies (PIAAC), PIAAC 2012–17.


International trend estimates

The literacy scores of U.S. adults decreased from 273 in IALS 1994 to 268 in ALL 2003 before increasing to 272 in PIAAC 2012/2014. However, the PIAAC 2012/2014 score was not significantly different from the IALS 1994 score.


Figure 2. Literacy scores of U.S. adults (ages 16–65) along international trend line: Selected years, 1994–2012/14

Line graph showing literacy scores of U.S. adults (ages 16–65) along international trend line for IALS 1994, ALL 2003, and PIAAC 2012/2014

* Significantly different (p < .05) from IALS 1994 estimate.
SOURCE: U.S. Department of Education, National Center for Education Statistics, Statistics Canada and Organization for Economic Cooperation and Development (OECD), International Adult Literacy Survey (IALS), 1994–98; Adult Literacy and Lifeskills Survey (ALL), 2003–08; and Program for the International Assessment of Adult Competencies (PIAAC), PIAAC 2012/14. See figure 1 in the International Comparisons of Adult Literacy and Numeracy Skills Over Time Data Point.


How to Access the Rescaled Data Files

More complex analyses can be conducted with the NALS, IALS, and ALL rescaled data files. These are restricted-use files and researchers must obtain a restricted-use license to access them. Further information about these files is available on the PIAAC Data Files page (see the “International Trend Data Files and Data Resources” and “National Trend Data Files and Data Resources” sections at the bottom of the page).

Additional resources:

By Emily Pawlowski, AIR, and Holly Xie, NCES


[1] In contrast, the 2003 National Assessment of Adult Literacy (NAAL), another assessment of adult literacy conducted in the United States, was not rescaled for trend analyses with PIAAC. For various reasons, including the lack of overlap between the NAAL and PIAAC literacy items, NAAL and PIAAC are thought to be the least comparable of the adult literacy assessments.

Real-World Responses in Real Time: Social Inequality in Access to COVID-19 Information

Whether you are a trained healthcare provider, a grocery store clerk, or a retired professor, you are probably seeking information about COVID-19 and how best to take care of yourself and others. However, your age, level of education, and other factors may influence how you seek information and how likely you are to benefit from it.

IES researchers Dr. Taka Yamashita (University of Maryland, Baltimore County) and Dr. Phyllis Cummins (Miami University) have been exploring the intersection of education, age, health, and problem solving (grant R305A170183). Program officer, Dr. Meredith Larson, spoke with them about their work and its implications during the current pandemic.

 

What has the COVID-19 epidemic brought to your attention?

For us, the COVID-19 epidemic has highlighted health information disparities by education and basic skills. In combination with other risk factors, like older age and lower income, some sub-populations are particularly vulnerable in the public health crisis. It is our sincere hope that our research underlines the urgent needs to promote education, literacy, and numeracy throughout the life course, not only to protect vulnerable populations but also to promote disaster preparedness and the well-being of our society.

What does your research suggest are important factors for predicting health information seeking behavior?

In general, we often seek health information only when we need it due to poor health. What is interesting is that, when we need health information, some sociodemographic characteristics and basic skills (for example, reading and numeracy skills) lead to differences. For example, in our analysis of PIAAC data, older age is related to lower usage of online health information sources while greater literacy skills are related to use of more online health information sources. Also, people with greater educational attainment and higher literacy skills are more likely to seek disease prevention and health promotion information from online sources and books. Overall, age, education, and literacy skills are closely and consistently related to what kind of health information sources people use and how much they do it.

How might these factors affect what people find or whether they benefit from it?

Age, education, and literacy skills differentiate how and where people seek health information. We believe that the Internet is one of the best information sources in terms of timeliness and amount of available information. However, some sub-populations, such as younger adults, those with higher education, and those with higher literacy skills, tend to take advantage of and benefit from the online health information more than their counterparts. As such, older adults, those with lower education, and those with lower literacy skills tend not only to miss online health information sources but also to under-utilize and/or mis-use health information.

What could information providers do to better reach different audiences?

Information providers could consider how and where they publish their information. For example, older adults, those with lower education, and those with lower literacy skills tend not to use online sources. Therefore, it is critical to provide accessible health information to offline sources such as newspapers and healthcare professionals (for example, face-to-face with a physician or nurse). Also, paying attention to the “at-risk” populations’ needs is important. There are simple things like using a large font, plain English and multiple languages (for example, English and Spanish) that can immediately enhance the quality of health communication.

Where can people go to learn more about this line of research?

More information about our studies on education and literacy can be found on our IES grant page. Also, one of our recent articles on literacy skills, language use, and online health information seeking among Hispanic adults in the United States has been featured in the CDC (Centers for Disease Control and Prevention) health communications science digest.


Written by Meredith Larson. This is the second in a series of blog posts that explores how researchers respond to various education-related issues and challenges. The first blog post can be found here.

Real-World Responses in Real Time: COVID-19 Information Needs to Consider Literacy Gaps

During the COVID-19 pandemic, when people have a heightened need for information, literacy barriers can be life threatening. In the United States, roughly 20 percent of adults read at the lowest level, with another 33 percent still below proficiency1. Thus, many may be struggling to understand written guidance on COVID-19.

IES researchers at the Center for the Study of Adult Literacy (R305C120001 and R305H180061) and their associated Adult Literacy Research Center at Georgia State University are working to address the needs of adults with literacy skill gaps. Dr. Meredith Larson spoke to Dr. Daphne Greenberg and Dr. Iris Feinberg about their work in this area.

What are your concerns for adults with low literacy during the pandemic?

Daphne GreenbergIris FeinbergWe have known for a long time that the high prevalence of adults with low basic skills has consequences for both the individual and society. These consequences are heightened during this pandemic. Many adults with low literacy have “essential” jobs and must continue to work. They often interact with many different people daily. So it is crucial that they understand COVID-19 precautions for their own health and because their ability to know and practice safe behaviors has a direct impact on disease transmission to others. To be quite frank, we are concerned about the health and safety of our learners and the health and safety of others.

In the United States, we receive an overwhelming amount information about COVID-19 daily. To make matters worse, there’s no uniform national guidance, some of the information is incorrect, and other information is conflicting. It is challenging for highly literate individuals to make sense of it all. For example: When can a COVID-positive person step out of quarantine? Can someone be re-infected? How many feet constitute safe distance? The list of questions goes on and on.

For someone with low literacy, it’s even more difficult to make sense of all the COVID-19 information. For example, people with low reading skills may not be able to read or understand all of the written information. Additionally, because much of the information is on the internet, adults with low digital skills and/or poor access to the internet have the added problem of not being able to find information that could possibly be helpful to them.

How are you trying to address their needs?

We’ve created a library with a large sample of materials written for 9th grade reading levels and below available on the ALRC website. These documents provide specific information on topics like how to stop the spread or what to do if someone in your home has COVID-19. We hope that providers who work with adults with low literacy skills—like adult educators, community organizers and healthcare providers—will use our library and find the high-interest/low-literacy materials. The library is also divided into “easier” and “harder” resources, so people can quickly find material at appropriate reading levels.

What could healthcare providers, the media, or others do to help?

We all must help those who may not know where to find information. Not everyone knows how or where to look for health information or whether the information they find is valid. Our analysis of PIAAC data found that people with low literacy rely more on TV and radio for information. Simple, short public service announcements that are action oriented would be great for anyone who relies on TV or radio but particularly for those who have low reading skills.

Also, we need to be better prepared for all kinds of emergencies by creating community-wide partnership plans among trusted sources for adults with low literacy like community organizations, healthcare providers, and adult education providers. In addition, we should be following plain language guidelines in all of our written and oral communications. Writing health information in plain language helps everyone and should not be an afterthought.

 


Written by Meredith Larson. This is the first in a series of blog posts that explores how researchers respond to various education-related issues and challenges.


About the PIAAC

The PIAAC is an international assessment for adults that assesses cognitive skills (literacy, numeracy, and problem solving) and contains data on educational background, workplace experiences and skills, and other items. For the purposes of this blog, the category of lowest levels is defined as Below Level 1 and Level 1 and below proficiency is Level 2. For more information about estimates of U.S. adult skills as measured by the PIAAC: https://nces.ed.gov/surveys/piaac/current_results.asp

 

New IES Grantee Focuses on Improving Adult Literacy

In her first IES grant, Dr. Elizabeth Tighe (Georgia State University) is taking expertise honed in both an NCER predoctoral fellowship and PIAAC methods training program to help further adult literacy research. Her earlier work includes developing assessments for adults with low literacy, leveraging statistical approaches to understand these adults’ abilities and difficulties, and using eye-tracking paradigms to explore their ability to self-monitor during reading. 

Program officer, Meredith Larson, interviewed Dr. Tighe about her previous work and new grant.

What is your general area of research, and why is it important?

I focus on adult struggling readers, which comprises roughly 36 million (1 in 6) adults in the U.S. Only a fraction of these adults enroll in adult education programs, which are plagued by insufficient funding, high teacher turnover rates, and a lack of research-based instructional practices and curricula. By better understanding these adults’ strengths and deficits and how best to measure their skills, I aim to inform and improve adult education programs.

What could people do with your research?

My research could directly inform how we help adults become stronger readers, and this can improve educational outcomes, such as GED attainment. I am working towards building better assessments for adult education practitioner and researcher use. My longer-term goal is to design a curriculum to teach morphology (e.g., prefixes and suffixes) and use this to improve adults’ vocabulary and reading comprehension.

What are you trying to learn through your new IES project?

For this grant, I’m using the Program for the International Assessment of Adult Competencies (PIAAC), a large-scale, international assessment of adult literacy, numeracy, and digital problem-solving skills, to create risk profiles of adults with low literacy skills. This sort of information could move us closer to being able to individualize instruction in adult education programs to match the needs of specific learners.

We will use PIAAC data to explore how demographic characteristics (age, race/ethnicity, educational background, employment status) and malleable factors (enjoyment of learning, frequency of computer use, reading and writing behaviors at home and at work) influence low literacy performance.

Further, we are examining whether risk factors differ by whether someone has a high school diploma and whether someone has participated in education or training recently. We will also explore whether reading components and literacy skills are predictive of low-skilled adults’ numeracy skills.

Our findings could have important implications for understanding risk factors and predictors of low literacy as well as low numeracy. As stated previously, 1 in 6 U.S. adults have low literacy skills and nearly 1 in 3 have low numeracy skills. For GED attainment, adults must demonstrate proficiency in both of these areas (along with science, social studies, and writing knowledge). It’s important to have targeted, individualized instruction for these adults because they may have time or resource barriers.

How did this particular research project arise?

I first learned about PIAAC at a summer institute. I was intrigued, in particular, because PIAAC is the first assessment of this size to include a reading component supplement for lower-skilled adults.

I recently attended a 3-day NCER/ETS PIAAC training workshop, which allowed me to work with PIAAC data and network with others. This workshop influenced my decision to apply for an IES grant. I felt that a 2-year grant using extant data would be a great way to combine my interests regarding individual differences in adults’ component skills and get my feet wet with IES as a new investigator. I am excited to bridge my interests and grow as a researcher by learning and working alongside two experts (Drs. Yaacov Petscher and John Sabatini) in the larger reading and education field!