Skip Navigation
REL Pacific

[Return to Ask A REL]

REL Pacific Ask A REL Response

Literacy
PDF icon

June 2019

Question

What does the research say about effective measures of adult literacy?

Response

Following an established REL Pacific research protocol, we conducted a web-based search for resources related to measures of adult literacy, with a focus on surveys and self-reporting questionnaires. Because of potential similarities in self-reporting methods, we also searched for survey methodology in the health literacy literature (see Methods section for search terms and resource selection criteria).

References are listed in alphabetical order, not necessarily in order of relevance. Descriptions of the resources are quoted directly from the publication abstracts. We have not evaluated the quality of references and the resources provided in this response. We offer them only for your reference. Also, our search included the most commonly used research resources, but they are not comprehensive and other relevant references and resources may exist.

Research References

Miller, C. D., Greenberg, D., Hendrick, R. C., & Nanda, A. (2017). Educational attainment: Limited implications for adult literacy learners. Journal of Research and Practice for Adult Literacy, Secondary, and Basic Education, 6(2), 21–36. Retrieved from https://eric.ed.gov/?id=EJ1150484.

From the abstract:
Social science research often uses educational qualification as a signifier for characteristics such as abilities, earnings potential, and civic participation in adulthood. This study focused on two types of adult literacy students who were native speakers of English and identified words at the 3rd to 5th grade levels but differed in one key demographic identifier. One group had dropped out prior to attending high school while the other group had graduated from high school. Differences between the two groups were examined in terms of their underlying reading skills, employment, voter registration status, reading pleasure, self-perception of reading ability, print reading practices, and technology based reading practices. Results showed very few statistically significant differences between the two groups. These findings suggest that for individuals who have difficulty reading, higher educational qualification levels do not necessarily imply differences in other characteristics. Implications for further research are discussed.

Morris, N. S., MacLean, C. D., Chew, L. D., & Littenberg, B. (2006). The Single Item Literacy Screener: Evaluation of a brief instrument to identify limited reading ability. BMC Family Practice, 7(21), 1–7. Retrieved from https://bmcfampract.biomedcentral.com/articles/10.1186/1471-2296-7-21.

From the abstract:
Reading skills are important for accessing health information, using health care services, managing one's health and achieving desirable health outcomes. Our objective was to assess the diagnostic accuracy of the Single Item Literacy Screener (SILS) to identify limited reading ability, one component of health literacy, as measured by the S-TOFHLA. Cross-sectional interview with 999 adults with diabetes residing in Vermont and bordering states. Participants were randomly recruited from Primary Care practices in the Vermont Diabetes Information System June 2003–December 2004. The main outcome was limited reading ability. The primary predictor was the SILS. Of the 999 persons screened, 169 (17%) had limited reading ability. The sensitivity of the SILS in detecting limited reading ability was 54% [95% CI: 47%, 61%] and the specificity was 83% [95% CI: 81%, 86%] with an area under the Receiver Operating Characteristics Curve (ROC) of 0.73 [95% CI: 0.69, 0.78]. Seven hundred seventy (77%) screened negative on the SILS and 692 of these subjects had adequate reading skills (negative predictive value = 0.90 [95% CI: 0.88, 0.92]). Of the 229 who scored positive on the SILS, 92 had limited reading ability (positive predictive value = 0.4 [95% CI: 0.34, 0.47]). The SILS is a simple instrument designed to identify patients with limited reading ability who need help reading health-related materials. The SILS performs moderately well at ruling out limited reading ability in adults and allows providers to target additional assessment of health literacy skills to those most in need. Further study of the use of the SILS in clinical settings and with more diverse populations is warranted.

Olson, K., Smyth, J. D., Wang, Y., & Pearson, J. E. (2011). The self-assessed literacy index: Reliability and validity. Sociology Department, Faculty Publications, 155. Retrieved from https://digitalcommons.unl.edu/ sociologyfacpub/155/.

From the abstract:
Literacy is associated with many outcomes of research interest as well as with respondents' ability to even participate in surveys, yet very few surveys attempt to measure it because doing so is often complex, requiring extensive tests. The central goal of this paper is to develop a parsimonious measure of respondents' reading ability that does not require a complex literacy test. We use data from the 2003 National Assessment of Adult Literacy to identify correlates of reading ability to form a literacy index. These correlates include self-assessments of one's ability to understand, read and write English, and literacy practices at home. Our literacy index reliably discerns literacy test scores above educational attainment, and the index shows high internal consistency (coefficient alpha = 0.78) and validity. The paper concludes with implications of these findings for survey research practitioners and suggestions for future research.

UNESCO. (2008). Using a literacy module in household surveys: A guidebook. Bangkok, Thailand: UNESCO Bangkok. Retrieved from https://unesdoc.unesco.org/ark:/48223/pf0000161938.

From the introduction:
Data on literacy status have long been collected as part of household and other surveys. conducted by governments and various development agencies nationally, regionally and internationally. These include censuses, special household surveys and special assessment studies. Such data feed into policy planning, programmed development and implementation of specific and need-based interventions to achieve a given set of goals within a given period. Thus, an important role of such data inputs would be their ability to provide a body of evidence based on which informed decisions could be taken. It must be noted, however, that such a body of evidence is acceptable only if it is robust, relevant, comparable and interpretable. These data are important not only for planning the achievement of targets set nationally, but also for planning the ones set internationally under the MDGs. To be robust and reliable, the data need to capture the local contexts that may act as barriers to development. Such barriers and their determinants need to be captured by providing for disaggregation of details at the local levels. For doing this effectively, literacy indicators need to be correlated with basic characteristics of the population surveyed. Such basic characteristics could be obtained from the main household survey chosen for inserting the module. Information from secondary sources could also help the process.

Wagner, D. A. (2008). Adult literacy: Monitoring and evaluation for practice and policy. International Review of Education, 54(5–6), 651–672. Retrieved from https://eric.ed.gov/?id=EJ815885.

From the abstract:
With the advent of the UN Literacy Decade launched in 2003, there is increased emphasis on monitoring and evaluation of adult literacy around the globe. The present paper presents an overview of the main approaches that have been taken to measure adult literacy within and across countries. A particular focus of the present review is to compare and contrast three models of literacy assessment: (1) the “traditional” model based largely on census enumeration, which has been used over recent decades to collect national and regional rates of illiteracy; (2) the large-scale survey techniques employed with the International Adult Literacy Survey and similar models; and (3) an intermediate type of assessment that borrows from the previous two models, and attempts to “tailor” the size and complexity of the survey methodology to the policy questions needing answers (called the SQC model). The present paper suggests that there is no single model or set of methods that are universally appropriate to monitoring and measuring in adult literacy around the world, but that blending both programmatic and comparative models through the SQC approach may bring greater involvement in, and insight into, adult literacy evaluations.

Welcome, S. E. & Meza, R. A. (2019). Dimensions of the Adult Reading History Questionnaire and their relationships with reading ability. Reading and Writing: An Interdisciplinary Journal, 32(5), 1295–1317. Retrieved from https://eric.ed.gov/?id=EJ1212653.

From the abstract:
The Adult Reading History Questionnaire (ARHQ; Lefly & Pennington, 2000) is a widely used measure of self-reported reading difficulties. We explored the factor structure underlying this questionnaire using both exploratory and confirmatory factor analysis. A six-factor solution emerged, with childhood reading ability, current reading attitude, spelling skill, reversal, print media use, and memory factors. We created subscales reflecting these factors and explored relationships between subscale scores and different reading abilities (word reading, nonword reading, and passage comprehension). The total ARHQ score, as well as scores on the childhood reading ability, current reading attitude, and spelling skill subscales were significantly associated with word reading, nonword reading, and passage comprehension. The spelling skills subscale showed a stronger relationship with nonword reading than other reading skills, while scores on the current reading attitude subscale showed a weaker relationship between nonword Reading skill and other measures of reading. Overall, the results suggest that reading history is multidimensional and should be regarded as such in future research. Importantly, this self-report measure of lifetime reading experience was associated with adult reading skill in a population of university students showing typical variation in reading skill, with different aspects of the scale relating to different reading subskills.

Additional Organizations to Consult

Literacy and Information Communication System (LINCS): https://lincs.ed.gov/

From the website:
The Literacy Information and Communication System (LINCS) is a national leadership initiative of the U.S. Department of Education, Office of Career, Technical, and Adult Education (OCTAE) to expand evidence-based practice in the field of adult education. LINCS demonstrates OCTAE's commitment to delivering high-quality, on-demand educational opportunities to practitioners of adult education, so those practitioners can help adult learners successfully transition to postsecondary education and 21st century jobs.

Methods

The following keywords and search strings were used to search the reference databases and other sources:

  • “measuring adult literacy”
  • “adult literacy” and “survey”
  • “adult literacy” and “measurement”
  • “adult literacy”
  • “data source” and “adult literacy”
  • “self-reporting” and “adult literacy”
  • “effective measures of adult literacy”
  • “adult literacy surveys”

Databases and Resources

We searched ERIC, a free online library of over 1.6 million citations of education research sponsored by the Institute of Education Sciences, for relevant resources. Additionally, we searched the academic databases Google Scholar and JSTOR.

Reference Search and Selection Criteria

REL Pacific searched ERIC and other academic journal databases for studies that were published in English-language peer-reviewed research journals within the last 20 years. REL Pacific prioritized documents that are accessible online and publicly available, and prioritized references that provide practical information based on peer-reviewed research in both social science and health literature bases for the community leaders who requested this Ask A REL. Sources included in this document were last accessed in June 2019. Methodological priorities and considerations were given in the review to the following:

  • study types—randomized control trials, quasi experiments, surveys, descriptive data analyses, and literature reviews;
  • target population and sample size, where applicable; and
  • limitations and generalizability of the findings and conclusions.


    This memorandum is one in a series of quick-turnaround responses to specific questions posed by educational stakeholders in the Pacific Region (American Samoa, the Commonwealth of the Northern Mariana Islands, the Federated States of Micronesia, Guam, Hawai'i, the Republic of the Marshall Islands, and the Republic of Palau), which is served by the Regional Educational Laboratory (REL Pacific) at McREL International. This memorandum was prepared by REL Pacific under a contract with the U.S. Department of Education's Institute of Education Sciences (IES), Contract ED-IES-17-C-0010, administered by McREL International. Its content does not necessarily reflect the views or policies of IES or the U.S. Department of Education, nor does mention of trade names, commercial products, or organizations imply endorsement by the U.S. Government.