IES Blog

Institute of Education Sciences

Building a Reading Comprehension Measure for Postsecondary Students

Assessments of both U.S. adults and 12th-grade students indicate that millions of learners may have significant reading skill gaps. Because these students may lack the fundamental reading and comprehension skills needed to thrive in college, postsecondary institutions need valid reading measures that accurately determine the source of student difficulties.

An IES-funded research team is developing and validating such a measure: Multiple-choice Online Causal Comprehension Assessment for Postsecondary Students (MOCCA-College). MOCCA-College aims to assess the reading comprehension abilities of postsecondary students and distinguish between common comprehension difficulties. This information could help students, faculty, and programs better determine who might need what type of additional reading instruction.

The current version of MOCCA-College is still being validated, but it already contains components that may interest postsecondary institutions, faculty, and students. For example, it suggests classroom interventions based on a student’s results and allows for different user roles, such as student, faculty member, or administrator. 

Results from pilot work indicate that MOCCA-College can reliably distinguish between postsecondary readers with strong comprehension skills and those who may need to build these skills. MOCCA-College uses both narrative and expository texts to determine student performance. The results indicate that both types of passages measure a single dimension of ability, though narrative passages may more easily and accurately discriminate between those who have good comprehension skills and those who do not.

This finding is in keeping with meta-analysis work that finds a similar pattern for narrative and expository items. Narrative passages appear to consistently measure inferential comprehension more accurately than expository passages for both younger and older readers. This holds even after matching texts for readability and demands on background knowledge.

As the researchers continue to validate MOCCA-College, we will continue to learn more about the needs of postsecondary readers, as well as how to identify and address these needs.

 


This research and articles referenced above are supported through NCER grant R305A180417: Multiple-choice Online Causal Comprehension Assessment for Postsecondary Students (MOCCA-College).

Dr. Meredith Larson, program officer for postsecondary and adult education, wrote this blog. Contact her at Meredith.Larson@ed.gov for additional information about MOCCA-College and postsecondary teaching and learning research.

 

World Braille Day: Research on Teaching Braille to Students with Visual Impairments

January 4 is World Braille Day, which aims to increase awareness of the importance of braille as a means of communication for those who are blind or with visual impairment. The date chosen honors the birthday of Louis Braille, who invented a reading and writing system – braille – consisting of raised dots that are read via touch. This system of reading and writing is an important component of education and literacy for many individuals. Recognizing this importance, Simon Fisher-Baum, Robert Englebretson, and Cay Holbrook were awarded a NCSER grant in 2019 to explore the knowledge, skills, and strategies teachers of students with visual impairments need to effectively teach braille reading and writing. We asked this team of researchers to answer a few questions about their work on teaching braille in recognition of World Braille Day.

What do we already know about the complexities surrounding learning braille for a person with visual impairment?

The ability to read and write braille is crucial for individuals who are blind, just as print literacy is crucial for individuals who are sighted. Braille literacy opens a host of opportunities for education, leisure, and employment. Learning to read and write braille depends on children having direct instruction from competent professionals who know braille and recognize its importance in facilitating literacy. Most children who learn braille do so under the instruction of a Teacher of Students with Visual Impairments (TVI) with support from their classroom teachers who generally are only familiar with print. One major challenge for children learning braille is having sufficient access to a TVI. A second challenge involves the differences between print and braille and the different perspectives required of a typically sighted TVI and the children with visual impairment. The sighted TVI has learned braille as a 'code' (and thinks about transliterating it to their much stronger knowledge of print) whereas children are learning braille as their primary system of reading and writing. It is this potential mismatch that we are focusing on for our project. We seek to understand the perceptual and cognitive underpinnings of braille as a writing system for its readers in contrast with the print-based 'code' perspective that TVIs often implicitly and unconsciously bring to their teaching.

Your project has the challenge of researching a low-incidence population. Describe how you are able to find your sample.

There are no current, reliable demographics of the number of individuals who read braille in the United States. But the fact remains that even in a large city like Houston, where two of the co-PIs are based, it would be a real challenge to find a sufficient number of braille readers to conduct studies with any degree of statistical power. Because of this, we recruit participants at summer conventions of blindness organizations where there are large numbers of braille readers present. At least we hope to do this again once people can gather safely. Meanwhile, we are developing experiments involving adult braille readers submitting braille writing samples online and, thanks to the support of the Braille Institute of America, we are analyzing spelling tests and writing samples from over 1000 braille-learning children from the U.S. and Canada who participated in a literacy-focused contest called the Braille Challenge. In addition, we have access to teachers of students with visual impairments who read and write braille through the Braille Institute and professional conferences as well as strong contacts of researchers involved in this grant.

Tell us a little more about the Braille Challenge.

The Braille Challenge is an annual contest for braille-reading children in grades 1-12 in the U.S. and Canada that celebrates braille literacy and the academic use of braille. Since 2003, the Braille Institute of America has sponsored this event. You can think of the Braille Challenge a bit like the Scripps National Spelling Bee for kids who read braille, with sub-contests in areas such as spelling, writing braille from dictation, reading comprehension, proofreading, and analyzing tactile charts and graphs. The written materials that students produce from these contests are a treasure trove of comparative data. They enable us to analyze the error patterns in the same words and sentences produced by a large number of students, track the development and error patterns in the same students over the years, and ultimately associate student outcomes with the specific attitudes, knowledge, and skills their TVIs (who attend the contest with their students) bring to the teaching of braille.

Your project is using some innovative data collection approaches, such as finger and eye tracking studies. What do you hope to learn from this part of your project that can be better understood by these data collection approaches? 

Eye tracking is, of course, central in the reading sciences for understanding key perceptual, cognitive, and linguistic processing aspects of reading standard print. There has been little work to address those same types of questions with braille readers using finger-tracking technology. Our finger-tracking experiments will help us compare the proficiency of adult braille readers with the ways in which braille is being taught. In addition, one area that has never been explored is the underpinnings of how TVIs read braille. Typically, sighted TVIs read braille by eye (not by touch), and we would like to understand how reading braille by eye is similar to or different from how these same individuals read print by eye, and in turn, how TVIs reading braille by sight is similar to and different from the typical way blind readers read braille by touch.

What impact do you hope your project will have on how TVIs are trained and how they teach braille to students?

We hope that by understanding how braille is conceptualized and read differently by TVIs, proficient braille-reading adults who are blind, and children learning to read and write braille, our project will ultimately lead to evidence-based interventions for both TVIs and learners. This may include improved curricula for university TVI personnel preparation programs and improved materials designed for children learning braille that leverage their unique perspectives as braille readers.

Tell us about your research team and the diversity of experiences among team members with braille.

Photo of Cay Holbrook, Simon Fischer-Baum, Robert Englebretson
Clockwise from top left: Cay Holbrook, Simon Fischer-Baum, Robert Englebretson

The three research team members complement each other in areas of expertise, as well as in experiences with braille. Robert Englebretson is currently chair of the Linguistics Department at Rice University. He teaches a course on braille from the perspective of cognitive science and linguistics research. He has been recognized internationally for his work updating and publishing the braille version of the International Phonetic Alphabet, which enables access to careers in the language sciences for those who are blind or visually impaired and has served as co-chair of the research committee of the Braille Authority of North America. He also brings to this project his perspective as a life-long braille reader and his lived experience of the importance of braille literacy.

Simon Fischer-Baum is an Associate Professor of Psychological Sciences at Rice University. He comes to this project as a cognitive scientist who focuses on understanding literacy, using a wide variety of methods, from the careful analyses of the errors people make when reading and writing to analysis of the patterns of brain activity generated when we read and the study of individuals who have lost the ability to read or write following stroke. He learned about braille as a part of this current collaboration and applies his skillset as a cognitive scientist of language to figuring out the mental representations and processes that underlie how braille is read and written.

Cay Holbrook is a professor at the University of British Columbia in Vancouver, BC, Canada. She learned braille during her undergraduate program as part of an initial teaching credential. She began working as a teacher of students with visual impairments in Rock Hill, South Carolina and has also worked directly with students in K-12 in parts of Georgia and Florida. Her commitment to direct, ongoing, and consistent instruction by qualified teachers has guided much of her work. Her research and scholarship have included the publication of more than 12 co-authored or co-edited textbooks as well as numerous peer-reviewed articles and book chapters. She holds a PhD in special education from Florida State University. She has prepared teachers of students with visual impairments in Canada and the U.S. and was a member of the original advisory committee for the Braille Challenge.

What other research do you think is needed in the area of learning braille? What are your future plans to continue research in this area?

There is still much to be learned about how braille is read and written, and there are many lines of inquiry in braille literacy that would benefit greatly from a multidisciplinary approach to research like we are taking here. After we complete this project, our next goal would be to develop and test interventions that bring the perspectives of TVIs closer to the learning challenges their students are facing. But there is also the opportunity for new lines of research. One key question is how braille is learned by people who become blind later in life, including school-age children and older adults. There is already some evidence that these readers approach reading by touch differently than individuals who have only learned braille, but more research is needed to explore how those who become blind after learning to read print approach learning braille and what kinds of instructional strategies would best support their literacy acquisition. It is also worth exploring how different service delivery models – that is, what role the TVI plays in the student’s education plan – impact how the student learns to read and write. Finally, we know little about whether learning differences that lead to dyslexia and dysgraphia in the print reading population also occur in the braille reading population. To our knowledge, these kinds of developmental differences have never been explored within the population of braille reading children, but if they do occur, it seems like additional interventions would be needed with these students to help them acquire literacy.

 

Recent Report Identifies Possible Categories of Adult Struggling Readers (and How to Help Them)

Nearly one in five U.S. adults aged 16 and over may struggle with basic literacy. These adults may struggle with any of the core components of reading, such as decoding, vocabulary, and comprehension. They may struggle for many different reasons—English is not their first language, possible cognitive declines from aging, or a lack of formal education. To identify the right instructional tools and curricula, we need to understand the varying needs of this heterogeneous group of adult struggling readers and design appropriate solutions.

In a recent report, IES-funded researchers conducted a latent class analysis of 542 adults (age 16- to 71-years old) enrolled in adult education programs whose reading scores indicate a reading level between the 3rd- and 8th-grade level. The analysis identified four possible subgroup categories of adult struggling readers based on their performance on lower-level competencies (phonological awareness, decoding, vocabulary) and higher-level competencies (comprehension, inferencing, background knowledge):

 

  • Globally Impaired Readers: adults who show difficulties in all competencies
  • Globally Better Readers: adults who are relatively strong in all competencies
  • Weak Decoders: readers who are relatively weaker in lower-level competencies but strong in higher-level competencies
  • Weak Language Comprehenders: readers who are strong in lower-level competencies but relatively weaker in higher-level competencies

 

On average, Weak Decoders were older than other categories, though Globally Impaired Readers were on average older than Globally Better Readers or Weak Language Comprehenders. Globally Better Readers and Weak Decoders included a larger proportion of native English speakers than the other two categories. Thus, both age and English proficiency may predict the pattern of strengths and weaknesses. However, having a high school diploma did not predict performance patterns.

Although Globally Better Readers tended to perform better on reading assessment than other categories, even this group of readers performed at the 6th-grade level on average. Thus, all groups of readers would benefit from additional instruction. The researchers suggest different approaches for addressing the needs of learners in the different categories. For example, Weak Language Comprehenders may benefit from technology-based solutions that help build their oral language competencies, whereas Globally Impaired Readers and Weak Decoders may benefit from direct instruction on decoding skills.

 


This research was conducted as part of the Center for the Study of Adult Literacy (CSAL): Developing Instructional Approaches Suited to the Cognitive and Motivational Needs for Struggling Adults funded in 2012 through NCER.

The abstract for the publication discussed above is available on ERIC; Identifying Profiles of Struggling Adult Readers: Relative Strengths and Weaknesses in Lower-Level and Higher-Level Competencies (Talwar, Amani; Greenberg, Daphne; Li, Hongli).

Dr. Meredith Larson, program officer for postsecondary and adult education, wrote this blog. Contact her at Meredith.Larson@ed.gov for additional information about CSAL and adult education research.

Research on Adult Literacy: A History of Investment in American Adults

Reading is fundamental, but it is also difficult to master, taking thousands of hours of instruction and practice. Roughly 52 percent of U.S. adults over the age of 16 may struggle with everyday literacy tasks. Of these adults, approximately 20 percent may perform at very low levels of literacy. For adults who are still mastering of this skill, the task can seem overwhelming.

Luckily, IES-funded researchers have been working towards solutions for adults with low basic reading skills and are creating and refining assessments, curricula, and software. These innovations aim to help adult learners, the instructors and tutors who work with them, and the programs that support them.

As part of our commemoration of National Adult Education and Family Literacy Week (September 20-26, 2020), we would like to recognize the history of adult literacy research at IES and its National Center for Education Research.  

Since 2004, IES-funded researchers have been developing assessments to help identify the needs of adults struggling with literacy and working on solutions to build adult literacy skills. This work fed into the measurement component of IES’s Reading for Understanding Initiative in 2010 and later returned back to addressing adult basic literacy measurement in 2016.

In 2012, IES funded the Center for the Study of Adult Literacy (CSAL), which developed a curriculum and technology for adults reading between the 3rd- and 8th-grade levels. CSAL demonstrates how adult literacy research benefits by integrating research conducted with students with disabilities and those in K-12 and postsecondary settings. In fact, the researchers pulled upon findings from eight prior IES grants funded by NCER and NCSER.

Our researchers are also developing a clearer picture of the adults who fall into the broad category of those with low literacy. They are leveraging the PIAAC data set to conduct exploratory work that informs both our understanding of those at the very low ends of literacy and also of whether basic skills may predict success in postsecondary career and technical education programs.

In 2020, IES funded additional development research to help refine an interactive, online reading comprehension program, AutoTutor for Adult Reading Comprehension (AT-ARC). Another project will recruit and train postdoctoral fellows to cultivate the next generation of researchers who can continue to build a research base for improving adult literacy outcomes.

Although IES researchers are making great strides to build knowledge, the field needs more information, and adult learners deserve tools and innovations developed for their specific needs and goals. IES hopes to continue to support such work.

 


To learn more about IES-wide efforts to understand and improve adult learners’ outcomes, visit the Adult Basic Skills topic page. Contact Dr. Meredith Larson for more information about the research supported by NCER.

 

Back to School During COVID19: Developers and Researchers Continue to Respond to Support In-Class and Remote Teaching and Learning

Many programs across the Federal government, such as the ED/IES Small Business Innovation Research (SBIR) and the IES Research Grants programs, fund projects to develop and evaluate new forms of education technology and interventions that can be implemented to support instruction and learning at schools and for remote learning. More than 150 of these technologies were demoed in January 2020 at the ED Games Expo, a showcase for learning games and technologies developed with support from IES and more than 30 other Federal programs.

Since the global outbreak of COVID19 and the closure of schools across the United States and the world, a group of government-supported developers and researchers responded to provide resources to educators, students, and families to facilitate remote learning. More than 50 developers and researchers offered 88 learning games and technologies at no cost through the end of the school year for use in distance learning settings with internet access (see this blog for the list). In addition, many of the developers and researchers provided technical assistance directly to individual teachers to support implementation at a distance, and many created new materials and worked to refine and adapt their products to optimize usability and feasibility for fully remote use. More than a million students and thousands of educators used these learning technologies during the spring.

In April and May 2020, more than 70 developers and researchers partnered to produce and participate in a series of free day-long virtual events, which were called “unconferences.” The events featured presentations on innovative models and approaches to teaching and learning remotely and provided an in-depth look at the learning games and technologies created by the presenters. More than 25,000 educators attended these virtual events in real-time, hundreds asked questions and made comments through chats during the events, and many thousands more have accessed these videos after the events. See this blog for the list of archived videos.

A New Resource: Guides to Education Technologies that are Ready Now

As schools begin re-opening for the new school year, a group of 70 developers and researchers have collaborated to produce a new series of Guides to Education Technologies. The guides present information on government-supported education technology products that are ready now for in-class and remote learning. All the resources are web-based and can be used on either computers, tablets, or personal devices. The resources in the guides include a mix of no-cost products as well as ones that are fee-based.  

With awards from government programs, all of the resources were developed through an iterative process with feedback from teachers and students, and most were evaluated through small pilot studies to measure the promise of the technologies to support improvements in student learning and relevant educational outcomes. All the products were used and demonstrated to be feasible for use in remote settings in the spring after the onset of the pandemic.

The guides present resources appropriate for young children through postsecondary students in education and special education, for English learners, and for teachers in education and special education across a wide range of educational topics. Many of the technologies personalize learning by adjusting content to students as they go and present information to educators to inform instruction.

The Guides focus on the following areas and can be accessed below:

 

Stay tuned to the Inside IES Blog for more information and resources about the response to the COVID-19 in education.


Edward Metz (Edward.Metz@ed.gov) is a research scientist and the program manager for the Small Business Innovation Research Program at the US Department of Education’s Institute of Education Sciences.