|Title:||What Types of Knowledge Matters for What Types of Comprehension? Exploring the Role of Background Knowledge on Students' Ability to Learn from Multiple Texts|
|Principal Investigator:||O'Reilly, Tenaha||Awardee:||Educational Testing Service (ETS)|
|Program:||Literacy [Program Details]|
|Award Period:||4 years (9/1/2015 – 8/31/2019)||Award Amount:||$1,600,000|
Co-Principal Investigators: Sabatini, John; McNamara, Danielle
Purpose: The purpose of this project was to explore the relationship between high school students' background knowledge and their reading comprehension. Research has shown that students with more background knowledge (namely the previously learned knowledge that may help students understand what they read by providing extra information that is not included in text) are better able to comprehend what they read than those students with little background knowledge. Although researchers know that background knowledge matters for reading, prior to this grant, there were still questions about how much background knowledge is necessary to understand text and whether some types of knowledge are more important for different types of comprehension. This project used assessment items developed through other IES grants (Assessing Reading for Understanding: A Theory-based, Developmental Approach, Measuring the Development of Vocabulary and Word Learning to Support Content Area Reading and Learning, and Developing Reading Comprehension Assessments Targeting Struggling Readers) in a series of studies with 9th- to 12th-grade students to examine what types of and how much background knowledge is related to reading comprehension. The results indicate that background knowledge can limit how much students understand text and that some types of knowledge are more important for certain types of comprehension. Educators should embrace the level of students' knowledge as a potential mechanism for understanding students' reading comprehension.
Project Activities: In a series of four studies, the researchers worked with 9th- through 12th-grade students to explore (1) how different types of background knowledge (such as simple facts or more in-depth understanding of a specific topic) are related to each other, (2) the extent to which different types of background knowledge are associated with reading comprehension, (3) whether there is a level of background knowledge below which students will have difficulty understanding what they read, and (4) if students with low or high background knowledge differ in how well they understand text that varies in the explicitness of connections between ideas.
Key Outcomes: The main findings of this exploratory study are as follows:
Setting: This study took place in 6 urban, 5 suburban and 8 rural high schools throughout the United States.
Sample: Approximately 8,600 students in 9th through 12th grade from 19 schools located across all four major geographic regions of the United States (northeast, midwest, south, west) participated in the studies. For those providing gender information (N=7273), the majority of students were female (54 percent). For those providing race/ethnicity information (n=5054), the majority of students were White (38 percent) followed by Hispanic/Latino (28 percent), multi-racial (18 percent), Black or African American (5 percent), Asian or Asian American (5 percent), American Indian or Alaskan Native (3 percent), and Native Hawaiian or Other Pacific Islander (1 percent) with the remaining as other (2 percent). Across the 19 schools, 62 percent of students participated in free and reduced lunch programs.
The researchers also leveraged over 3,500 additional high school students' data for secondary analyses.
Malleable Factor: In this exploratory study, the researchers examined the relationship between background knowledge and reading comprehension. Malleable factors included background knowledge breadth and depth. This grant provided foundational research that can be used to develop measures of students' knowledge, and preliminary methods to help identify students who are at risk of failing to comprehend due to insufficient knowledge and help untangle what types of knowledge are more important for different types of comprehension.
Research Design and Methods: Researchers began this project by assembling assessments of background knowledge and reading comprehension across four content domains (science, history, American football, and cooking), including various subdomains in science (such as ecosystems, biology) and history (for example immigration, colonial America). Items were selected from released forms of the National Assessment of Educational Progress (NAEP), state content tests, the Study Aid and Reading Assistant (SARA), and a global comprehension assessment called the Global, Integrated, Scenario-Based Assessments (GISA). In Study 1, over 5,000 students in 9th through 12th grade completed assessments of background knowledge to determine form psychometric characteristics and examine how different types of background knowledge (namely basic background knowledge vs. conceptual background knowledge) relate to each other. Study 2 involved over 2,900 students in 9th through 12th grade to examine the relationship between different types of background knowledge and different types of reading comprehension. Study 2 also explored whether reading comprehension in history, science, and non-academic subjects is facilitated by specific or general basic background knowledge (namely vocabulary, simple facts), and specific or general conceptual knowledge. Study 3, with over 3,500 students in 9th through 12th grade, examined whether there is a threshold of background knowledge below which students may have difficulty comprehending text. Study 4 involved over 600 students in 9th through 12th grade to explore whether students of varying background knowledge differ in reading comprehension performance when text cohesion is low or high.
Control Condition: Due to the nature of the research design, there is no control condition.
Key Measures: In this grant, the researchers adapted and designed various measures of background knowledge, reading comprehension, foundational reading skills, and interest and engagement measures for this research. Basic and conceptual background knowledge was measured by a combination of topical vocabulary items, and selected response and constructed response items. The GISA was used to measure deep comprehension while traditional style forms were developed to measure basic comprehension. The SARA was used to measure foundational reading skills.
Data Analytic Strategy: Study 1 used classical test theory and item response theory to refine background knowledge items. Additionally, researchers used factor analysis to examine the dimensionality of the background knowledge assessments. The research team also used hierarchical regression and linear mixed-effects models to explore the relationship between background knowledge and reading comprehension in Study 2. Researchers used broken line regression in Study 3 to identify whether there is a minimal amount of background knowledge necessary to understand or learn from novel texts (i.e., knowledge threshold hypothesis). Finally, Study 4 used a linear mixed-effects model to determine the impact of background knowledge text cohesion level (low or high) and reading ability on text comprehension.
Related IES Projects: Developing Reading Comprehension Assessments Targeting Struggling Readers (R305G040065), Measuring the Development of Vocabulary and Word Learning to Support Content Area Reading and Learning (R305A080647), and Assessing Reading for Understanding: A Theory-based, Developmental Approach (R305F100005)
Publications and Products
O'Reilly, T., Sabatini, J., & Wang, Z. (2018). Using Scenario-Based Assessments to Measure Deep Learning. In K. Millis, D. Long, J. Magliano, & K. Weimer (Eds.), Deep learning: Multi-disciplinary approaches (pp. 197–208). New York, NY: Routledge. Full text
Journal article, monograph, or newsletter
McCarthy, K., Guerrero, T., Kent, K., Allen, L., McNamara, D., Chao, S., Steinberg, J., O'Reilly, T., & Sabatini, J. (2018). Comprehension in a scenario-based assessment: Domain and topic-specific background knowledge. Discourse Processes, 55(5-6), 510-524. DOI: 10.1080/0163853X.2018.1460159 Full text
O'Reilly, T., Sabatini, J., & Wang, Z. (2019). What you don't know won't hurt you, unless you don't know you're wrong. Reading Psychology, 40 (7), 638-677, DOI: 10.1080/02702711.2019.1658668
O'Reilly, T., Wang, Z., & Sabatini, J. (2019). How much knowledge is too little? When knowledge becomes a barrier to comprehension. Psychological Science, 1-8. DOI: 10.1177/0956797619862276
Sabatini, J., Halderman, L., O'Reilly, T., & Weeks, J. (2016). Assessing comprehension in kindergarten through third grade. Topics in Language Disorders, 36(4): 334–355. DOI: 10.1097/TLD.0000000000000104 Full text
Nongovernment report, issue brief, or practice guide
Deane, P., O'Reilly, T., Chao, S, & Dreier, K. (2018). Writing Processes in Short Written Responses to Questions Probing Prior Knowledge. (Research Report No. RR-18-39)
Additional online resources and information
The ETS-published report on America's reading crisis includes a discussion on the role of background knowledge. The Principal Investigator also published an outreach article on how background knowledge may play a role in helping solve America's reading crisis.
Publicly available data
The data collected from this project is housed in an ETS data repository. Researchers interested in collaborating can contact the principal investigator to apply to use the data in this project for secondary analyses.