Project Activities
In a series of four studies, the researchers worked with 9th- through 12th-grade students to explore (1) how different types of background knowledge (such as simple facts or more in-depth understanding of a specific topic) are related to each other, (2) the extent to which different types of background knowledge are associated with reading comprehension, (3) whether there is a level of background knowledge below which students will have difficulty understanding what they read, and (4) if students with low or high background knowledge differ in how well they understand text that varies in the explicitness of connections between ideas.
Structured Abstract
Setting
This study took place in 6 urban, 5 suburban and 8 rural high schools throughout the United States.
Sample
Approximately 8,600 students in 9th through 12th grade from 19 schools located across all four major geographic regions of the United States (northeast, midwest, south, west) participated in the studies. For those providing gender information (N=7273), the majority of students were female (54 percent). For those providing race/ethnicity information (n=5054), the majority of students were White (38 percent) followed by Hispanic/Latino (28 percent), multi-racial (18 percent), Black or African American (5 percent), Asian or Asian American (5 percent), American Indian or Alaskan Native (3 percent), and Native Hawaiian or Other Pacific Islander (1 percent) with the remaining as other (2 percent). Across the 19 schools, 62 percent of students participated in free and reduced lunch programs.
Research design and methods
Researchers began this project by assembling assessments of background knowledge and reading comprehension across four content domains (science, history, American football, and cooking), including various subdomains in science (such as ecosystems, biology) and history (for example immigration, colonial America). Items were selected from released forms of the National Assessment of Educational Progress (NAEP), state content tests, the Study Aid and Reading Assistant (SARA), and a global comprehension assessment called the
Global, Integrated, Scenario-Based Assessments (GISA). In Study 1, over 5,000 students in 9th through 12th grade completed assessments of background knowledge to determine form psychometric characteristics and examine how different types of background knowledge (namely basic background knowledge vs. conceptual background knowledge) relate to each other. Study 2 involved over 2,900 students in 9th through 12th grade to examine the relationship between different types of background knowledge and different types of reading comprehension. Study 2 also explored whether reading comprehension in history, science, and non-academic subjects is facilitated by specific or general basic background knowledge (namely vocabulary, simple facts), and specific or general conceptual knowledge. Study 3, with over 3,500 students in 9th through 12th grade, examined whether there is a threshold of background knowledge below which students may have difficulty comprehending text. Study 4 involved over 600 students in 9th through 12th grade to explore whether students of varying background knowledge differ in reading comprehension performance when text cohesion is low or high.
Control condition
Due to the nature of the research design, there is no control condition.
Key measures
In this grant, the researchers adapted and designed various measures of background knowledge, reading comprehension, foundational reading skills, and interest and engagement measures for this research. Basic and conceptual background knowledge was measured by a combination of topical vocabulary items, and selected response and constructed response items. The GISA was used to measure deep comprehension while traditional style forms were developed to measure basic comprehension. The SARA was used to measure foundational reading skills.
Data analytic strategy
Study 1 used classical test theory and item response theory to refine background knowledge items. Additionally, researchers used factor analysis to examine the dimensionality of the background knowledge assessments. The research team also used hierarchical regression and linear mixed-effects models to explore the relationship between background knowledge and reading comprehension in Study 2. Researchers used broken line regression in Study 3 to identify whether there is a minimal amount of background knowledge necessary to understand or learn from novel texts (i.e., knowledge threshold hypothesis). Finally, Study 4 used a linear mixed-effects model to determine the impact of background knowledge text cohesion level (low or high) and reading ability on text comprehension.
Key outcomes
The main findings of this exploratory study are as follows:
People and institutions involved
IES program contact(s)
Products and publications
Book chapter
O'Reilly, T., Sabatini, J., & Wang, Z. (2018). Using Scenario-Based Assessments to Measure Deep Learning. In K. Millis, D. Long, J. Magliano, & K. Weimer (Eds.), Deep learning: Multi-disciplinary approaches (pp. 197-208). New York, NY: Routledge. Full text
Journal article, monograph, or newsletter
McCarthy, K., Guerrero, T., Kent, K., Allen, L., McNamara, D., Chao, S., Steinberg, J., O'Reilly, T., & Sabatini, J. (2018). Comprehension in a scenario-based assessment: Domain and topic-specific background knowledge. Discourse Processes, 55(5-6), 510-524. DOI: 10.1080/0163853X.2018.1460159 Full text
O'Reilly, T., Sabatini, J., & Wang, Z. (2019). What you don't know won't hurt you, unless you don't know you're wrong. Reading Psychology, 40 (7), 638-677, DOI: 10.1080/02702711.2019.1658668
O'Reilly, T., Wang, Z., & Sabatini, J. (2019). How much knowledge is too little? When knowledge becomes a barrier to comprehension. Psychological Science, 1-8. DOI: 10.1177/0956797619862276
Sabatini, J., Halderman, L., O'Reilly, T., & Weeks, J. (2016). Assessing comprehension in kindergarten through third grade. Topics in Language Disorders, 36(4): 334-355. DOI: 10.1097/TLD.0000000000000104 Full text
Nongovernment report, issue brief, or practice guide
Deane, P., O'Reilly, T., Chao, S, & Dreier, K. (2018). Writing Processes in Short Written Responses to Questions Probing Prior Knowledge. (Research Report No. RR-18-39)
Additional online resources and information
The ETS-published report on America's reading crisis includes a discussion on the role of background knowledge. The Principal Investigator also published an outreach article on how background knowledge may play a role in helping solve America's reading crisis.
Publicly available data
The data collected from this project is housed in an ETS data repository. Researchers interested in collaborating can contact the principal investigator to apply to use the data in this project for secondary analyses.
Related projects
Supplemental information
Co-Principal Investigators: Sabatini, John; McNamara, Danielle
- There are a variety of reliable ways to measure student background knowledge ranging from topical knowledge assessments to constructed response and process measures. While conceptual knowledge items are more difficult than basic knowledge items, together they represent a single construct rather than distinct factors.
- Domain knowledge that is relevant to the topic of the texts is a good predictor of whether students will learn information when given the opportunity.
- There is a metacognitive element of background knowledge; students who indicated they did not know various pieces of knowledge, comprehended and learned more than students who have incorrect knowledge before reading texts.
- There is evidence for the knowledge threshold hypothesis which states that students may need to have a minimum amount of background knowledge to understand text; below this level, deep comprehension may be harder to attain.
The researchers also leveraged over 3,500 additional high school students' data for secondary analyses.
Malleable Factor: In this exploratory study, the researchers examined the relationship between background knowledge and reading comprehension. Malleable factors included background knowledge breadth and depth. This grant provided foundational research that can be used to develop measures of students' knowledge, and preliminary methods to help identify students who are at risk of failing to comprehend due to insufficient knowledge and help untangle what types of knowledge are more important for different types of comprehension.
Questions about this project?
To answer additional questions about this project or provide feedback, please contact the program officer.
